summaryrefslogtreecommitdiff
path: root/Documentation/kbuild
AgeCommit message (Collapse)Author
2020-04-23Documentation: kbuild: fix the section title formatMasahiro Yamada
Make it consistent with the other sections. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-04-09kbuild: support LLVM=1 to switch the default tools to Clang/LLVMMasahiro Yamada
As Documentation/kbuild/llvm.rst implies, building the kernel with a full set of LLVM tools gets very verbose and unwieldy. Provide a single switch LLVM=1 to use Clang and LLVM tools instead of GCC and Binutils. You can pass it from the command line or as an environment variable. Please note LLVM=1 does not turn on the integrated assembler. You need to pass LLVM_IAS=1 to use it. When the upstream kernel is ready for the integrated assembler, I think we can make it default. We discussed what we need, and we agreed to go with a simple boolean flag that switches both target and host tools: https://lkml.org/lkml/2020/3/28/494 https://lkml.org/lkml/2020/4/3/43 Some items discussed, but not adopted: - LLVM_DIR When multiple versions of LLVM are installed, I just thought supporting LLVM_DIR=/path/to/my/llvm/bin/ might be useful. CC = $(LLVM_DIR)clang LD = $(LLVM_DIR)ld.lld ... However, we can handle this by modifying PATH. So, we decided to not do this. - LLVM_SUFFIX Some distributions (e.g. Debian) package specific versions of LLVM with naming conventions that use the version as a suffix. CC = clang$(LLVM_SUFFIX) LD = ld.lld(LLVM_SUFFIX) ... will allow a user to pass LLVM_SUFFIX=-11 to use clang-11 etc., but the suffixed versions in /usr/bin/ are symlinks to binaries in /usr/lib/llvm-#/bin/, so this can also be handled by PATH. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: Nathan Chancellor <natechancellor@gmail.com> # build Tested-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-09kbuild: replace AS=clang with LLVM_IAS=1Masahiro Yamada
The 'AS' variable is unused for building the kernel. Only the remaining usage is to turn on the integrated assembler. A boolean flag is a better fit for this purpose. AS=clang was added for experts. So, I replaced it with LLVM_IAS=1, breaking the backward compatibility. Suggested-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
2020-04-09Documentation/llvm: fix the name of llvm-sizeFangrui Song
The tool is called llvm-size, not llvm-objsize. Fixes: fcf1b6a35c16 ("Documentation/llvm: add documentation on building w/ Clang/LLVM") Signed-off-by: Fangrui Song <maskray@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-03-31Merge tag 'kbuild-v5.7' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: "Build system: - add CONFIG_UNUSED_KSYMS_WHITELIST, which will be useful to define a fixed set of export symbols for Generic Kernel Image (GKI) - allow to run 'make dt_binding_check' without .config - use full schema for checking DT examples in *.yaml files - make modpost fail for missing MODULE_IMPORT_NS(), which makes more sense because we know the produced modules are never loadable - Remove unused 'AS' variable Kconfig: - sanitize DEFCONFIG_LIST, and remove ARCH_DEFCONFIG from Kconfig files - relax the 'imply' behavior so that symbols implied by 'y' can become 'm' - make 'imply' obey 'depends on' in order to make 'imply' really weak Misc: - add documentation on building the kernel with Clang/LLVM - revive __HAVE_ARCH_STRLEN for 32bit sparc to use optimized strlen() - fix warning from deb-pkg builds when CONFIG_DEBUG_INFO=n - various script and Makefile cleanups" * tag 'kbuild-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (34 commits) Makefile: Update kselftest help information kbuild: deb-pkg: fix warning when CONFIG_DEBUG_INFO is unset kbuild: add outputmakefile to no-dot-config-targets kbuild: remove AS variable net: wan: wanxl: refactor the firmware rebuild rule net: wan: wanxl: use $(M68KCC) instead of $(M68KAS) for rebuilding firmware net: wan: wanxl: use allow to pass CROSS_COMPILE_M68k for rebuilding firmware kbuild: add comment about grouped target kbuild: add -Wall to KBUILD_HOSTCXXFLAGS kconfig: remove unused variable in qconf.cc sparc: revive __HAVE_ARCH_STRLEN for 32bit sparc kbuild: refactor Makefile.dtbinst more kbuild: compute the dtbs_install destination more simply Makefile: disallow data races on gcc-10 as well kconfig: make 'imply' obey the direct dependency kconfig: allow symbols implied by y to become m net: drop_monitor: use IS_REACHABLE() to guard net_dm_hw_report() modpost: return error if module is missing ns imports and MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=n modpost: rework and consolidate logging interface kbuild: allow to run dt_binding_check without kernel configuration ...
2020-03-30Merge tag 'docs-5.7' of git://git.lwn.net/linuxLinus Torvalds
Pull documentation updates from Jonathan Corbet: "This has been a busy cycle for documentation work. Highlights include: - Lots of RST conversion work by Mauro, Daniel ALmeida, and others. Maybe someday we'll get to the end of this stuff...maybe... - Some organizational work to bring some order to the core-api manual. - Various new docs and additions to the existing documentation. - Typo fixes, warning fixes, ..." * tag 'docs-5.7' of git://git.lwn.net/linux: (123 commits) Documentation: x86: exception-tables: document CONFIG_BUILDTIME_TABLE_SORT MAINTAINERS: adjust to filesystem doc ReST conversion docs: deprecated.rst: Add BUG()-family doc: zh_CN: add translation for virtiofs doc: zh_CN: index files in filesystems subdirectory docs: locking: Drop :c:func: throughout docs: locking: Add 'need' to hardirq section docs: conf.py: avoid thousands of duplicate label warning on Sphinx docs: prevent warnings due to autosectionlabel docs: fix reference to core-api/namespaces.rst docs: fix pointers to io-mapping.rst and io_ordering.rst files Documentation: Better document the softlockup_panic sysctl docs: hw-vuln: tsx_async_abort.rst: get rid of an unused ref docs: perf: imx-ddr.rst: get rid of a warning docs: filesystems: fuse.rst: supress a Sphinx warning docs: translations: it: avoid duplicate refs at programming-language.rst docs: driver.rst: supress two ReSt warnings docs: trace: events.rst: convert some new stuff to ReST format Documentation: Add io_ordering.rst to driver-api manual Documentation: Add io-mapping.rst to driver-api manual ...
2020-03-17modpost: move the namespace field in Module.symvers lastJessica Yu
In order to preserve backwards compatability with kmod tools, we have to move the namespace field in Module.symvers last, as the depmod -e -E option looks at the first three fields in Module.symvers to check symbol versions (and it's expected they stay in the original order of crc, symbol, module). In addition, update an ancient comment above read_dump() in modpost that suggested that the export type field in Module.symvers was optional. I suspect that there were historical reasons behind that comment that are no longer accurate. We have been unconditionally printing the export type since 2.6.18 (commit bd5cbcedf44), which is over a decade ago now. Fix up read_dump() to treat each field as non-optional. I suspect the original read_dump() code treated the export field as optional in order to support pre <= 2.6.18 Module.symvers (which did not have the export type field). Note that although symbol namespaces are optional, the field will not be omitted from Module.symvers if a symbol does not have a namespace. In this case, the field will simply be empty and the next delimiter or end of line will follow. Cc: stable@vger.kernel.org Fixes: cb9b55d21fe0 ("modpost: add support for symbol namespaces") Tested-by: Matthias Maennich <maennich@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Jessica Yu <jeyu@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-03-13kconfig: make 'imply' obey the direct dependencyMasahiro Yamada
The 'imply' statement may create unmet direct dependency when the implied symbol depends on m. [Test Code] config FOO tristate "foo" imply BAZ config BAZ tristate "baz" depends on BAR config BAR def_tristate m config MODULES def_bool y option modules If you set FOO=y, BAZ is also promoted to y, which results in the following .config file: CONFIG_FOO=y CONFIG_BAZ=y CONFIG_BAR=m CONFIG_MODULES=y This does not meet the dependency 'BAZ depends on BAR'. Unlike 'select', what is worse, Kconfig never shows the 'WARNING: unmet direct dependencies detected for ...' for this case. Because 'imply' is considered to be weaker than 'depends on', Kconfig should take the direct dependency into account. For clarification, describe this case in kconfig-language.rst too. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Pitre <nico@fluxnic.net> Tested-by: Geert Uytterhoeven <geert@linux-m68k.org>
2020-03-13kconfig: allow symbols implied by y to become mMasahiro Yamada
The 'imply' keyword restricts a symbol to y or n, excluding m when it is implied by y. This is the original behavior since commit 237e3ad0f195 ("Kconfig: Introduce the "imply" keyword"). However, the author of this feature, Nicolas Pitre, stated that the 'imply' keyword should not impose any restrictions. (https://lkml.org/lkml/2020/2/19/714) I agree, and want to get rid of this tricky behavior. Suggested-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Pitre <nico@fluxnic.net>
2020-03-13kbuild: doc: fix references to other documentsMasahiro Yamada
All the files in Documentation/kbuild/ were converted to reST. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-03-10docs: move gcc-plugins to the kbuild manualJonathan Corbet
Information about GCC plugins is relevant to kernel building, so move this document to the kbuild manual. Acked-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2020-03-03Documentation/llvm: add documentation on building w/ Clang/LLVMNick Desaulniers
Added to kbuild documentation. Provides more official info on building kernels with Clang and LLVM than our wiki. Suggested-by: Kees Cook <keescook@chromium.org> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-27kbuild: get rid of trailing slash from subdir- exampleMasahiro Yamada
obj-* needs a trailing slash for a directory, but subdir-* does not. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-27kbuild: remove wrong documentation about mandatory-yMasahiro Yamada
This sentence does not make sense in the section about mandatory-y. This seems to be a copy-paste mistake of commit fcc8487d477a ("uapi: export all headers under uapi directories"). The correct description would be "The convention is to list one mandatory-y per line ...". I just removed it instead of fixing it. If such information is needed, it could be commented in include/asm-generic/Kbuild and include/uapi/asm-generic/Kbuild. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-04kbuild: rename hostprogs-y/always to hostprogs/always-yMasahiro Yamada
In old days, the "host-progs" syntax was used for specifying host programs. It was renamed to the current "hostprogs-y" in 2004. It is typically useful in scripts/Makefile because it allows Kbuild to selectively compile host programs based on the kernel configuration. This commit renames like follows: always -> always-y hostprogs-y -> hostprogs So, scripts/Makefile will look like this: always-$(CONFIG_BUILD_BIN2C) += ... always-$(CONFIG_KALLSYMS) += ... ... hostprogs := $(always-y) $(always-m) I think this makes more sense because a host program is always a host program, irrespective of the kernel configuration. We want to specify which ones to compile by CONFIG options, so always-y will be handier. The "always", "hostprogs-y", "hostprogs-m" will be kept for backward compatibility for a while. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-04kbuild: fix the document to use extra-y for vmlinux.ldsMasahiro Yamada
The difference between "always" and "extra-y" is that the targets listed in $(always) are always built, whereas the ones in $(extra-y) are built only when KBUILD_BUILTIN is set. So, "make modules" does not build the targets in $(extra-y). vmlinux.lds is only needed for linking vmlinux. So, adding it to extra-y is more correct. In fact, arch/x86/kernel/Makefile does this. Fix the example code. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-02-01Merge tag 'kconfig-v5.6' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kconfig updates from Masahiro Yamada: - add 'yes2modconfig' and 'mod2yesconfig' targets (useful mainly for turning syzbot configs into more modular ones as a step to minimizing the result) - sanitize help text - various code cleanups * tag 'kconfig-v5.6' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: kconfig: fix documentation typos kconfig: fix an "implicit declaration of function" warning kconfig: fix nesting of symbol help text kconfig: distinguish between dependencies and visibility in help text kconfig: list all definitions of a symbol in help text kconfig: Add yes2modconfig and mod2yesconfig targets. kconfig: use $(PERL) in Makefile kconfig: fix too deep indentation in Makefile kconfig: localmodconfig: fix indentation for closing brace kconfig: localmodconfig: remove unused $config kconfig: squash prop_alloc() into menu_add_prop() kconfig: remove sym from struct property kconfig: remove 'prompt' argument from menu_add_prop() kconfig: move prompt handling to menu_add_prompt() from menu_add_prop() kconfig: remove 'prompt' symbol kconfig: drop T_WORD from the RHS of 'prompt' symbol kconfig: use parent->dep as the parentdep of 'menu' kconfig: remove the rootmenu check in menu_add_prop()
2020-01-22kconfig: fix documentation typosBjorn Helgaas
Fix a couple typos in kconfig-language documentation. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-01-07kbuild: create modules.builtin without Makefile.modbuiltin or tristate.confMasahiro Yamada
Commit bc081dd6e9f6 ("kbuild: generate modules.builtin") added infrastructure to generate modules.builtin, the list of all builtin modules. Basically, it works like this: - Kconfig generates include/config/tristate.conf, the list of tristate CONFIG options with a value in a capital letter. - scripts/Makefile.modbuiltin makes Kbuild descend into directories to collect the information of builtin modules. I am not a big fan of it because Kbuild ends up with traversing the source tree twice. I am not sure how perfectly it should work, but this approach cannot avoid false positives; even if the relevant CONFIG option is tristate, some Makefiles forces obj-m to obj-y. Some examples are: arch/powerpc/platforms/powermac/Makefile: obj-$(CONFIG_NVRAM:m=y) += nvram.o net/ipv6/Makefile: obj-$(subst m,y,$(CONFIG_IPV6)) += inet6_hashtables.o net/netlabel/Makefile: obj-$(subst m,y,$(CONFIG_IPV6)) += netlabel_calipso.o Nobody has complained about (or noticed) it, so it is probably fine to have false positives in modules.builtin. This commit simplifies the implementation. Let's exploit the fact that every module has MODULE_LICENSE(). (modpost shows a warning if MODULE_LICENSE is missing. If so, 0-day bot would already have blocked such a module.) I added MODULE_FILE to <linux/module.h>. When the code is being compiled as builtin, it will be filled with the file path of the module, and collected into modules.builtin.info. Then, scripts/link-vmlinux.sh extracts the list of builtin modules out of it. This new approach fixes the false-positives above, but adds another type of false-positives; non-modular code may have MODULE_LICENSE() by mistake. This is not a big deal, it is just the code is always orphan. We can clean it up if we like. You can see cleanup examples by: $ git log --grep='make.* explicitly non-modular' To sum up, this commits deletes lots of code, but still produces almost equivalent results. Please note it does not increase the vmlinux size at all. As you can see in include/asm-generic/vmlinux.lds.h, the .modinfo section is discarded in the link stage. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2019-12-22kbuild: clarify the difference between obj-y and obj-m w.r.t. descendingMasahiro Yamada
Kbuild descends into a directory by either 'y' or 'm', but there is an important difference. Kbuild combines the built-in objects into built-in.a in each directory. The built-in.a in the directory visited by obj-y is merged into the built-in.a in the parent directory. This merge happens recursively when Kbuild is ascending back towards the top directory, then built-in objects are linked into vmlinux eventually. This works properly only when the Makefile specifying obj-y is reachable by the chain of obj-y. On the other hand, Kbuild does not take built-in.a from the directory visited by obj-m. This it, all the objects in that directory are supposed to be modular. If Kbuild descends into a directory by obj-m, but the Makefile in the sub-directory specifies obj-y, those objects are just left orphan. The current statement "Kbuild only uses this information to decide that it needs to visit the directory" is misleading. Clarify the difference. Reported-by: Johan Hovold <johan@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Johan Hovold <johan@kernel.org>
2019-12-18kconfig: remove ---help--- from documentationMasahiro Yamada
Since commit 84af7a6194e4 ("checkpatch: kconfig: prefer 'help' over '---help---'"), scripts/checkpatch.pl warns the use of ---help---. Kconfig still supports ---help---, but new code should avoid using it. Let's stop advertising it in documentation. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2019-11-15kbuild: remove header compile testMasahiro Yamada
There are both positive and negative options about this feature. At first, I thought it was a good idea, but actually Linus stated a negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it is ugly and annoying. The baseline I'd like to keep is the compile-test of uapi headers. (Otherwise, kernel developers have no way to ensure the correctness of the exported headers.) I will maintain a small build rule in usr/include/Makefile. Remove the other header test functionality. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-11-11kbuild: do not read $(KBUILD_EXTMOD)/Module.symversMasahiro Yamada
Since commit 040fcc819a2e ("kbuild: improved modversioning support for external modules"), the external module build reads Module.symvers in the directory of the module itself, then dumps symbols back into it. It accumulates stale symbols in the file when you build an external module incrementally. The idea behind it was, as the commit log explained, you can copy Modules.symvers from one module to another when you need to pass symbol information between two modules. However, the manual copy of the file sounds questionable to me, and containing stale symbols is a downside. Some time later, commit 0d96fb20b7ed ("kbuild: Add new Kbuild variable KBUILD_EXTRA_SYMBOLS") introduced a saner approach. So, this commit removes the former one. Going forward, the external module build dumps symbols into Module.symvers to be carried via KBUILD_EXTRA_SYMBOLS, but never reads it automatically. With the -I option removed, there is no one to set the external_module flag unless KBUILD_EXTRA_SYMBOLS is passed. Now the -i option does it instead. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-10-11Merge tag 'modules-for-v5.4-rc3' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux Pull module fixes from Jessica Yu: "Code cleanups and kbuild/namespace related fixups from Masahiro. Most importantly, it fixes a namespace-related modpost issue for external module builds - Fix broken external module builds due to a modpost bug in read_dump(), where the namespace was not being strdup'd and sym->namespace would be set to bogus data. - Various namespace-related kbuild fixes and cleanups thanks to Masahiro Yamada" * tag 'modules-for-v5.4-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux: doc: move namespaces.rst from kbuild/ to core-api/ nsdeps: make generated patches independent of locale nsdeps: fix hashbang of scripts/nsdeps kbuild: fix build error of 'make nsdeps' in clean tree module: rename __kstrtab_ns_* to __kstrtabns_* to avoid symbol conflict modpost: fix broken sym->namespace for external module builds module: swap the order of symbol.namespace scripts: add_namespace: Fix coccicheck failed
2019-10-08doc: move namespaces.rst from kbuild/ to core-api/Masahiro Yamada
We discussed a better location for this file, and agreed that core-api/ is a good fit. Rename it to symbol-namespaces.rst for disambiguation, and also add it to index.rst and MAINTAINERS. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Matthias Maennich <maennich@google.com> Signed-off-by: Jessica Yu <jeyu@kernel.org>
2019-10-05kheaders: make headers archive reproducibleDmitry Goldin
In commit 43d8ce9d65a5 ("Provide in-kernel headers to make extending kernel easier") a new mechanism was introduced, for kernels >=5.2, which embeds the kernel headers in the kernel image or a module and exposes them in procfs for use by userland tools. The archive containing the header files has nondeterminism caused by header files metadata. This patch normalizes the metadata and utilizes KBUILD_BUILD_TIMESTAMP if provided and otherwise falls back to the default behaviour. In commit f7b101d33046 ("kheaders: Move from proc to sysfs") it was modified to use sysfs and the script for generation of the archive was renamed to what is being patched. Signed-off-by: Dmitry Goldin <dgoldin+lkml@protonmail.ch> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-10-05kbuild: two minor updates for Documentation/kbuild/modules.rstMasahiro Yamada
Capitalize the first word in the sentence. Use obj-m instead of obj-y. obj-y still works, but we have no built-in objects in external module builds. So, obj-m is better IMHO. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-10-01kbuild: correct formatting of header in kbuild module docsAlex Gaynor
Minor formatting fixup. Fixes: cd238effefa2 ("docs: kbuild: convert docs to ReST and rename to *.rst") Signed-off-by: Alex Gaynor <alex.gaynor@gmail.com> Signed-off-by: Matthew Garrett <mjg59@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-10-01kbuild: remove ar-option and KBUILD_ARFLAGSMasahiro Yamada
Commit 40df759e2b9e ("kbuild: Fix build with binutils <= 2.19") introduced ar-option and KBUILD_ARFLAGS to deal with old binutils. According to Documentation/process/changes.rst, the current minimal supported version of binutils is 2.21 so you can assume the 'D' option is always supported. Not only GNU ar but also llvm-ar supports it. With the 'D' option hard-coded, there is no more user of ar-option or KBUILD_ARFLAGS. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Tested-by: Nick Desaulniers <ndesaulniers@google.com>
2019-09-22Merge tag 'modules-for-v5.4' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux Pull modules updates from Jessica Yu: "The main bulk of this pull request introduces a new exported symbol namespaces feature. The number of exported symbols is increasingly growing with each release (we're at about 31k exports as of 5.3-rc7) and we currently have no way of visualizing how these symbols are "clustered" or making sense of this huge export surface. Namespacing exported symbols allows kernel developers to more explicitly partition and categorize exported symbols, as well as more easily limiting the availability of namespaced symbols to other parts of the kernel. For starters, we have introduced the USB_STORAGE namespace to demonstrate the API's usage. I have briefly summarized the feature and its main motivations in the tag below. Summary: - Introduce exported symbol namespaces. This new feature allows subsystem maintainers to partition and categorize their exported symbols into explicit namespaces. Module authors are now required to import the namespaces they need. Some of the main motivations of this feature include: allowing kernel developers to better manage the export surface, allow subsystem maintainers to explicitly state that usage of some exported symbols should only be limited to certain users (think: inter-module or inter-driver symbols, debugging symbols, etc), as well as more easily limiting the availability of namespaced symbols to other parts of the kernel. With the module import requirement, it is also easier to spot the misuse of exported symbols during patch review. Two new macros are introduced: EXPORT_SYMBOL_NS() and EXPORT_SYMBOL_NS_GPL(). The API is thoroughly documented in Documentation/kbuild/namespaces.rst. - Some small code and kbuild cleanups here and there" * tag 'modules-for-v5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux: module: Remove leftover '#undef' from export header module: remove unneeded casts in cmp_name() module: move CONFIG_UNUSED_SYMBOLS to the sub-menu of MODULES module: remove redundant 'depends on MODULES' module: Fix link failure due to invalid relocation on namespace offset usb-storage: export symbols in USB_STORAGE namespace usb-storage: remove single-use define for debugging docs: Add documentation for Symbol Namespaces scripts: Coccinelle script for namespace dependencies. modpost: add support for generating namespace dependencies export: allow definition default namespaces in Makefiles or sources module: add config option MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS modpost: add support for symbol namespaces module: add support for symbol namespaces. export: explicitly align struct kernel_symbol module: support reading multiple values per modinfo tag
2019-09-20Merge tag 'kbuild-v5.4' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - add modpost warn exported symbols marked as 'static' because 'static' and EXPORT_SYMBOL is an odd combination - break the build early if gold linker is used - optimize the Bison rule to produce .c and .h files by a single pattern rule - handle PREEMPT_RT in the module vermagic and UTS_VERSION - warn CONFIG options leaked to the user-space except existing ones - make single targets work properly - rebuild modules when module linker scripts are updated - split the module final link stage into scripts/Makefile.modfinal - fix the missed error code in merge_config.sh - improve the error message displayed on the attempt of the O= build in unclean source tree - remove 'clean-dirs' syntax - disable -Wimplicit-fallthrough warning for Clang - add CONFIG_CC_OPTIMIZE_FOR_SIZE_O3 for ARC - remove ARCH_{CPP,A,C}FLAGS variables - add $(BASH) to run bash scripts - change *CFLAGS_<basetarget>.o to take the relative path to $(obj) instead of the basename - stop suppressing Clang's -Wunused-function warnings when W=1 - fix linux/export.h to avoid genksyms calculating CRC of trimmed exported symbols - misc cleanups * tag 'kbuild-v5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (63 commits) genksyms: convert to SPDX License Identifier for lex.l and parse.y modpost: use __section in the output to *.mod.c modpost: use MODULE_INFO() for __module_depends export.h, genksyms: do not make genksyms calculate CRC of trimmed symbols export.h: remove defined(__KERNEL__), which is no longer needed kbuild: allow Clang to find unused static inline functions for W=1 build kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN kbuild: refactor scripts/Makefile.extrawarn merge_config.sh: ignore unwanted grep errors kbuild: change *FLAGS_<basetarget>.o to take the path relative to $(obj) modpost: add NOFAIL to strndup modpost: add guid_t type definition kbuild: add $(BASH) to run scripts with bash-extension kbuild: remove ARCH_{CPP,A,C}FLAGS kbuild,arc: add CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3 for ARC kbuild: Do not enable -Wimplicit-fallthrough for clang for now kbuild: clean up subdir-ymn calculation in Makefile.clean kbuild: remove unneeded '+' marker from cmd_clean kbuild: remove clean-dirs syntax kbuild: check clean srctree even earlier ...
2019-09-15Documentation: kbuild: Add document about reproducible buildsBen Hutchings
In the Distribution Kernels track at Linux Plumbers Conference there was some discussion around the difficulty of making kernel builds reproducible. This is a solved problem, but the solutions don't appear to be documented in one place. This document lists the issues I know about and the settings needed to ensure reproducibility. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2019-09-10docs: Add documentation for Symbol NamespacesMatthias Maennich
Describe using Symbol Namespaces from a perspective of a user. I.e. module authors or subsystem maintainers. Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Jessica Yu <jeyu@kernel.org>
2019-09-10modpost: add support for symbol namespacesMatthias Maennich
Add support for symbols that are exported into namespaces. For that, extract any namespace suffix from the symbol name. In addition, emit a warning whenever a module refers to an exported symbol without explicitly importing the namespace that it is defined in. This patch consistently adds the namespace suffix to symbol names exported into Module.symvers. Example warning emitted by modpost in case of the above violation: WARNING: module ums-usbat uses symbol usb_stor_resume from namespace USB_STORAGE, but does not import it. Co-developed-by: Martijn Coenen <maco@android.com> Signed-off-by: Martijn Coenen <maco@android.com> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Jessica Yu <jeyu@kernel.org>
2019-09-06kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARNMasahiro Yamada
KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning options for GCC, but now it is a historical misnomer since we use it also for Clang, DTC, and even kernel-doc. Rename it to more sensible, shorter KBUILD_EXTRA_WARN. For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still supported (but not advertised in the documentation). I also fixed up 'make help', and updated the documentation. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>
2019-09-04kbuild: remove ARCH_{CPP,A,C}FLAGSMasahiro Yamada
These flags were added by commit 61754c18752f ("kbuild: Allow arch Makefiles to override {cpp,ld,c}flags") to allow ARC to override -O2. We did not see any other usage after all. Now that ARC switched to CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3, there is no more user of these variables. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-29kbuild: remove clean-dirs syntaxMasahiro Yamada
The only the difference between clean-files and clean-dirs is the -r option passed to the 'rm' command. You can always pass -r, and then remove the clean-dirs syntax. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-29docs: kbuild: remove cc-ldoption from document againMasahiro Yamada
Commit 055efab3120b ("kbuild: drop support for cc-ldoption") correctly removed the cc-ldoption from Documentation/kbuild/makefiles.txt, but commit cd238effefa2 ("docs: kbuild: convert docs to ReST and rename to *.rst") revived it. I guess it was a rebase mistake. Remove it again. Fixes: cd238effefa2 ("docs: kbuild: convert docs to ReST and rename to *.rst") Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-29docs: kbuild: fix invalid ReST syntaxMasahiro Yamada
I see the following warnings when I open this document with a ReST viewer, retext: /home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1142: (WARNING/2) Inline emphasis start-string without end-string. /home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1152: (WARNING/2) Inline emphasis start-string without end-string. /home/masahiro/ref/linux/Documentation/kbuild/makefiles.rst:1154: (WARNING/2) Inline emphasis start-string without end-string. These hunks were added by commit e846f0dc57f4 ("kbuild: add support for ensuring headers are self-contained") and commit 1e21cbfada87 ("kbuild: support header-test-pattern-y"), respectively. They were written not for ReST but for the plain text, and merged via the kbuild tree. In the same development cycle, this document was converted to ReST by commit cd238effefa2 ("docs: kbuild: convert docs to ReST and rename to *.rst"), and merged via the doc sub-system. Merging them together into Linus' tree resulted in the current situation. To fix the syntax, surround the asterisks with back-quotes, and use :: for the code sample. Fixes: 39ceda5ce1b0 ("Merge tag 'kbuild-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild") Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-21kbuild: rebuild modules when module linker scripts are updatedMasahiro Yamada
Currently, the timestamp of module linker scripts are not checked. Add them to the dependency of modules so they are correctly rebuilt. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-21kbuild: move KBUILD_LDS, KBUILD_VMLINUX_{OBJS,LIBS} to makefiles.rstMasahiro Yamada
These three variables are not intended to be tweaked by users. Move them from kbuild.rst to makefiles.rst. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-08-21kbuild: do not descend to ./Kbuild when cleaningMasahiro Yamada
'make clean' descends into ./Kbuild, but does not clean anything since everything is added to no-clean-files. There is no need to descend to ./Kbuild in the first place. We can drop the no-clean-files assignment. With this, there is no more user of no-clean-files. I will keep it for a while to see whether a new user will appear. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-20Merge tag 'kbuild-v5.3-2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull more Kbuild updates from Masahiro Yamada: - match the directory structure of the linux-libc-dev package to that of Debian-based distributions - fix incorrect include/config/auto.conf generation when Kconfig creates it along with the .config file - remove misleading $(AS) from documents - clean up precious tag files by distclean instead of mrproper - add a new coccinelle patch for devm_platform_ioremap_resource migration - refactor module-related scripts to read modules.order instead of $(MODVERDIR)/*.mod files to get the list of created modules - remove MODVERDIR - update list of header compile-test - add -fcf-protection=none flag to avoid conflict with the retpoline flags when CONFIG_RETPOLINE=y - misc cleanups * tag 'kbuild-v5.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (25 commits) kbuild: add -fcf-protection=none when using retpoline flags kbuild: update compile-test header list for v5.3-rc1 kbuild: split out *.mod out of {single,multi}-used-m rules kbuild: remove 'prepare1' target kbuild: remove the first line of *.mod files kbuild: create *.mod with full directory path and remove MODVERDIR kbuild: export_report: read modules.order instead of .tmp_versions/*.mod kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod kbuild: modsign: read modules.order instead of $(MODVERDIR)/*.mod kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver kbuild: remove duplication from modules.order in sub-directories kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin} kbuild: do not create empty modules.order in the prepare stage coccinelle: api: add devm_platform_ioremap_resource script kbuild: compile-test headers listed in header-test-m as well kbuild: remove unused hostcc-option kbuild: remove tag files by distclean instead of mrproper kbuild: add --hash-style= and --build-id unconditionally kbuild: get rid of misleading $(AS) from documents ...
2019-07-17kbuild: get rid of misleading $(AS) from documentsMasahiro Yamada
The assembler files in the kernel are *.S instead of *.s, so they must be preprocessed. Since 'as' of GNU binutils is not able to preprocess, we always use $(CC) as an assembler driver. $(AS) is almost unused in Kbuild. As of v5.2, there is just one place that directly invokes $(AS). $ git grep -e '$(AS)' -e '${AS}' -e '$AS' -e '$(AS:' -e '${AS:' -- :^Documentation drivers/net/wan/Makefile: AS68K = $(AS) The documentation about *_AFLAGS* sounds like the flags were passed to $(AS). This is somewhat misleading. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com>
2019-07-15docs: kbuild: fix build with pdf and fix some minor issuesMauro Carvalho Chehab
The tag ".. include" should be replaced by ".. literalinclude" at issues.rst, otherwise it causes TeX to crash due to excessive usage of stack with Sphinx 2.0. While here, solve a few minor issues at the kbuild book output by adding extra blank lines. Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
2019-07-15docs: add some directories to the main documentation indexMauro Carvalho Chehab
The contents of those directories were orphaned at the documentation body. While those directories could likely be moved to be inside some guide, I'm opting to just adding their indexes to the main one, removing the :orphan: and adding the SPDX header. For the drivers, the rationale is that the documentation contains a mix of Kernelspace, uAPI and admin-guide. So, better to keep them on separate directories, as we've be doing with similar subsystem-specific docs that were not split yet. For the others, well... I'm too lazy to do the move. Also, it seems to make sense to keep at least some of those at the main dir (like kbuild, for example). In any case, a latter patch could do the move. Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> Acked-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
2019-07-12Merge tag 'kbuild-v5.3' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - remove headers_{install,check}_all targets - remove unreasonable 'depends on !UML' from CONFIG_SAMPLES - re-implement 'make headers_install' more cleanly - add new header-test-y syntax to compile-test headers - compile-test exported headers to ensure they are compilable in user-space - compile-test headers under include/ to ensure they are self-contained - remove -Waggregate-return, -Wno-uninitialized, -Wno-unused-value flags - add -Werror=unknown-warning-option for Clang - add 128-bit built-in types support to genksyms - fix missed rebuild of modules.builtin - propagate 'No space left on device' error in fixdep to Make - allow Clang to use its integrated assembler - improve some coccinelle scripts - add a new flag KBUILD_ABS_SRCTREE to request Kbuild to use absolute path for $(srctree). - do not ignore errors when compression utility is missing - misc cleanups * tag 'kbuild-v5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (49 commits) kbuild: use -- separater intead of $(filter-out ...) for cc-cross-prefix kbuild: Inform user to pass ARCH= for make mrproper kbuild: fix compression errors getting ignored kbuild: add a flag to force absolute path for srctree kbuild: replace KBUILD_SRCTREE with boolean building_out_of_srctree kbuild: remove src and obj from the top Makefile scripts/tags.sh: remove unused environment variables from comments scripts/tags.sh: drop SUBARCH support for ARM kbuild: compile-test kernel headers to ensure they are self-contained kheaders: include only headers into kheaders_data.tar.xz kheaders: remove meaningless -R option of 'ls' kbuild: support header-test-pattern-y kbuild: do not create wrappers for header-test-y kbuild: compile-test exported headers to ensure they are self-contained init/Kconfig: add CONFIG_CC_CAN_LINK kallsyms: exclude kasan local symbols on s390 kbuild: add more hints about SUBDIRS replacement coccinelle: api/stream_open: treat all wait_.*() calls as blocking coccinelle: put_device: Add a cast to an expression for an assignment coccinelle: put_device: Adjust a message construction ...
2019-07-11kbuild: add a flag to force absolute path for srctreeMasahiro Yamada
In old days, Kbuild always used an absolute path for $(srctree). Since commit 890676c65d69 ("kbuild: Use relative path when building in the source tree"), $(srctree) is '.' when O= was not passed from the command line. Yet, using absolute paths is useful in some cases even without O=, for instance, to create a cscope file with absolute path tags. 'O=.' was known to work as a workaround to force Kbuild to use absolute paths even when you are building in the source tree. Since commit 25b146c5b8ce ("kbuild: allow Kbuild to start from any directory"), Kbuild is too clever to be tricked. Even if you pass 'O=.' Kbuild notices you are building in the source tree, then use '.' for $(srctree). So, 'make O=. cscope' is no help to create absolute path tags. We cannot force one or the other according to commit e93bc1a0cab3 ("Revert "kbuild: specify absolute paths for cscope""). Both of relative path and absolute path have pros and cons. This commit adds a new flag KBUILD_ABS_SRCTREE to allow users to choose the absolute path for $(srctree). 'make KBUILD_ABS_SRCTREE=1 cscope' will work as a replacement of 'make O=. cscope'. Reported-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-09kbuild: support header-test-pattern-yMasahiro Yamada
In my view, most of headers can be self-contained. So, it would be tedious to add every header to header-test-y explicitly. We usually end up with "all headers with some exceptions". There are two types in exceptions: [1] headers that are never compiled as standalone units For examples, include/linux/compiler-gcc.h is not intended for direct inclusion. We should always exclude such ones. [2] headers that are conditionally compiled as standalone units Some headers can be compiled only for particular architectures. For example, include/linux/arm-cci.h can be compiled only for arm/arm64 because it requires <asm/arm-cci.h> to exist. Clang can compile include/soc/nps/mtm.h only for arc because it contains an arch-specific register in inline assembler. So, you can write Makefile like this: header-test- += linux/compiler-gcc.h header-test-$(CONFIG_ARM) += linux/arm-cci.h header-test-$(CONFIG_ARM64) += linux/arm-cci.h header-test-$(CONFIG_ARC) += soc/nps/mtm.h The new syntax header-test-pattern-y will be useful to specify "the rest". The typical usage is like this: header-test-pattern-y += */*.h This will add all the headers in sub-directories to the test coverage, excluding $(header-test-). In this regards, header-test-pattern-y behaves like a weaker variant of header-test-y. Caveat: The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/ but not $(objtree)/$(obj)/. Stale generated headers are often left over when you traverse the git history without cleaning. Wildcard patterns for $(objtree) may match to stale headers, which could fail to compile. One pitfall is $(srctree)/$(src)/ and $(objtree)/$(obj)/ point to the same directory for in-tree building. So, header-test-pattern-y should be used with care since it can potentially match to stale headers. Caveat2: You could use wildcard for header-test-. For example, header-test- += asm-generic/% ... will exclude headers in asm-generic directory. Unfortunately, the wildcard character is '%' instead of '*' here because this is evaluated by $(filter-out ...) whereas header-test-pattern-y is evaluated by $(wildcard ...). This is a kludge, but seems useful in some places... Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Jani Nikula <jani.nikula@intel.com>
2019-07-09kbuild: do not create wrappers for header-test-yMasahiro Yamada
header-test-y does not work with headers in sub-directories. For example, you may want to write a Makefile, like this: include/linux/Kbuild: header-test-y += mtd/nand.h This entry will create a wrapper include/linux/mtd/nand.hdrtest.c with the following content: #include "mtd/nand.h" To make this work, we need to add $(srctree)/include/linux to the header search path. It would be tedious to add ccflags-y. Instead, we could change the *.hdrtest.c rule to wrap: #include "nand.h" This works for in-tree build since #include "..." searches in the relative path from the header with this directive. For O=... build, we need to add $(srctree)/include/linux/mtd to the header search path, which will be even more tedious. After all, I thought it would be handier to compile headers directly without creating wrappers. I added a new build rule to compile %.h into %.h.s The target is %.h.s instead of %.h.o because it is slightly faster. Also, as for GCC, an empty assembly is smaller than an empty object. I wrote the build rule: $(CC) $(c_flags) -S -o $@ -x c /dev/null -include $< instead of: $(CC) $(c_flags) -S -o $@ -x c $< Both work fine with GCC, but the latter is bad for Clang. This comes down to the difference in the -Wunused-function policy. GCC does not warn about unused 'static inline' functions at all. Clang does not warn about the ones in included headers, but does about the ones in the source. So, we should handle headers as headers, not as source files. In fact, this has been hidden since commit abb2ea7dfd82 ("compiler, clang: suppress warning for unused static inline functions"), but we should not rely on that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Jani Nikula <jani.nikula@intel.com> Tested-by: Jani Nikula <jani.nikula@intel.com>