blob: 800b83a89de0675eca0c491131081c809601f0ba [file] [log] [blame]
Simon Glass79cc9be2022-11-09 19:14:43 -07001.. SPDX-License-Identifier: GPL-2.0+
2
3Buildman build tool
4===================
5
Simon Glass79cc9be2022-11-09 19:14:43 -07006Quick-start
7-----------
8
9If you just want to quickly set up buildman so you can build something (for
10example Raspberry Pi 2):
11
12.. code-block:: bash
13
14 cd /path/to/u-boot
15 PATH=$PATH:`pwd`/tools/buildman
16 buildman --fetch-arch arm
17 buildman -k rpi_2
18 ls ../current/rpi_2
19 # u-boot.bin is the output image
20
21
22What is this?
23-------------
24
25This tool handles building U-Boot to check that you have not broken it
26with your patch series. It can build each individual commit and report
27which boards fail on which commits, and which errors come up. It aims
28to make full use of multi-processor machines.
29
30A key feature of buildman is its output summary, which allows warnings,
31errors or image size increases in a particular commit or board to be
32quickly identified and the offending commit pinpointed. This can be a big
33help for anyone working with >10 patches at a time.
34
35
36Caveats
37-------
38
39Buildman can be stopped and restarted, in which case it will continue
40where it left off. This should happen cleanly and without side-effects.
41If not, it is a bug, for which a patch would be welcome.
42
43Buildman gets so tied up in its work that it can ignore the outside world.
44You may need to press Ctrl-C several times to quit it. Also it will print
45out various exceptions when stopped. You may have to kill it since the
46Ctrl-C handling is somewhat broken.
47
48
49Theory of Operation
50-------------------
51
52(please read this section in full twice or you will be perpetually confused)
53
54Buildman is a builder. It is not make, although it runs make. It does not
55produce any useful output on the terminal while building, except for
56progress information (but see -v below). All the output (errors, warnings and
57binaries if you ask for them) is stored in output directories, which you can
58look at from a separate 'buildman -s' instance while the build is progressing,
59or when it is finished.
60
61Buildman is designed to build entire git branches, i.e. muliple commits. It
62can be run repeatedly on the same branch after making changes to commits on
63that branch. In this case it will automatically rebuild commits which have
64changed (and remove its old results for that commit). It is possible to build
65a branch for one board, then later build it for another board. This adds to
66the output, so now you have results for two boards. If you want buildman to
67re-build a commit it has already built (e.g. because of a toolchain update),
68use the -f flag.
69
70Buildman produces a concise summary of which boards succeeded and failed.
71It shows which commit introduced which board failure using a simple
72red/green colour coding (with yellow/cyan for warnings). Full error
73information can be requested, in which case it is de-duped and displayed
74against the commit that introduced the error. An example workflow is below.
75
76Buildman stores image size information and can report changes in image size
77from commit to commit. An example of this is below.
78
79Buildman starts multiple threads, and each thread builds for one board at
80a time. A thread starts at the first commit, configures the source for your
81board and builds it. Then it checks out the next commit and does an
82incremental build (i.e. not using 'make xxx_defconfig' unless you use -C).
83Eventually the thread reaches the last commit and stops. If a commit causes
84an error or warning, buildman will try it again after reconfiguring (but see
85-Q). Thus some commits may be built twice, with the first result silently
86discarded. Lots of errors and warnings will causes lots of reconfigures and your
87build will be very slow. This is because a file that produces just a warning
88would not normally be rebuilt in an incremental build. Once a thread finishes
89building all the commits for a board, it starts on the commits for another
90board.
91
92Buildman works in an entirely separate place from your U-Boot repository.
93It creates a separate working directory for each thread, and puts the
94output files in the working directory, organised by commit name and board
95name, in a two-level hierarchy (but see -P).
96
97Buildman is invoked in your U-Boot directory, the one with the .git
98directory. It clones this repository into a copy for each thread, and the
99threads do not affect the state of your git repository. Any checkouts done
100by the thread affect only the working directory for that thread.
101
102Buildman automatically selects the correct tool chain for each board. You
103must supply suitable tool chains (see --fetch-arch), but buildman takes care
104of selecting the right one.
105
106Buildman generally builds a branch (with the -b flag), and in this case
107builds the upstream commit as well, for comparison. So even if you have one
108commit in your branch, two commits will be built. Put all your commits in a
109branch, set the branch's upstream to a valid value, and all will be well.
110Otherwise buildman will perform random actions. Use -n to check what the
111random actions might be.
112
113Buildman effectively has two modes: without -s it builds, with -s it
114summarises the results of previous (or active) builds.
115
116If you just want to build the current source tree, leave off the -b flag.
117This will display results and errors as they happen. You can still look at
118them later using -se. Note that buildman will assume that the source has
119changed, and will build all specified boards in this case.
120
121Buildman is optimised for building many commits at once, for many boards.
122On multi-core machines, Buildman is fast because it uses most of the
123available CPU power. When it gets to the end, or if you are building just
124a few commits or boards, it will be pretty slow. As a tip, if you don't
125plan to use your machine for anything else, you can use -T to increase the
126number of threads beyond the default.
127
128
129Selecting which boards to build
130-------------------------------
131
132Buildman lets you build all boards, or a subset. Specify the subset by passing
133command-line arguments that list the desired build target, architecture,
134CPU, board name, vendor, SoC or options. Multiple arguments are allowed. Each
135argument will be interpreted as a regular expression, so behaviour is a superset
136of exact or substring matching. Examples are:
137
138- 'tegra20' - all boards with a Tegra20 SoC
139- 'tegra' - all boards with any Tegra Soc (Tegra20, Tegra30, Tegra114...)
140- '^tegra[23]0$' - all boards with either Tegra20 or Tegra30 SoC
141- 'powerpc' - all PowerPC boards
142
143While the default is to OR the terms together, you can also make use of
144the '&' operator to limit the selection:
145
146- 'freescale & arm sandbox' - all Freescale boards with ARM architecture, plus
147 sandbox
148
149You can also use -x to specifically exclude some boards. For example:
150
151 buildman arm -x nvidia,freescale,.*ball$
152
153means to build all arm boards except nvidia, freescale and anything ending
154with 'ball'.
155
156For building specific boards you can use the --boards (or --bo) option, which
157takes a comma-separated list of board target names and be used multiple times
158on the command line:
159
160.. code-block:: bash
161
162 buildman --boards sandbox,snow --boards
163
164It is convenient to use the -n option to see what will be built based on
165the subset given. Use -v as well to get an actual list of boards.
166
167Buildman does not store intermediate object files. It optionally copies
168the binary output into a directory when a build is successful (-k). Size
169information is always recorded. It needs a fair bit of disk space to work,
170typically 250MB per thread.
171
172
173Setting up
174----------
175
176#. Get the U-Boot source. You probably already have it, but if not these
177 steps should get you started with a repo and some commits for testing.
178
179 .. code-block:: bash
180
181 cd /path/to/u-boot
182 git clone git://git.denx.de/u-boot.git .
183 git checkout -b my-branch origin/master
184 # Add some commits to the branch, reading for testing
185
186#. Create ~/.buildman to tell buildman where to find tool chains (see
187 buildman_settings_ for details). As an example::
188
189 # Buildman settings file
190
191 [toolchain]
192 root: /
193 rest: /toolchains/*
194 eldk: /opt/eldk-4.2
195 arm: /opt/linaro/gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux
196 aarch64: /opt/linaro/gcc-linaro-aarch64-none-elf-4.8-2013.10_linux
197
Simon Glassc325dbe2022-11-09 19:14:45 -0700198 [toolchain-prefix]
199 arc = /opt/arc/arc_gnu_2021.03_prebuilt_elf32_le_linux_install/bin/arc-elf32-
200
Simon Glass79cc9be2022-11-09 19:14:43 -0700201 [toolchain-alias]
Simon Glass8897c7d2022-11-09 19:14:46 -0700202 riscv = riscv32
203 sh = sh4
Simon Glass79cc9be2022-11-09 19:14:43 -0700204 x86: i386
Simon Glass79cc9be2022-11-09 19:14:43 -0700205
206
207 This selects the available toolchain paths. Add the base directory for
208 each of your toolchains here. Buildman will search inside these directories
209 and also in any '/usr' and '/usr/bin' subdirectories.
210
211 Make sure the tags (here root: rest: and eldk:) are unique.
212
213 The toolchain-alias section indicates that the i386 toolchain should be used
214 to build x86 commits.
215
216 Note that you can also specific exactly toolchain prefixes if you like::
217
218 [toolchain-prefix]
219 arm: /opt/arm-eabi-4.6/bin/arm-eabi-
220
221 or even::
222
223 [toolchain-prefix]
224 arm: /opt/arm-eabi-4.6/bin/arm-eabi-gcc
225
226 This tells buildman that you want to use this exact toolchain for the arm
227 architecture. This will override any toolchains found by searching using the
228 [toolchain] settings.
229
230 Since the toolchain prefix is an explicit request, buildman will report an
231 error if a toolchain is not found with that prefix. The current PATH will be
232 searched, so it is possible to use::
233
234 [toolchain-prefix]
235 arm: arm-none-eabi-
236
237 and buildman will find arm-none-eabi-gcc in /usr/bin if you have it
238 installed.
239
240 Another example::
241
242 [toolchain-wrapper]
243 wrapper: ccache
244
245 This tells buildman to use a compiler wrapper in front of CROSS_COMPILE. In
246 this example, ccache. It doesn't affect the toolchain scan. The wrapper is
247 added when CROSS_COMPILE environtal variable is set. The name in this
248 section is ignored. If more than one line is provided, only the last one
249 is taken.
250
251#. Make sure you have the require Python pre-requisites
252
253 Buildman uses multiprocessing, Queue, shutil, StringIO, ConfigParser and
254 urllib2. These should normally be available, but if you get an error like
255 this then you will need to obtain those modules::
256
257 ImportError: No module named multiprocessing
258
259
260#. Check the available toolchains
261
262 Run this check to make sure that you have a toolchain for every architecture::
263
264 $ ./tools/buildman/buildman --list-tool-chains
265 Scanning for tool chains
266 - scanning prefix '/opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-'
267 Tool chain test: OK, arch='x86', priority 1
268 - scanning prefix '/opt/arm-eabi-4.6/bin/arm-eabi-'
269 Tool chain test: OK, arch='arm', priority 1
270 - scanning path '/toolchains/gcc-4.9.0-nolibc/i386-linux'
271 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/.'
272 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin'
273 - found '/toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc'
274 - looking in '/toolchains/gcc-4.9.0-nolibc/i386-linux/usr/bin'
275 Tool chain test: OK, arch='i386', priority 4
276 - scanning path '/toolchains/gcc-4.9.0-nolibc/aarch64-linux'
277 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/.'
278 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin'
279 - found '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc'
280 - looking in '/toolchains/gcc-4.9.0-nolibc/aarch64-linux/usr/bin'
281 Tool chain test: OK, arch='aarch64', priority 4
282 - scanning path '/toolchains/gcc-4.9.0-nolibc/microblaze-linux'
283 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/.'
284 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin'
285 - found '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc'
286 - looking in '/toolchains/gcc-4.9.0-nolibc/microblaze-linux/usr/bin'
287 Tool chain test: OK, arch='microblaze', priority 4
288 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips64-linux'
289 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/.'
290 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin'
291 - found '/toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc'
292 - looking in '/toolchains/gcc-4.9.0-nolibc/mips64-linux/usr/bin'
293 Tool chain test: OK, arch='mips64', priority 4
294 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc64-linux'
295 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/.'
296 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin'
297 - found '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc'
298 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc64-linux/usr/bin'
299 Tool chain test: OK, arch='sparc64', priority 4
300 - scanning path '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi'
301 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/.'
302 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin'
303 - found '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc'
304 - looking in '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/usr/bin'
305 Tool chain test: OK, arch='arm', priority 3
306 Toolchain '/toolchains/gcc-4.9.0-nolibc/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc' at priority 3 will be ignored because another toolchain for arch 'arm' has priority 1
307 - scanning path '/toolchains/gcc-4.9.0-nolibc/sparc-linux'
308 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/.'
309 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin'
310 - found '/toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc'
311 - looking in '/toolchains/gcc-4.9.0-nolibc/sparc-linux/usr/bin'
312 Tool chain test: OK, arch='sparc', priority 4
313 - scanning path '/toolchains/gcc-4.9.0-nolibc/mips-linux'
314 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/.'
315 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin'
316 - found '/toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc'
317 - looking in '/toolchains/gcc-4.9.0-nolibc/mips-linux/usr/bin'
318 Tool chain test: OK, arch='mips', priority 4
319 - scanning path '/toolchains/gcc-4.9.0-nolibc/x86_64-linux'
320 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/.'
321 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin'
322 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc'
323 - found '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc'
324 - looking in '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/usr/bin'
325 Tool chain test: OK, arch='x86_64', priority 4
326 Tool chain test: OK, arch='x86_64', priority 4
327 Toolchain '/toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-x86_64-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
328 - scanning path '/toolchains/gcc-4.9.0-nolibc/m68k-linux'
329 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/.'
330 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin'
331 - found '/toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc'
332 - looking in '/toolchains/gcc-4.9.0-nolibc/m68k-linux/usr/bin'
333 Tool chain test: OK, arch='m68k', priority 4
334 - scanning path '/toolchains/gcc-4.9.0-nolibc/powerpc-linux'
335 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/.'
336 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin'
337 - found '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
338 - looking in '/toolchains/gcc-4.9.0-nolibc/powerpc-linux/usr/bin'
339 Tool chain test: OK, arch='powerpc', priority 4
340 - scanning path '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux'
341 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/.'
342 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin'
343 - found '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc'
344 - looking in '/toolchains/gcc-4.6.3-nolibc/bfin-uclinux/usr/bin'
345 Tool chain test: OK, arch='bfin', priority 6
346 - scanning path '/toolchains/gcc-4.6.3-nolibc/sparc-linux'
347 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/.'
348 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin'
349 - found '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc'
350 - looking in '/toolchains/gcc-4.6.3-nolibc/sparc-linux/usr/bin'
351 Tool chain test: OK, arch='sparc', priority 4
352 Toolchain '/toolchains/gcc-4.6.3-nolibc/sparc-linux/bin/sparc-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'sparc' has priority 4
353 - scanning path '/toolchains/gcc-4.6.3-nolibc/mips-linux'
354 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/.'
355 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin'
356 - found '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc'
357 - looking in '/toolchains/gcc-4.6.3-nolibc/mips-linux/usr/bin'
358 Tool chain test: OK, arch='mips', priority 4
359 Toolchain '/toolchains/gcc-4.6.3-nolibc/mips-linux/bin/mips-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'mips' has priority 4
360 - scanning path '/toolchains/gcc-4.6.3-nolibc/m68k-linux'
361 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/.'
362 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin'
363 - found '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc'
364 - looking in '/toolchains/gcc-4.6.3-nolibc/m68k-linux/usr/bin'
365 Tool chain test: OK, arch='m68k', priority 4
366 Toolchain '/toolchains/gcc-4.6.3-nolibc/m68k-linux/bin/m68k-linux-gcc' at priority 4 will be ignored because another toolchain for arch 'm68k' has priority 4
367 - scanning path '/toolchains/gcc-4.6.3-nolibc/powerpc-linux'
368 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/.'
369 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin'
370 - found '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc'
371 - looking in '/toolchains/gcc-4.6.3-nolibc/powerpc-linux/usr/bin'
372 Tool chain test: OK, arch='powerpc', priority 4
373 Tool chain test: OK, arch='or32', priority 4
374 - scanning path '/'
375 - looking in '/.'
376 - looking in '/bin'
377 - looking in '/usr/bin'
378 - found '/usr/bin/i586-mingw32msvc-gcc'
379 - found '/usr/bin/c89-gcc'
380 - found '/usr/bin/x86_64-linux-gnu-gcc'
381 - found '/usr/bin/gcc'
382 - found '/usr/bin/c99-gcc'
383 - found '/usr/bin/arm-linux-gnueabi-gcc'
384 - found '/usr/bin/aarch64-linux-gnu-gcc'
385 - found '/usr/bin/winegcc'
386 - found '/usr/bin/arm-linux-gnueabihf-gcc'
387 Tool chain test: OK, arch='i586', priority 11
388 Tool chain test: OK, arch='c89', priority 11
389 Tool chain test: OK, arch='x86_64', priority 4
390 Toolchain '/usr/bin/x86_64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'x86_64' has priority 4
391 Tool chain test: OK, arch='sandbox', priority 11
392 Tool chain test: OK, arch='c99', priority 11
393 Tool chain test: OK, arch='arm', priority 4
394 Toolchain '/usr/bin/arm-linux-gnueabi-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
395 Tool chain test: OK, arch='aarch64', priority 4
396 Toolchain '/usr/bin/aarch64-linux-gnu-gcc' at priority 4 will be ignored because another toolchain for arch 'aarch64' has priority 4
397 Tool chain test: OK, arch='sandbox', priority 11
398 Toolchain '/usr/bin/winegcc' at priority 11 will be ignored because another toolchain for arch 'sandbox' has priority 11
399 Tool chain test: OK, arch='arm', priority 4
400 Toolchain '/usr/bin/arm-linux-gnueabihf-gcc' at priority 4 will be ignored because another toolchain for arch 'arm' has priority 1
401 List of available toolchains (34):
402 aarch64 : /toolchains/gcc-4.9.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc
403 alpha : /toolchains/gcc-4.9.0-nolibc/alpha-linux/bin/alpha-linux-gcc
404 am33_2.0 : /toolchains/gcc-4.9.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc
405 arm : /opt/arm-eabi-4.6/bin/arm-eabi-gcc
406 bfin : /toolchains/gcc-4.6.3-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc
407 c89 : /usr/bin/c89-gcc
408 c99 : /usr/bin/c99-gcc
409 frv : /toolchains/gcc-4.9.0-nolibc/frv-linux/bin/frv-linux-gcc
410 h8300 : /toolchains/gcc-4.9.0-nolibc/h8300-elf/bin/h8300-elf-gcc
411 hppa : /toolchains/gcc-4.9.0-nolibc/hppa-linux/bin/hppa-linux-gcc
412 hppa64 : /toolchains/gcc-4.9.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc
413 i386 : /toolchains/gcc-4.9.0-nolibc/i386-linux/bin/i386-linux-gcc
414 i586 : /usr/bin/i586-mingw32msvc-gcc
415 ia64 : /toolchains/gcc-4.9.0-nolibc/ia64-linux/bin/ia64-linux-gcc
416 m32r : /toolchains/gcc-4.9.0-nolibc/m32r-linux/bin/m32r-linux-gcc
417 m68k : /toolchains/gcc-4.9.0-nolibc/m68k-linux/bin/m68k-linux-gcc
418 microblaze: /toolchains/gcc-4.9.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc
419 mips : /toolchains/gcc-4.9.0-nolibc/mips-linux/bin/mips-linux-gcc
420 mips64 : /toolchains/gcc-4.9.0-nolibc/mips64-linux/bin/mips64-linux-gcc
421 or32 : /toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc
422 powerpc : /toolchains/gcc-4.9.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc
423 powerpc64 : /toolchains/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc
424 ppc64le : /toolchains/gcc-4.9.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc
425 s390x : /toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc
426 sandbox : /usr/bin/gcc
427 sh4 : /toolchains/gcc-4.6.3-nolibc/sh4-linux/bin/sh4-linux-gcc
428 sparc : /toolchains/gcc-4.9.0-nolibc/sparc-linux/bin/sparc-linux-gcc
429 sparc64 : /toolchains/gcc-4.9.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc
430 tilegx : /toolchains/gcc-4.6.2-nolibc/tilegx-linux/bin/tilegx-linux-gcc
431 x86 : /opt/gcc-4.6.3-nolibc/x86_64-linux/bin/x86_64-linux-gcc
432 x86_64 : /toolchains/gcc-4.9.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc
433
434
435 You can see that everything is covered, even some strange ones that won't
436 be used (c88 and c99). This is a feature.
437
438
439#. Install new toolchains if needed
440
441 You can download toolchains and update the [toolchain] section of the
442 settings file to find them.
443
444 To make this easier, buildman can automatically download and install
445 toolchains from kernel.org. First list the available architectures::
446
447 $ ./tools/buildman/buildman --fetch-arch list
448 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
449 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
450 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
451 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.2.4/
452 Available architectures: alpha am33_2.0 arm bfin cris crisv32 frv h8300
453 hppa hppa64 i386 ia64 m32r m68k mips mips64 or32 powerpc powerpc64 s390x sh4
454 sparc sparc64 tilegx x86_64 xtensa
455
456 Then pick one and download it::
457
458 $ ./tools/buildman/buildman --fetch-arch or32
459 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.3/
460 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.6.2/
461 Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1/
462 Downloading: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.5.1//x86_64-gcc-4.5.1-nolibc_or32-linux.tar.xz
463 Unpacking to: /home/sjg/.buildman-toolchains
464 Testing
465 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/.'
466 - looking in '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin'
467 - found '/home/sjg/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc'
468 Tool chain test: OK
469
470 Or download them all from kernel.org and move them to /toolchains directory:
471
472 .. code-block:: bash
473
474 ./tools/buildman/buildman --fetch-arch all
475 sudo mkdir -p /toolchains
476 sudo mv ~/.buildman-toolchains/*/* /toolchains/
477
478 For those not available from kernel.org, download from the following links:
479
480 - `Arc Toolchain`_
Simon Glass79cc9be2022-11-09 19:14:43 -0700481
482 Buildman should now be set up to use your new toolchain.
483
484 At the time of writing, U-Boot has these architectures:
485
Simon Glasse93918b2022-11-09 19:14:47 -0700486 arc, arm, m68k, microblaze, mips, nios2, powerpc, sandbox, sh, x86, xtensa
Simon Glass79cc9be2022-11-09 19:14:43 -0700487
Simon Glass79cc9be2022-11-09 19:14:43 -0700488
489How to run it
490-------------
491
492First do a dry run using the -n flag: (replace <branch> with a real, local
493branch with a valid upstream):
494
495.. code-block:: bash
496
497 ./tools/buildman/buildman -b <branch> -n
498
499If it can't detect the upstream branch, try checking out the branch, and
500doing something like 'git branch --set-upstream-to upstream/master'
501or something similar. Buildman will try to guess a suitable upstream branch
502if it can't find one (you will see a message like "Guessing upstream as ...").
503You can also use the -c option to manually specify the number of commits to
504build.
505
506As an example::
507
508 Dry run, so not doing much. But I would do this:
509
510 Building 18 commits for 1059 boards (4 threads, 1 job per thread)
511 Build directory: ../lcd9b
512 5bb3505 Merge branch 'master' of git://git.denx.de/u-boot-arm
513 c18f1b4 tegra: Use const for pinmux_config_pingroup/table()
514 2f043ae tegra: Add display support to funcmux
515 e349900 tegra: fdt: Add pwm binding and node
516 424a5f0 tegra: fdt: Add LCD definitions for Tegra
517 0636ccf tegra: Add support for PWM
518 a994fe7 tegra: Add SOC support for display/lcd
519 fcd7350 tegra: Add LCD driver
520 4d46e9d tegra: Add LCD support to Nvidia boards
521 991bd48 arm: Add control over cachability of memory regions
522 54e8019 lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
523 d92aff7 lcd: Add support for flushing LCD fb from dcache after update
524 dbd0677 tegra: Align LCD frame buffer to section boundary
525 0cff9b8 tegra: Support control of cache settings for LCD
526 9c56900 tegra: fdt: Add LCD definitions for Seaboard
527 5cc29db lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
528 cac5a23 tegra: Enable display/lcd support on Seaboard
529 49ff541 wip
530
531 Total boards to build for each commit: 1059
532
533This shows that it will build all 1059 boards, using 4 threads (because
534we have a 4-core CPU). Each thread will run with -j1, meaning that each
535make job will use a single CPU. The list of commits to be built helps you
536confirm that things look about right. Notice that buildman has chosen a
537'base' directory for you, immediately above your source tree.
538
539Buildman works entirely inside the base directory, here ../lcd9b,
540creating a working directory for each thread, and creating output
541directories for each commit and board.
542
543
544Suggested Workflow
545------------------
546
547To run the build for real, take off the -n:
548
549.. code-block:: bash
550
551 ./tools/buildman/buildman -b <branch>
552
553Buildman will set up some working directories, and get started. After a
554minute or so it will settle down to a steady pace, with a display like this::
555
556 Building 18 commits for 1059 boards (4 threads, 1 job per thread)
557 528 36 124 /19062 -18374 1:13:30 : SIMPC8313_SP
558
559This means that it is building 19062 board/commit combinations. So far it
560has managed to successfully build 528. Another 36 have built with warnings,
561and 124 more didn't build at all. It has 18374 builds left to complete.
562Buildman expects to complete the process in around an hour and a quarter.
563Use this time to buy a faster computer.
564
565
566To find out how the build went, ask for a summary with -s. You can do this
567either before the build completes (presumably in another terminal) or
568afterwards. Let's work through an example of how this is used::
569
570 $ ./tools/buildman/buildman -b lcd9b -s
571 ...
572 01: Merge branch 'master' of git://git.denx.de/u-boot-arm
573 powerpc: + galaxy5200_LOWBOOT
574 02: tegra: Use const for pinmux_config_pingroup/table()
575 03: tegra: Add display support to funcmux
576 04: tegra: fdt: Add pwm binding and node
577 05: tegra: fdt: Add LCD definitions for Tegra
578 06: tegra: Add support for PWM
579 07: tegra: Add SOC support for display/lcd
580 08: tegra: Add LCD driver
581 09: tegra: Add LCD support to Nvidia boards
582 10: arm: Add control over cachability of memory regions
583 11: lcd: Add CONFIG_LCD_ALIGNMENT to select frame buffer alignment
584 12: lcd: Add support for flushing LCD fb from dcache after update
585 arm: + lubbock
586 13: tegra: Align LCD frame buffer to section boundary
587 14: tegra: Support control of cache settings for LCD
588 15: tegra: fdt: Add LCD definitions for Seaboard
589 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
590 17: tegra: Enable display/lcd support on Seaboard
591 18: wip
592
593This shows which commits have succeeded and which have failed. In this case
594the build is still in progress so many boards are not built yet (use -u to
595see which ones). But already we can see a few failures. The galaxy5200_LOWBOOT
596never builds correctly. This could be a problem with our toolchain, or it
597could be a bug in the upstream. The good news is that we probably don't need
598to blame our commits. The bad news is that our commits are not tested on that
599board.
600
601Commit 12 broke lubbock. That's what the '+ lubbock', in red, means. The
602failure is never fixed by a later commit, or you would see lubbock again, in
603green, without the +.
604
605To see the actual error::
606
607 $ ./tools/buildman/buildman -b <branch> -se
608 ...
609 12: lcd: Add support for flushing LCD fb from dcache after update
610 arm: + lubbock
611 +common/libcommon.o: In function `lcd_sync':
612 +common/lcd.c:120: undefined reference to `flush_dcache_range'
613 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
614 +make: *** [build/u-boot] Error 139
615 13: tegra: Align LCD frame buffer to section boundary
616 14: tegra: Support control of cache settings for LCD
617 15: tegra: fdt: Add LCD definitions for Seaboard
618 16: lcd: Add CONFIG_CONSOLE_SCROLL_LINES option to speed console
619 -common/lcd.c:120: undefined reference to `flush_dcache_range'
620 +common/lcd.c:125: undefined reference to `flush_dcache_range'
621 17: tegra: Enable display/lcd support on Seaboard
622 18: wip
623
624So the problem is in lcd.c, due to missing cache operations. This information
625should be enough to work out what that commit is doing to break these
626boards. (In this case pxa did not have cache operations defined).
627
628Note that if there were other boards with errors, the above command would
629show their errors also. Each line is shown only once. So if lubbock and snow
630produce the same error, we just see::
631
632 12: lcd: Add support for flushing LCD fb from dcache after update
633 arm: + lubbock snow
634 +common/libcommon.o: In function `lcd_sync':
635 +common/lcd.c:120: undefined reference to `flush_dcache_range'
636 +arm-none-linux-gnueabi-ld: BFD (Sourcery G++ Lite 2010q1-202) 2.19.51.20090709 assertion fail /scratch/julian/2010q1-release-linux-lite/obj/binutils-src-2010q1-202-arm-none-linux-gnueabi-i686-pc-linux-gnu/bfd/elf32-arm.c:12572
637 +make: *** [build/u-boot] Error 139
638
639But if you did want to see just the errors for lubbock, use:
640
641.. code-block:: bash
642
643 ./tools/buildman/buildman -b <branch> -se lubbock
644
645If you see error lines marked with '-', that means that the errors were fixed
646by that commit. Sometimes commits can be in the wrong order, so that a
647breakage is introduced for a few commits and fixed by later commits. This
648shows up clearly with buildman. You can then reorder the commits and try
649again.
650
651At commit 16, the error moves: you can see that the old error at line 120
652is fixed, but there is a new one at line 126. This is probably only because
653we added some code and moved the broken line further down the file.
654
655As mentioned, if many boards have the same error, then -e will display the
656error only once. This makes the output as concise as possible. To see which
657boards have each error, use -l. So it is safe to omit the board name - you
658will not get lots of repeated output for every board.
659
660Buildman tries to distinguish warnings from errors, and shows warning lines
661separately with a 'w' prefix. Warnings introduced show as yellow. Warnings
662fixed show as cyan.
663
664The full build output in this case is available in::
665
666 ../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
667
668Files:
669
670done
671 Indicates the build was done, and holds the return code from make. This is 0
672 for a good build, typically 2 for a failure.
673
674err
675 Output from stderr, if any. Errors and warnings appear here.
676
677log
678 Output from stdout. Normally there isn't any since buildman runs in silent
679 mode. Use -V to force a verbose build (this passes V=1 to 'make')
680
681toolchain
682 Shows information about the toolchain used for the build.
683
684sizes
685 Shows image size information.
686
687It is possible to get the build binary output there also. Use the -k option
688for this. In that case you will also see some output files, like:
689
690- System.map
691- toolchain
692- u-boot
693- u-boot.bin
694- u-boot.map
695- autoconf.mk
696- SPL/TPL versions like u-boot-spl and u-boot-spl.bin if available
697
698
699Checking Image Sizes
700--------------------
701
702A key requirement for U-Boot is that you keep code/data size to a minimum.
703Where a new feature increases this noticeably it should normally be put
704behind a CONFIG flag so that boards can leave it disabled and keep the image
705size more or less the same with each new release.
706
707To check the impact of your commits on image size, use -S. For example::
708
709 $ ./tools/buildman/buildman -b us-x86 -sS
710 Summary of 10 commits for 1066 boards (4 threads, 1 job per thread)
711 01: MAKEALL: add support for per architecture toolchains
712 02: x86: Add function to get top of usable ram
713 x86: (for 1/3 boards) text -272.0 rodata +41.0
714 03: x86: Add basic cache operations
715 04: x86: Permit bootstage and timer data to be used prior to relocation
716 x86: (for 1/3 boards) data +16.0
717 05: x86: Add an __end symbol to signal the end of the U-Boot binary
718 x86: (for 1/3 boards) text +76.0
719 06: x86: Rearrange the output input to remove BSS
720 x86: (for 1/3 boards) bss -2140.0
721 07: x86: Support relocation of FDT on start-up
722 x86: + coreboot-x86
723 08: x86: Add error checking to x86 relocation code
724 09: x86: Adjust link device tree include file
725 10: x86: Enable CONFIG_OF_CONTROL on coreboot
726
727
728You can see that image size only changed on x86, which is good because this
729series is not supposed to change any other board. From commit 7 onwards the
730build fails so we don't get code size numbers. The numbers are fractional
731because they are an average of all boards for that architecture. The
732intention is to allow you to quickly find image size problems introduced by
733your commits.
734
735Note that the 'text' region and 'rodata' are split out. You should add the
736two together to get the total read-only size (reported as the first column
737in the output from binutil's 'size' utility).
738
739A useful option is --step which lets you skip some commits. For example
740--step 2 will show the image sizes for only every 2nd commit (so it will
741compare the image sizes of the 1st, 3rd, 5th... commits). You can also use
742--step 0 which will compare only the first and last commits. This is useful
743for an overview of how your entire series affects code size. It will build
744only the upstream commit and your final branch commit.
745
746You can also use -d to see a detailed size breakdown for each board. This
747list is sorted in order from largest growth to largest reduction.
748
749It is even possible to go a little further with the -B option (--bloat). This
750shows where U-Boot has bloated, breaking the size change down to the function
751level. Example output is below::
752
753 $ ./tools/buildman/buildman -b us-mem4 -sSdB
754 ...
755 19: Roll crc32 into hash infrastructure
756 arm: (for 10/10 boards) all -143.4 bss +1.2 data -4.8 rodata -48.2 text -91.6
757 paz00 : all +23 bss -4 rodata -29 text +56
758 u-boot: add: 1/0, grow: 3/-2 bytes: 168/-104 (64)
759 function old new delta
760 hash_command 80 160 +80
761 crc32_wd_buf - 56 +56
762 ext4fs_read_file 540 568 +28
763 insert_var_value_sub 688 692 +4
764 run_list_real 1996 1992 -4
765 do_mem_crc 168 68 -100
766 trimslice : all -9 bss +16 rodata -29 text +4
767 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
768 function old new delta
769 hash_command 80 160 +80
770 crc32_wd_buf - 56 +56
771 ext4fs_iterate_dir 672 668 -4
772 ext4fs_read_file 568 548 -20
773 do_mem_crc 168 68 -100
774 whistler : all -9 bss +16 rodata -29 text +4
775 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
776 function old new delta
777 hash_command 80 160 +80
778 crc32_wd_buf - 56 +56
779 ext4fs_iterate_dir 672 668 -4
780 ext4fs_read_file 568 548 -20
781 do_mem_crc 168 68 -100
782 seaboard : all -9 bss -28 rodata -29 text +48
783 u-boot: add: 1/0, grow: 3/-2 bytes: 160/-104 (56)
784 function old new delta
785 hash_command 80 160 +80
786 crc32_wd_buf - 56 +56
787 ext4fs_read_file 548 568 +20
788 run_list_real 1996 2000 +4
789 do_nandboot 760 756 -4
790 do_mem_crc 168 68 -100
791 colibri_t20 : all -9 rodata -29 text +20
792 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-112 (28)
793 function old new delta
794 hash_command 80 160 +80
795 crc32_wd_buf - 56 +56
796 read_abs_bbt 204 208 +4
797 do_nandboot 760 756 -4
798 ext4fs_read_file 576 568 -8
799 do_mem_crc 168 68 -100
800 ventana : all -37 bss -12 rodata -29 text +4
801 u-boot: add: 1/0, grow: 1/-3 bytes: 136/-124 (12)
802 function old new delta
803 hash_command 80 160 +80
804 crc32_wd_buf - 56 +56
805 ext4fs_iterate_dir 672 668 -4
806 ext4fs_read_file 568 548 -20
807 do_mem_crc 168 68 -100
808 harmony : all -37 bss -16 rodata -29 text +8
809 u-boot: add: 1/0, grow: 2/-3 bytes: 140/-124 (16)
810 function old new delta
811 hash_command 80 160 +80
812 crc32_wd_buf - 56 +56
813 nand_write_oob_syndrome 428 432 +4
814 ext4fs_iterate_dir 672 668 -4
815 ext4fs_read_file 568 548 -20
816 do_mem_crc 168 68 -100
817 medcom-wide : all -417 bss +28 data -16 rodata -93 text -336
818 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
819 function old new delta
820 crc32_wd_buf - 56 +56
821 do_fat_read_at 2872 2904 +32
822 hash_algo 16 - -16
823 do_mem_crc 168 68 -100
824 hash_command 420 160 -260
825 tec : all -449 bss -4 data -16 rodata -93 text -336
826 u-boot: add: 1/-1, grow: 1/-2 bytes: 88/-376 (-288)
827 function old new delta
828 crc32_wd_buf - 56 +56
829 do_fat_read_at 2872 2904 +32
830 hash_algo 16 - -16
831 do_mem_crc 168 68 -100
832 hash_command 420 160 -260
833 plutux : all -481 bss +16 data -16 rodata -93 text -388
834 u-boot: add: 1/-1, grow: 1/-3 bytes: 68/-408 (-340)
835 function old new delta
836 crc32_wd_buf - 56 +56
837 do_load_serial_bin 1688 1700 +12
838 hash_algo 16 - -16
839 do_fat_read_at 2904 2872 -32
840 do_mem_crc 168 68 -100
841 hash_command 420 160 -260
842 powerpc: (for 5/5 boards) all +37.4 data -3.2 rodata -41.8 text +82.4
843 MPC8610HPCD : all +55 rodata -29 text +84
844 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
845 function old new delta
846 hash_command - 176 +176
847 do_mem_crc 184 88 -96
848 MPC8641HPCN : all +55 rodata -29 text +84
849 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
850 function old new delta
851 hash_command - 176 +176
852 do_mem_crc 184 88 -96
853 MPC8641HPCN_36BIT: all +55 rodata -29 text +84
854 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
855 function old new delta
856 hash_command - 176 +176
857 do_mem_crc 184 88 -96
858 sbc8641d : all +55 rodata -29 text +84
859 u-boot: add: 1/0, grow: 0/-1 bytes: 176/-96 (80)
860 function old new delta
861 hash_command - 176 +176
862 do_mem_crc 184 88 -96
863 xpedite517x : all -33 data -16 rodata -93 text +76
864 u-boot: add: 1/-1, grow: 0/-1 bytes: 176/-112 (64)
865 function old new delta
866 hash_command - 176 +176
867 hash_algo 16 - -16
868 do_mem_crc 184 88 -96
869 ...
870
871
872This shows that commit 19 has reduced codesize for arm slightly and increased
873it for powerpc. This increase was offset in by reductions in rodata and
874data/bss.
875
876Shown below the summary lines are the sizes for each board. Below each board
877are the sizes for each function. This information starts with:
878
879add
880 number of functions added / removed
881
882grow
883 number of functions which grew / shrunk
884
885bytes
886 number of bytes of code added to / removed from all functions, plus the total
887 byte change in brackets
888
889The change seems to be that hash_command() has increased by more than the
890do_mem_crc() function has decreased. The function sizes typically add up to
891roughly the text area size, but note that every read-only section except
892rodata is included in 'text', so the function total does not exactly
893correspond.
894
895It is common when refactoring code for the rodata to decrease as the text size
896increases, and vice versa.
897
898
899.. _buildman_settings:
900
901The .buildman settings file
902---------------------------
903
904The .buildman file provides information about the available toolchains and
905also allows build flags to be passed to 'make'. It consists of several
906sections, with the section name in square brackets. Within each section are
907a set of (tag, value) pairs.
908
Tom Rini93ebd462022-11-09 19:14:53 -0700909'[global]' section
910 allow-missing
911 Indicates the policy to use for missing blobs. Note that the flags
912 ``--allow-missing`` (``-M``) and ``--no-allow-missing`` (``--no-a``)
913 override these setting.
914
915 always
916 Run with ``-M`` by default.
917
918 multiple
919 Run with ``-M`` if more than one board is being built.
920
921 branch
922 Run with ``-M`` if a branch is being built.
923
924 Note that the last two can be given together::
925
926 allow-missing = multiple branch
927
Simon Glass79cc9be2022-11-09 19:14:43 -0700928'[toolchain]' section
929 This lists the available toolchains. The tag here doesn't matter, but
930 make sure it is unique. The value is the path to the toolchain. Buildman
931 will look in that path for a file ending in 'gcc'. It will then execute
932 it to check that it is a C compiler, passing only the --version flag to
933 it. If the return code is 0, buildman assumes that it is a valid C
934 compiler. It uses the first part of the name as the architecture and
935 strips off the last part when setting the CROSS_COMPILE environment
936 variable (parts are delimited with a hyphen).
937
938 For example powerpc-linux-gcc will be noted as a toolchain for 'powerpc'
939 and CROSS_COMPILE will be set to powerpc-linux- when using it.
940
941'[toolchain-alias]' section
942 This converts toolchain architecture names to U-Boot names. For example,
943 if an x86 toolchains is called i386-linux-gcc it will not normally be
944 used for architecture 'x86'. Adding 'x86: i386 x86_64' to this section
945 will tell buildman that the i386 and x86_64 toolchains can be used for
946 the x86 architecture.
947
948'[make-flags]' section
949 U-Boot's build system supports a few flags (such as BUILD_TAG) which
950 affect the build product. These flags can be specified in the buildman
951 settings file. They can also be useful when building U-Boot against other
952 open source software.
953
954 [make-flags]
955 at91-boards=ENABLE_AT91_TEST=1
956 snapper9260=${at91-boards} BUILD_TAG=442
957 snapper9g45=${at91-boards} BUILD_TAG=443
958
959 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260
960 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special
961 variable ${target} is available to access the target name (snapper9260
962 and snapper9g20 in this case). Variables are resolved recursively. Note
963 that variables can only contain the characters A-Z, a-z, 0-9, hyphen (-)
964 and underscore (_).
965
966 It is expected that any variables added are dealt with in U-Boot's
967 config.mk file and documented in the README.
968
969 Note that you can pass ad-hoc options to the build using environment
970 variables, for example:
971
972 SOME_OPTION=1234 ./tools/buildman/buildman my_board
973
974
975Quick Sanity Check
976------------------
977
978If you have made changes and want to do a quick sanity check of the
979currently checked-out source, run buildman without the -b flag. This will
980build the selected boards and display build status as it runs (i.e. -v is
981enabled automatically). Use -e to see errors/warnings as well.
982
983
984Building Ranges
985---------------
986
987You can build a range of commits by specifying a range instead of a branch
988when using the -b flag. For example::
989
990 buildman -b upstream/master..us-buildman
991
992will build commits in us-buildman that are not in upstream/master.
993
994
995Building Faster
996---------------
997
998By default, buildman doesn't execute 'make mrproper' prior to building the
999first commit for each board. This reduces the amount of work 'make' does, and
1000hence speeds up the build. To force use of 'make mrproper', use -the -m flag.
1001This flag will slow down any buildman invocation, since it increases the amount
1002of work done on any build.
1003
1004One possible application of buildman is as part of a continual edit, build,
1005edit, build, ... cycle; repeatedly applying buildman to the same change or
1006series of changes while making small incremental modifications to the source
1007each time. This provides quick feedback regarding the correctness of recent
1008modifications. In this scenario, buildman's default choice of build directory
1009causes more build work to be performed than strictly necessary.
1010
1011By default, each buildman thread uses a single directory for all builds. When a
1012thread builds multiple boards, the configuration built in this directory will
1013cycle through various different configurations, one per board built by the
1014thread. Variations in the configuration will force a rebuild of affected source
1015files when a thread switches between boards. Ideally, such buildman-induced
1016rebuilds would not happen, thus allowing the build to operate as efficiently as
1017the build system and source changes allow. buildman's -P flag may be used to
1018enable this; -P causes each board to be built in a separate (board-specific)
1019directory, thus avoiding any buildman-induced configuration changes in any
1020build directory.
1021
1022U-Boot's build system embeds information such as a build timestamp into the
1023final binary. This information varies each time U-Boot is built. This causes
1024various files to be rebuilt even if no source changes are made, which in turn
1025requires that the final U-Boot binary be re-linked. This unnecessary work can
1026be avoided by turning off the timestamp feature. This can be achieved by
1027setting the SOURCE_DATE_EPOCH environment variable to 0.
1028
1029Combining all of these options together yields the command-line shown below.
1030This will provide the quickest possible feedback regarding the current content
1031of the source tree, thus allowing rapid tested evolution of the code::
1032
1033 SOURCE_DATE_EPOCH=0 ./tools/buildman/buildman -P tegra
1034
1035
1036Checking configuration
1037----------------------
1038
1039A common requirement when converting CONFIG options to Kconfig is to check
1040that the effective configuration has not changed due to the conversion.
1041Buildman supports this with the -K option, used after a build. This shows
1042differences in effective configuration between one commit and the next.
1043
1044For example::
1045
1046 $ buildman -b kc4 -sK
1047 ...
1048 43: Convert CONFIG_SPL_USBETH_SUPPORT to Kconfig
1049 arm:
1050 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1
1051 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1
1052 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1
1053 am335x_evm_usbspl :
1054 + u-boot.cfg: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_NET=1
1055 + u-boot-spl.cfg: CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1
1056 + all: CONFIG_SPL_ENV_SUPPORT=1 CONFIG_SPL_MMC=1 CONFIG_SPL_NAND_SUPPORT=1 CONFIG_SPL_NET=1
1057 44: Convert CONFIG_SPL_USB_HOST to Kconfig
1058 ...
1059
1060This shows that commit 44 enabled three new options for the board
1061am335x_evm_usbspl which were not enabled in commit 43. There is also a
1062summary for 'arm' showing all the changes detected for that architecture.
1063In this case there is only one board with changes, so 'arm' output is the
1064same as 'am335x_evm_usbspl'/
1065
1066The -K option uses the u-boot.cfg, spl/u-boot-spl.cfg and tpl/u-boot-tpl.cfg
1067files which are produced by a build. If all you want is to check the
1068configuration you can in fact avoid doing a full build, using -D. This tells
1069buildman to configuration U-Boot and create the .cfg files, but not actually
1070build the source. This is 5-10 times faster than doing a full build.
1071
1072By default buildman considers the follow two configuration methods
1073equivalent::
1074
1075 #define CONFIG_SOME_OPTION
1076
1077 CONFIG_SOME_OPTION=y
1078
1079The former would appear in a header filer and the latter in a defconfig
1080file. The achieve this, buildman considers 'y' to be '1' in configuration
1081variables. This avoids lots of useless output when converting a CONFIG
1082option to Kconfig. To disable this behaviour, use --squash-config-y.
1083
1084
1085Checking the environment
1086------------------------
1087
1088When converting CONFIG options which manipulate the default environment,
1089a common requirement is to check that the default environment has not
1090changed due to the conversion. Buildman supports this with the -U option,
1091used after a build. This shows differences in the default environment
1092between one commit and the next.
1093
1094For example::
1095
1096 $ buildman -b squash brppt1 -sU
1097 Summary of 2 commits for 3 boards (3 threads, 3 jobs per thread)
1098 01: Migrate bootlimit to Kconfig
1099 02: Squashed commit of the following:
1100 c brppt1_mmc: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1101 c brppt1_spi: altbootcmd=mmc dev 1; run mmcboot0; -> mmc dev 1; run mmcboot0
1102 + brppt1_nand: altbootcmd=run usbscript
1103 - brppt1_nand: altbootcmd=run usbscript
1104 (no errors to report)
1105
1106This shows that commit 2 modified the value of 'altbootcmd' for 'brppt1_mmc'
1107and 'brppt1_spi', removing a trailing semicolon. 'brppt1_nand' gained an a
1108value for 'altbootcmd', but lost one for ' altbootcmd'.
1109
1110The -U option uses the u-boot.env files which are produced by a build.
Simon Glasse96f70a2023-02-21 12:40:26 -07001111Internally, buildman writes out an out-env file into the build directory for
1112later comparison.
Simon Glass79cc9be2022-11-09 19:14:43 -07001113
1114
1115Building with clang
1116-------------------
1117
1118To build with clang (sandbox only), use the -O option to override the
1119toolchain. For example:
1120
1121.. code-block:: bash
1122
1123 buildman -O clang-7 --board sandbox
1124
1125
Simon Glassf6bfcca2023-02-21 12:40:28 -07001126Building without LTO
1127--------------------
1128
1129Link-time optimisation (LTO) is designed to reduce code size by globally
1130optimising the U-Boot build. Unfortunately this can dramatically slow down
1131builds. This is particularly noticeable when running a lot of builds.
1132
1133Use the -L (--no-lto) flag to disable LTO.
1134
1135.. code-block:: bash
1136
1137 buildman -L --board sandbox
1138
1139
Simon Glass79cc9be2022-11-09 19:14:43 -07001140Doing a simple build
1141--------------------
1142
1143In some cases you just want to build a single board and get the full output, use
1144the -w option, for example:
1145
1146.. code-block:: bash
1147
1148 buildman -o /tmp/build --board sandbox -w
1149
1150This will write the full build into /tmp/build including object files. You must
1151specify the output directory with -o when using -w.
1152
1153
1154Support for IDEs (Integrated Development Environments)
1155------------------------------------------------------
1156
1157Normally buildman summarises the output and shows information indicating the
1158meaning of each line of output. For example a '+' symbol appears at the start of
1159each error line. Also, buildman prints information about what it is about to do,
1160along with a summary at the end.
1161
1162When using buildman from an IDE, it is helpful to drop this behaviour. Use the
1163-I/--ide option for that. You might find -W helpful also so that warnings do
1164not cause the build to fail:
1165
1166.. code-block:: bash
1167
1168 buildman -o /tmp/build --board sandbox -wWI
1169
1170
Tom Rini93ebd462022-11-09 19:14:53 -07001171Support for binary blobs
1172------------------------
1173
1174U-Boot is moving to using Binman (see :doc:`../develop/package/binman`) for
1175dealing with the complexities of packaging U-Boot along with binary files from
1176other projects. These are called 'external blobs' by Binman.
1177
1178Typically a missing external blob causes a build failure. For build testing of
1179a lot of boards, or boards for which you do not have the blobs, you can use the
1180-M flag to allow missing blobs. This marks the build as if it succeeded,
1181although with warnings shown, including 'Some images are invalid'. If any boards
1182fail in this way, buildman exits with status 101.
1183
1184To convert warnings to errors, use -E. To make buildman return success with
1185these warnings, use -W.
1186
1187It is generally safe to default to enabling -M for all runs of buildman, so long
1188as you check the exit code. To do this, add::
1189
1190 allow-missing = "always"
1191
1192to the top of the buildman_settings_ file.
1193
1194
Simon Glass79cc9be2022-11-09 19:14:43 -07001195Changing the configuration
1196--------------------------
1197
1198Sometimes it is useful to change the CONFIG options for a build on the fly. This
1199can be used to build a board (or multiple) with a few changes to see the impact.
1200The -a option supports this:
1201
1202.. code-block:: bash
1203
1204 -a <cfg>
1205
1206where <cfg> is a CONFIG option (with or without the `CONFIG_` prefix) to enable.
1207For example:
1208
1209.. code-block:: bash
1210
1211 buildman -a CMD_SETEXPR_FMT
1212
1213will build with CONFIG_CMD_SETEXPR_FMT enabled.
1214
1215You can disable options by preceding them with tilde (~). You can specify the
1216-a option multiple times:
1217
1218.. code-block:: bash
1219
1220 buildman -a CMD_SETEXPR_FMT -a ~CMDLINE
1221
1222Some options have values, in which case you can change them:
1223
1224.. code-block:: bash
1225
1226 buildman -a 'BOOTCOMMAND="echo hello"' CONFIG_SYS_LOAD_ADDR=0x1000
1227
1228Note that you must put quotes around string options and the whole thing must be
1229in single quotes, to make sure the shell leave it alone.
1230
1231If you try to set an option that does not exist, or that cannot be changed for
1232some other reason (e.g. it is 'selected' by another option), then buildman
1233shows an error::
1234
1235 $ buildman --board sandbox -a FRED
1236 Building current source for 1 boards (1 thread, 32 jobs per thread)
1237 0 0 0 /1 -1 (starting)errs
1238 Some CONFIG adjustments did not take effect. This may be because
1239 the request CONFIGs do not exist or conflict with others.
1240
1241 Failed adjustments:
1242
1243 FRED Missing expected line: CONFIG_FRED=y
1244
1245
1246One major caveat with this feature with branches (-b) is that buildman does not
1247name the output directories differently when you change the configuration, so
1248doing the same build again with different configuration will not trigger a
1249rebuild. You can use -f to work around that.
1250
1251
1252Other options
1253-------------
1254
1255Buildman has various other command-line options. Try --help to see them.
1256
1257To find out what toolchain prefix buildman will use for a build, use the -A
1258option.
1259
1260To request that compiler warnings be promoted to errors, use -E. This passes the
1261-Werror flag to the compiler. Note that the build can still produce warnings
1262with -E, e.g. the migration warnings::
1263
1264 ===================== WARNING ======================
1265 This board does not use CONFIG_DM_MMC. Please update
1266 ...
1267 ====================================================
1268
1269When doing builds, Buildman's return code will reflect the overall result::
1270
1271 0 (success) No errors or warnings found
1272 100 Errors found
1273 101 Warnings found (only if no -W)
1274
1275You can use -W to tell Buildman to return 0 (success) instead of 101 when
1276warnings are found. Note that it can be useful to combine -E and -W. This means
1277that all compiler warnings will produce failures (code 100) and all other
1278warnings will produce success (since 101 is changed to 0).
1279
1280If there are both warnings and errors, errors win, so buildman returns 100.
1281
1282The -y option is provided (for use with -s) to ignore the bountiful device-tree
1283warnings. Similarly, -Y tells buildman to ignore the migration warnings.
1284
1285Sometimes you might get an error in a thread that is not handled by buildman,
1286perhaps due to a failure of a tool that it calls. You might see the output, but
1287then buildman hangs. Failing to handle any eventuality is a bug in buildman and
1288should be reported. But you can use -T0 to disable threading and hopefully
1289figure out the root cause of the build failure.
1290
1291Build summary
1292-------------
1293
1294When buildman finishes it shows a summary, something like this::
1295
1296 Completed: 5 total built, duration 0:00:21, rate 0.24
1297
1298This shows that a total of 5 builds were done across all selected boards, it
1299took 21 seconds and the builds happened at the rate of 0.24 per second. The
1300latter number depends on the speed of your machine and the efficiency of the
1301U-Boot build.
1302
1303
Simon Glass79cc9be2022-11-09 19:14:43 -07001304Using boards.cfg
1305----------------
1306
1307This file is no-longer needed by buildman but it is still generated in the
1308working directory. This helps avoid a delay on every build, since scanning all
1309the Kconfig files takes a few seconds. Use the -R flag to force regeneration
1310of the file - in that case buildman exits after writing the file. with exit code
13112 if there was an error in the maintainer files.
1312
1313You should use 'buildman -nv <criteria>' instead of greoing the boards.cfg file,
1314since it may be dropped altogether in future.
1315
1316
Simon Glass1382b1d2023-02-21 12:40:27 -07001317Checking the command
1318--------------------
1319
1320Buildman writes out the toolchain information to a `toolchain` file within the
1321output directory. It also writes the commands used to build U-Boot in an
1322`out-cmd` file. You can check these if you suspect something strange is
1323happening.
1324
Simon Glass79cc9be2022-11-09 19:14:43 -07001325TODO
1326----
1327
1328Many improvements have been made over the years. There is still quite a bit of
1329scope for more though, e.g.:
1330
1331- easier access to log files
1332- 'hunting' for problems, perhaps by building a few boards for each arch, or
1333 checking commits for changed files and building only boards which use those
1334 files
1335
1336
1337Credits
1338-------
1339
1340Thanks to Grant Grundler <grundler@chromium.org> for his ideas for improving
1341the build speed by building all commits for a board instead of the other
1342way around.
1343
Simon Glassc325dbe2022-11-09 19:14:45 -07001344.. _`Arc Toolchain`: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2021.03-release/arc_gnu_2021.03_prebuilt_elf32_le_linux_install.tar.gz
Simon Glass79cc9be2022-11-09 19:14:43 -07001345
1346.. sectionauthor:: Simon Glass
1347.. sectionauthor:: Copyright (c) 2013 The Chromium OS Authors.
1348.. sectionauthor:: sjg@chromium.org
1349.. Halloween 2012
1350.. Updated 12-12-12
1351.. Updated 23-02-13
1352.. Updated 09-04-20