| Simon Glass | d8e3635 | 2016-07-03 09:40:33 -0600 | [diff] [blame^] | 1 | Testing in U-Boot |
| 2 | ================= |
| 3 | |
| 4 | U-Boot has a large amount of code. This file describes how this code is |
| 5 | tested and what tests you should write when adding a new feature. |
| 6 | |
| 7 | |
| 8 | Sandbox |
| 9 | ------- |
| 10 | U-Boot can be built as a user-space application (e.g. for Linux). This |
| 11 | allows test to be executed without needing target hardware. The 'sandbox' |
| 12 | target provides this feature and it is widely used in tests. |
| 13 | |
| 14 | |
| 15 | Pytest Suite |
| 16 | ------------ |
| 17 | |
| 18 | Many tests are available using the pytest suite, in test/py. This can run |
| 19 | either on sandbox or on real hardware. It relies on the U-Boot console to |
| 20 | inject test commands and check the result. It is slower to run than C code, |
| 21 | but provides the ability to unify lots of test and summarise their results. |
| 22 | |
| 23 | You can run the tests on sandbox with: |
| 24 | |
| 25 | ./test/py/test.py --bd sandbox --build |
| 26 | |
| 27 | This will produce HTML output in build-sandbox/test-log.html |
| 28 | |
| 29 | See test/py/README.md for more information about the pytest suite. |
| 30 | |
| 31 | |
| 32 | tbot |
| 33 | ---- |
| 34 | |
| 35 | Tbot provides a way to execute tests on target hardware. It is intended for |
| 36 | trying out both U-Boot and Linux (and potentially other software) on a |
| 37 | number of boards automatically. It can be used to create a continuous test |
| 38 | environment. See tools/tbot/README for more information. |
| 39 | |
| 40 | |
| 41 | Ad-hoc tests |
| 42 | ------------ |
| 43 | |
| 44 | There are several ad-hoc tests which run outside the pytest environment: |
| 45 | |
| 46 | test/fs - File system test (shell script) |
| 47 | test/image - FIT and lagacy image tests (shell script and Python) |
| 48 | test/stdint - A test that stdint.h can be used in U-Boot (shell script) |
| 49 | trace - Test for the tracing feature (shell script) |
| 50 | vboot - Test for verified boot (shell script) |
| 51 | |
| 52 | The above should be converted to run as part of the pytest suite. |
| 53 | |
| 54 | |
| 55 | When to write tests |
| 56 | ------------------- |
| 57 | |
| 58 | If you add code to U-Boot without a test you are taking a risk. Even if you |
| 59 | perform thorough manual testing at the time of submission, it may break when |
| 60 | future changes are made to U-Boot. It may even break when applied to mainline, |
| 61 | if other changes interact with it. A good mindset is that untested code |
| 62 | probably doesn't work and should be deleted. |
| 63 | |
| 64 | You can assume that the Pytest suite will be run before patches are accepted |
| 65 | to mainline, so this provides protection against future breakage. |
| 66 | |
| 67 | On the other hand there is quite a bit of code that is not covered with tests, |
| 68 | or is covered sparingly. So here are some suggestions: |
| 69 | |
| 70 | - If you are adding a new uclass, add a sandbox driver and a test that uses it |
| 71 | - If you are modifying code covered by an existing test, add a new test case |
| 72 | to cover your changes |
| 73 | - If the code you are modifying has not tests, consider writing one. Even a |
| 74 | very basic test is useful, and may be picked up and enhanced by others. It |
| 75 | is much easier to add onto a test - writing a new large test can seem |
| 76 | daunting to most contributors. |
| 77 | |
| 78 | |
| 79 | Future work |
| 80 | ----------- |
| 81 | |
| 82 | Converting existing shell scripts into pytest tests. |