Skip to content

U-Boot integration [WIP]#1411

Open
sjg20 wants to merge 93 commits intolabgrid-project:masterfrom
sjg20:u-boot-integration
Open

U-Boot integration [WIP]#1411
sjg20 wants to merge 93 commits intolabgrid-project:masterfrom
sjg20:u-boot-integration

Conversation

@sjg20
Copy link
Copy Markdown
Contributor

@sjg20 sjg20 commented May 21, 2024

Cover the drivers, scripts, pytest and the Gitlab information.

Note: This PR exists just so others can replicate my lab. I have created
separate PRs for features and will dribble those in as they get reviewed:

#1482
#1483
#1484
#1571

Cover-letter:
U-Boot integration
END

Changes in v9:

  • Rebase on upstream
  • Add ~/.lgrc support for site-wide labgrid defaults
  • Add automatic file logging via LG_LOG_DIR for both client and pytest
  • Add timestamps and CI_JOB_ID to auto-log filenames
  • Make log directories world-writable for multi-user sharing
  • Enable automatic labgrid-client logging in lg-env
  • Wait for USB resources instead of using a fixed sleep
  • Extract _prepare_for_send() from _reset_for_send()
  • Add get_send_driver_name() for USB resource discovery
  • Reopen console when power-cycling for send
  • Increase USB resource timeout and add power_off_delay
  • Add support for the Badgerd SDWire mux
  • Add QEMU disk-image and writer-arguments support for EFI
  • Use snapshot mode for the QEMU EFI root disk
  • Add check_lab.py for board reliability checking

Changes in v8:

  • Fairly minor expansions to deal with new use cases

Changes in v6:

  • Fix reference to crossbar in _ub-bisect-try and ub-smoke
  • Add -x option to specify the extra board directory
  • Add comments to the example lg-env
  • Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
  • Add a few more comments to ub-pyt
  • Show the build path with ub-pyt -v
  • Add --allow-unmatched to ub-smoke to avoid QEMU error
  • Add a new section on compatibility
  • Add a new section on script usage

Changes in v5:

  • Rebase on latest grpc branch
  • Fix pylint errors and warnings

Changes in v4:

  • Support for Beagleplay, which needs files from two separate U-Boot
    builds
  • Support for a 'recovery' button needed to boot the image
  • Tidy up the internal-console support
  • Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab
    mode)

Changes in v3:

  • Rebase on top of grpc branch
  • Don't mess with terminal setting unless stdin is a terminal
  • Don't show an error if there are no resources when auto-acquiring
  • Support QEMU in UBootWriter

Some changes in v2:

  • Rationalise the flags for the U-Boot scripts
  • Support tracing with em100
  • Support an internal terminal instead of microcom
  • Add a -D flag for debugging
  • Support send-only boards
  • Add a way to build the U-Boot config
  • Add a control for buildman's process-limit
  • allow the build-dir to be specified in a variable
  • add documentation about U-Boot-pytest integration
  • add source_dir and config_file to UBootProviderDriver
  • add an internal terminal
  • expand the U-Boot scripts
  • significantly improve the U-Boot-pytest integration

@sjg20 sjg20 force-pushed the u-boot-integration branch 2 times, most recently from 4cdfb54 to c4b13af Compare June 10, 2024 22:14
@sjg20 sjg20 changed the title U boot integration [WIP] U-Boot integration [WIP] Jun 25, 2024
@sjg20 sjg20 force-pushed the u-boot-integration branch from c4b13af to 9b99e1a Compare June 25, 2024 12:28
@sjg20
Copy link
Copy Markdown
Contributor Author

sjg20 commented Jun 25, 2024

Changes for v3:

  • Rebase on top of grpc branch
  • Don't mess with terminal setting unless stdin is a terminal
  • Don't show an error if there are no resources when auto-acquiring
  • Support QEMU in UBootWriter

trini added a commit to trini/u-boot-backup that referenced this pull request Jul 9, 2024
Simon Glass <sjg@chromium.org> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrote a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html
@sjg20 sjg20 force-pushed the u-boot-integration branch from 9b99e1a to 743e46d Compare July 29, 2024 02:22
@sjg20 sjg20 requested a review from krevsbech as a code owner July 29, 2024 02:22
@sjg20
Copy link
Copy Markdown
Contributor Author

sjg20 commented Jul 29, 2024

Changes in v4:

  • Support for Beagleplay, which needs files from two separate U-Boot
    builds
  • Support for a 'recovery' button needed to boot the image
  • Tidy up the internal-console support
  • Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab
    mode)

@sjg20 sjg20 force-pushed the u-boot-integration branch 3 times, most recently from 8f0e69e to eb89445 Compare August 16, 2024 14:21
@sjg20 sjg20 force-pushed the u-boot-integration branch from eb89445 to 8fbcc19 Compare August 29, 2024 16:38
trini added a commit to trini/u-boot-backup that referenced this pull request Nov 13, 2024
Simon Glass <sjg@chromium.org> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrong a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect <target>
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html

Link: https://lore.kernel.org/r/20241112141326.643128-1-sjg@chromium.org
trini added a commit to trini/u-boot-backup that referenced this pull request Nov 13, 2024
Simon Glass <sjg@chromium.org> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrong a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect <target>
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html

Link: https://lore.kernel.org/r/20241112141326.643128-1-sjg@chromium.org
trini added a commit to trini/u-boot-backup that referenced this pull request Nov 13, 2024
Simon Glass <sjg@chromium.org> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrong a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect <target>
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html

Link: https://lore.kernel.org/r/20241112141326.643128-1-sjg@chromium.org
[trini: Move the sjg-lab job to prior to world build, to fix pipeline
        status]
Signed-off-by: Tom Rini <trini@konsulko.com>
trini added a commit to trini/u-boot-backup that referenced this pull request Nov 13, 2024
Simon Glass <sjg@chromium.org> says:

Labgrid provides access to a hardware lab in an automated way. It is
possible to boot U-Boot on boards in the lab without physically touching
them. It relies on relays, USB UARTs and SD muxes, among other things.

By way of background, about 4 years ago I wrong a thing called Labman[1]
which allowed my lab of about 30 devices to be operated remotely, using
tbot for the console and build integration. While it worked OK and I
used it for many bisects, I didn't take it any further.

It turns out that there was already an existing program, called Labgrid,
which I did not know about at time (thank you Tom for telling me). It is
more rounded than Labman and has a number of advantages:

- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices

It lacks a 'lab check' feature and a few other things, but these can be
remedied.

On and off over the past several weeks I have been experimenting with
Labgrid. I have managed to create an initial U-Boot integration (this
series) by adding various features to Labgrid[2] and the U-Boot test
hooks.

I hope that this might inspire others to set up boards and run tests
automatically, rather than relying on infrequent, manual test. Perhaps
it may even be possible to have a number of labs available.

Included in the integration are a number of simple scripts which make it
easy to connect to boards and run tests:

ub-int <target>
    Build and boot on a target, starting an interactive session

ub-cli <target>
    Build and boot on a target, ensure U-Boot starts and provide an interactive
    session from there

ub-smoke <target>
    Smoke test U-Boot to check that it boots to a prompt on a target

ub-bisect <target>
    Bisect a git tree to locate a failure on a particular target

ub-pyt <target> <testspec>
    Run U-Boot pytests on a target

Some of these help to provide the same tbot[4] workflow which I have
relied on for several years, albeit much simpler versions.

The goal here is to create some sort of script which can collect
patches from the mailing list, apply them and test them on a selection
of boards. I suspect that script already exists, so please let me know
what you suggest.

I hope you find this interesting and take a look!

[1] https://github.com/sjg20/u-boot/tree/lab6a
[2] labgrid-project/labgrid#1411
[3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
[4] https://tbot.tools/index.html

Link: https://lore.kernel.org/r/20241112141326.643128-1-sjg@chromium.org
[trini: Move the sjg-lab job to prior to world build, to fix pipeline
        status]
Signed-off-by: Tom Rini <trini@konsulko.com>
trini added a commit to trini/u-boot-backup that referenced this pull request Jan 25, 2025
…e sjg lab"

Simon Glass <sjg@chromium.org> says:

This board fought valiantly against attempts to add it to my lab. After
several hours of debugging, I found problems in the Labgrid integration
(not included here), test.py and buildman

This series fixes these and the board now seems to be reliable enough.

Note that the board fails test_dm_compat

Link: labgrid-project/labgrid#1411
Link: https://lore.kernel.org/r/20241214182024.1954974-1-sjg@chromium.org
@sjg20 sjg20 force-pushed the u-boot-integration branch from dc97a27 to a91429a Compare February 21, 2025 20:18
@sjg20 sjg20 force-pushed the u-boot-integration branch from a91429a to 89a6fc5 Compare July 23, 2025 13:46
sjg20 added a commit to sjg20/u-boot-concept that referenced this pull request Jan 19, 2026
…e sjg lab"

Simon Glass <sjg@chromium.org> says:

This board fought valiantly against attempts to add it to my lab. After
several hours of debugging, I found problems in the Labgrid integration
(not included here), test.py and buildman

This series fixes these and the board now seems to be reliable enough.

Note that the board fails test_dm_compat

Link: labgrid-project/labgrid#1411
Link: https://lore.kernel.org/r/20241214182024.1954974-1-sjg@chromium.org

(cherry picked from commit 292278d6828e6c8c6ae11bc8658586d1acc411d4)

[Empty commit: All individual commits from this merge are already applied in ci/master:
 - ba87a2fcfbf -> 2f2000e (test/py: Provide the correct U_BOOT_SOURCE_DIR to getrole)
 - 64d5a0550a1 -> 171ebcc (test/py: Handle u-boot-test-getrole failure)
 - dd7138dde77 -> e22893f (buildman: Drop unused OUTPUT_FILE constant)
 - bd7883f3467 -> 0452cf1 (buildman: Record an error if a toolchain is missing)
 - 1135dc50a3e -> 6dd2171 (gitlab: Add a StarFive VisionFive2 to the sjg lab)]
sjg20 added 8 commits April 2, 2026 13:34
When running a process it is sometimes necessary to pass environment
variables and to run the process in a particular directory. Add support
for these features.

Series-changes: 8
- Enable logging of the cwd

Signed-off-by: Simon Glass <sjg@chromium.org>
It isn't generally useful to print out every command before executing
it. Now that the client shows failed commands fully, it seems better to
change this to show only when debugging.

Signed-off-by: Simon Glass <sjg@chromium.org>
When labgrid is invoked by a GitLab CI runner or other automation,
there is no shell script like lg-env to set environment variables
such as LG_LOG_DIR. Rather than requiring every CI config to set
these, read defaults from ~/.lgrc at startup.

The file uses a simple key=value format with comment support.
Existing environment variables are not overridden, so explicit
settings always take precedence.

Both labgrid-client and the pytest plugin call apply_rcfile()
early in their startup paths.

Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
The current subprocess helper assumes it is the only thing running. In
some cases (such as the Dediprog em100 tool) we want to start a process
and have it run for a while in the background.

Rather than invent another helper, modify the existing helper to support
this.

For existing use cases, no functional change is intended.

Signed-off-by: Simon Glass <sjg@chromium.org>
Some drivers need to be told what to do from the command line. This is
particularly true of complex strategy drivers, such as the one to write
U-Boot to board media, or send it over USB.

Provide a way to specify driver-specific variables with a string value,
so that they can control the strategy. Examples include whether to
bootstrap the device or just boot the existing image, whether to build a
new image or use an existing one and whether to write the image to boot
media or send it over USB.

With these variables a useful selection of shell scripts can provide a
highly productive development and test environment, without a huge
prolification of very similar drivers. This approach also avoids the
complex interactions that would be necessary between different drivers
aimed towards the same goal.

Signed-off-by: Simon Glass <sjg@chromium.org>
The EM100-pro is a SPI-flash emulator which supports a variety of chips.
It can be used to provide an image to a booting system.

Add a driver which allows images to be loaded ready for use. Provide a
trace facility for use when the board does not boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
Some boards have a way to put them into a 'recovery' mode provided by
the SoC. Recovery is a signal asserted at power-on (or when the board is
reset). It causes the SoC to listen on USB.

Add a protocol for this feature.

Signed-off-by: Simon Glass <sjg@chromium.org>
The reset protocol only supports doing a reset. Sometimes we want to
assert reset while something is done (e.g. a power cycle or a recovery
signal), then de-assert it afterwards.

Add a new set_reset_enable() method for this.

The name is a little strange: one might expect set_enable() instead. But
with things like the PowerResetMixin we want to avoid ambiguity over
whether this refers to powering the device or resetting it.

Signed-off-by: Simon Glass <sjg@chromium.org>
sjg20 added 29 commits April 3, 2026 09:27
Transferring a file around is an operation which happens internally
within Labgrid. It is confusing to show these transfers as they happen.
Use quiet mode with rsync to avoid unwanted output.

Signed-off-by: Simon Glass <sjg@chromium.org>
This class has a lot of members whose type and purpose is hard to figure
out. Add some documentation at the top.

Signed-off-by: Simon Glass <sjg@chromium.org>
This function lacks documentation. Tidy it up.

While we are here, reverse the order of the 'active' and 'activate'
arguments, to separate the arguments into those at the start which
provide the filter and the one at the end which requests an action.

Signed-off-by: Simon Glass <sjg@chromium.org>
Some strategies may want to use a driver if it exists, without failing
if it does not. Add an argument to get_driver() to support this.

Signed-off-by: Simon Glass <sjg@chromium.org>
U-Boot prints a banner when it starts up, which contains the version
string. Collect this so it can be used by the strategy driver as needed.

Series-change: 8
- Support the U-Boot Concept version string too

Signed-off-by: Simon Glass <sjg@chromium.org>
When something goes wrong it is useful to see what output was processed
as a clue to what happened. Collect everything up to the autoboot
prompt, since once it hits that things are normally working OK.

Series-changes: 8
- Avoid clearing the output when reading it

Signed-off-by: Simon Glass <sjg@chromium.org>
When this driver is activated it looks for a banner, but if we are
connecting to a running board, there is no reset and so this will not be
forthcoming. Add a way to signal this to the driver.

Series-changes: 8
- Add new ubootdriver patch to allow connecting to a running board

Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes we need to push a button on a board to cause it to power on.
Add a driver to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
This causes a warning with 'pip install' on recent versions. Use a the
'license' property instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
At present the 'start' state of U-Boot consumes all the initial U-Boot
output, including the SPL and proper headers and any version
information. This makes it impossible to run the U-Boot pytests, since
this output is expected from the board.

There are two ways to fix this:
- Show the output
- Print a message indicating that U-Boot has booted, so tests don't
  wait for the headers

Do both, the first to make tests work existing U-Boot work with tests,
the second to resolve the problem for future U-Boot versions, once the
U-Boot pytest patch is accepted:

   test: Allow signaling that U-Boot is ready

Series-changes: 8
- Provide a flag to indicate whether the output should be cleared

Signed-off-by: Simon Glass <sjg@chromium.org>
The U-Boot strategy only allows bringing up a U-Boot prompt. If for some
reason this fails (e.g. due to a bug), nothing is shown on the console.

For interactive use, it is helpful to see any output, even if it does
not ultimately result in reach the U-Boot prompt.

Add a new 'start' state, to start U-Boot. Use this from the 'uboot'
state.

Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to write a new U-Boot version to the board before starting
it up. This is particularly valuable when using labgrid interactively,
to test out / debug U-Boot on various boards.

We end up with several different paths through the strategy:

- just power on/reset the board
- bootstrap: write an existing U-Boot to the board, then power it on
- build and bootstrap: build a new U-Boot, write it to the board, then
  power it on

All of these are useful in an interactive context.

Implement this by calling the U-Boot provider to build U-Boot, then the
U-Boot writer to write it to the board. Finally, power on/reset the
board as normal.

This is controlled by two variables:

   do-bootstrap: 1 to bootstrap the board, 0 to leave it along
   do-build:     1 to build U-Boot, 0 to use the existing build

Series-change: 8
- Check for None with image_dirs[1] instead of 'None'

Signed-off-by: Simon Glass <sjg@chromium.org>
Some boards do not have boot media, or it cannot be accessed by the test
system.

In this case it is useful to send U-Boot to the board using the
USB-recovery mechanism. Add the logic for this, controlled by a new
'do-send' variable.

Even for boards which do have boot media, it can still be useful to use
the USB mechanism.

Add the logic for this, using a new 'do-send' variable to control it. If
'do-send' is 1, U-Boot is written over USB. If it is 0, then U-Boot is
written to the boot media.

Series-changes: 7
- Drop comment about console not appearing

Signed-off-by: Simon Glass <sjg@chromium.org>
When Labgrid fails to get to the U-Boot command line, show the output
that was received, so that the user can see what is going on.

Series-change: 8
- Don't clear the console output afterwards

Signed-off-by: Simon Glass <sjg@chromium.org>
Some boards boot from internal flash unless a 'recovery' button is
pressed. Add support for this, so that these boards can be used in a
lab.

When Labgrid fails to get to the U-Boot command line, show the output
that was received, so that the user can see what is going on.

Series-changes: 7
- Allow recovery to be missing for the board

Signed-off-by: Simon Glass <sjg@chromium.org>
U-Boot's pytest framework has lots of logic to detect the U-Boot banner
and deal with failures to do so.

Labgrid supports this in its U-Boot driver.

We don't need both and Labgrid's implementation is better. It has more
knowledge of what is going on, since it is in charge of bringing the
board to life. It can be configured to know how long the board takes to
start up.

Add a 'lab mode' where Labgrid does all the work and just announces when
things are ready. This is enabled by presence of the U_BOOT_SOURCE_DIR
environment variable, which is set by U-Boot's pytest system.

This greatly simplifies operation of lab testing and ensures that pytest
is not trying to guess what happened.

Signed-off-by: Simon Glass <sjg@chromium.org>
Some boards need power to be applied before the console appears. Add
support for this with a new property.

Series-changes: 7
- Add patch to support boards which need power for console

Signed-off-by: Simon Glass <sjg@chromium.org>
When the 'power' device is also the 'reset' device, we don't want to
adjust both of them. Detect this so that boards don't bounce between
powered off an on too much.

Series-changes: 7
- Add patch to support boards which share power and reset

Signed-off-by: Simon Glass <sjg@chromium.org>
Some board refuse to start if reset is held while they are powering up.
Add a setting for this.

Series-changes: 7
- Add support for boards which cannot power on with reset

Signed-off-by: Simon Glass <sjg@chromium.org>
Some boards don't power on when power is applied, but need a button to
be pressed first. Add support for this.

Series-changes: 7
- Add patch to support boards which need a button to power on

Signed-off-by: Simon Glass <sjg@chromium.org>
The logic for starting up a board in these two cases has become quite
long. Move it into a function for flashing and a function for sending.

Series-changes: 8
- Add patch to move flash and reset code into their own functions

Signed-off-by: Simon Glass <sjg@chromium.org>
Create a new state to restart U-Boot by the easiest means possible. For
USB-attached boards, U-Boot must be resent, but otherwise we can
normally just reset the board.

Series-changes: 8
- Add patch to provide a way to restart U-Boot

Signed-off-by: Simon Glass <sjg@chromium.org>
Move the reset-cycle logic (hold reset, power on, enable recovery,
release reset) into its own method. This makes _reset_for_send() a
thin wrapper and prepares for adding retry logic around the reset
cycle.

No functional change.

Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
After releasing reset in _reset_for_send(), use await_resources() to
poll for the USB driver's resources rather than relying on a fixed
0.5s sleep. This fixes intermittent failures on boards like the TK1
where USB enumeration sometimes takes longer than expected.

If the resource does not appear within 2s, the full reset sequence is
retried once before giving up.

Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
When there is no separate reset driver (e.g. colibrimx8), asserting
reset powers off the board, which can disrupt the serial connection
and leave a stale FD. This causes EIO on the next read.

Deactivate and reactivate the console before releasing reset so the
serial connection is cleanly reopened. For boards with a separate
reset driver, the console is not disrupted by holding reset, so
only the existing activate is needed.

Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Some boards (e.g. iMX8) fail to enter USB recovery mode when
power-cycled too quickly after running Linux. The SoC or USB hub
may retain stale state if not held off long enough.

Add a power_off_delay property to UBootStrategy, used in
_prepare_for_send() to wait after powering off before proceeding
with recovery-mode setup. This defaults to 0 (no delay) so
existing boards are unaffected.

Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Provide some scripts which facilitate U-Boot testing and development
using Labgrid. Also add check_lab.py, a tool that queries recent GitLab
CI pipelines and summarises the pass/fail rate for each board in the
sjg-lab stage.

Series-changes: 6
- Fix reference to crossbar in _ub-bisect-try and ub-smoke
- Add -x option to specify the extra board directory
- Add comments to the example lg-env
- Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
- Add a few more comments to ub-pyt
- Show the build path with ub-pyt -v
- Add --allow-unmatched to ub-smoke to avoid QEMU error

Series-changes: 7
- Use LABGRID_SJG environment variable install of LABGRID
- Set HOSTNAME variable and use it in ub-pyt
- Rename lab config from env.cfg to kea_env.cfg

Series-changes: 8
- Pass -q to pytest when bisecting, to make it quieter
- Allow writing console log to a file in pytest
- Rename no_prompt_wait to use_running_system
- Allow use_running_system with pytest
- Provide a way to adjust U-Boot build CONFIGs on the fly
- Provide a way to send Labgrid logging to a file
- Fix up operation of em100 tracing
- Add some comments on how variables are passed from pytest to scripts
- Move default location of hooks to inside the U-Boot tree
- Add an empty pytest.ini so that pytest can find conftest.py

Signed-off-by: Simon Glass <sjg@chromium.org>
Co-developed-by: Claude Opus 4.6 <noreply@anthropic.com>
Add an example lab, based on my one, so people can see how to configure
things for this integration.

Series-changes: 6
- Add a patch with an example lab

Signed-off-by: Simon Glass <sjg@chromium.org>
Cover the drivers, scripts, pytest and the Gitlab information.

Note: This PR exists just so others can replicate my lab. I have created
separate PRs for features and will dribble those in as they get reviewed:

labgrid-project#1482
labgrid-project#1483
labgrid-project#1484
labgrid-project#1571

Cover-letter:
U-Boot integration
END

Changes in v9:
- Rebase on upstream
- Add ~/.lgrc support for site-wide labgrid defaults
- Add automatic file logging via LG_LOG_DIR for both client and pytest
- Add timestamps and CI_JOB_ID to auto-log filenames
- Make log directories world-writable for multi-user sharing
- Enable automatic labgrid-client logging in lg-env
- Wait for USB resources instead of using a fixed sleep
- Extract _prepare_for_send() from _reset_for_send()
- Add get_send_driver_name() for USB resource discovery
- Reopen console when power-cycling for send
- Increase USB resource timeout and add power_off_delay
- Add support for the Badgerd SDWire mux
- Add QEMU disk-image and writer-arguments support for EFI
- Use snapshot mode for the QEMU EFI root disk
- Add check_lab.py for board reliability checking

Changes in v8:
- Fairly minor expansions to deal with new use cases

Changes in v6:
- Fix reference to crossbar in _ub-bisect-try and ub-smoke
- Add -x option to specify the extra board directory
- Add comments to the example lg-env
- Define USE_LABGRID to tell u-boot-hooks to use labgrid hooks
- Add a few more comments to ub-pyt
- Show the build path with ub-pyt -v
- Add --allow-unmatched to ub-smoke to avoid QEMU error
- Add a new section on compatibility
- Add a new section on script usage

Changes in v5:
- Rebase on latest grpc branch
- Fix pylint errors and warnings

Changes in v4:
- Support for Beagleplay, which needs files from two separate U-Boot
  builds
- Support for a 'recovery' button needed to boot the image
- Tidy up the internal-console support
- Fix pytest behaviour with an unpatched U-Boot (that doesn't have lab
  mode)

Changes in v3:
- Rebase on top of grpc branch
- Don't mess with terminal setting unless stdin is a terminal
- Don't show an error if there are no resources when auto-acquiring
- Support QEMU in UBootWriter

Some changes in v2:
- Rationalise the flags for the U-Boot scripts
- Support tracing with em100
- Support an internal terminal instead of microcom
- Add a -D flag for debugging
- Support send-only boards
- Add a way to build the U-Boot config
- Add a control for buildman's process-limit
- allow the build-dir to be specified in a variable
- add documentation about U-Boot-pytest integration
- add source_dir and config_file to UBootProviderDriver
- add an internal terminal
- expand the U-Boot scripts
- significantly improve the U-Boot-pytest integration

The approximate diffstat is:
 contrib/sync-places.py                |  23 +-
 contrib/u-boot/.gitignore             |   1 +
 contrib/u-boot/_ub-bisect-try         |  47 ++
 contrib/u-boot/conftest.py            |  21 +
 contrib/u-boot/get_args.sh            | 128 +++++
 contrib/u-boot/index.rst              | 232 +++++++++
 contrib/u-boot/lg-client              |  11 +
 contrib/u-boot/lg-env                 |  10 +
 contrib/u-boot/test_smoke.py          |   3 +
 contrib/u-boot/ub-bisect              |  44 ++
 contrib/u-boot/ub-cli                 |  39 ++
 contrib/u-boot/ub-int                 |  41 ++
 contrib/u-boot/ub-pyt                 |  69 +++
 contrib/u-boot/ub-smoke               |  48 ++
 doc/configuration.rst                 | 611 ++++++++++++++++++++++-
 doc/usage.rst                         | 288 +++++++++++
 labgrid/driver/__init__.py            |  10 +-
 labgrid/driver/common.py              |  11 +
 labgrid/driver/consoleexpectmixin.py  |   7 +
 labgrid/driver/powerdriver.py         |  29 ++
 labgrid/driver/qemudriver.py          |  84 ++--
 labgrid/driver/recoverydriver.py      |  25 +
 labgrid/driver/resetdriver.py         |   7 +
 labgrid/driver/servodriver.py         | 157 ++++++
 labgrid/driver/sfemulatordriver.py    | 102 ++++
 labgrid/driver/ubootdriver.py         |  27 +-
 labgrid/driver/ubootproviderdriver.py | 323 ++++++++++++
 labgrid/driver/ubootwriterdriver.py   | 160 ++++++
 labgrid/driver/usbhidrelay.py         |   7 +-
 labgrid/driver/usbloader.py           | 170 ++++++-
 labgrid/driver/usbstoragedriver.py    |  18 +-
 labgrid/factory.py                    |   4 +-
 labgrid/protocol/__init__.py          |   1 +
 labgrid/protocol/bootstrapprotocol.py |   8 +-
 labgrid/protocol/recoveryprotocol.py  |  14 +
 labgrid/protocol/resetprotocol.py     |  12 +
 labgrid/pytestplugin/fixtures.py      |  21 +-
 labgrid/pytestplugin/hooks.py         |   8 +
 labgrid/remote/client.py              | 351 ++++++++-----
 labgrid/remote/config.py              |   7 +-
 labgrid/remote/exporter.py            | 122 ++++-
 labgrid/resource/__init__.py          |   6 +
 labgrid/resource/remote.py            |  24 +
 labgrid/resource/servo.py             | 485 ++++++++++++++++++
 labgrid/resource/sfemulator.py        |  33 ++
 labgrid/resource/suggest.py           |   6 +
 labgrid/resource/udev.py              |  33 ++
 labgrid/strategy/ubootstrategy.py     | 151 +++++-
 labgrid/target.py                     | 120 ++++-
 labgrid/util/helper.py                | 220 ++++----
 labgrid/util/ssh.py                   |   3 +-
 labgrid/util/term.py                  | 184 +++++++
 labgrid/var_dict.py                   |   8 +
 man/labgrid-client.1                  |   6 +
 man/labgrid-client.rst                |   4 +
 man/labgrid-device-config.5           |   4 +
 56 files changed, 4267 insertions(+), 321 deletions(-)

[1] https://github.com/labgrid-project/labgrid/issues/created_by/sjg20

Signed-off-by: Simon Glass <sjg@chromium.org>
@sjg20 sjg20 force-pushed the u-boot-integration branch from 89a6fc5 to b0f73a5 Compare April 3, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants