Basic workflow

The following shows how the different components in the BuildBox ecosystem are brought up in a minimal scenario.

See the Installation section for instructions on how to build the different components.

1. buildbox-worker

The worker is a long-running process that connects to a Remote Execution server and constantly polls it to request work. When it receives an action to perform it will launch a buildbox runner to carry out the work, thus it requires a path to a runner binary.

This command starts an instance of buildbox-worker with a minimal set of arguments:

./buildbox-worker \
  --bots-remote=$BOTS_URL \
  --cas-remote=unix:/tmp/cache/casd.sock \
  --buildbox-run=$PATH_TO_RUNNER_BINARY

  • The --bots-remote and --cas--remote point the worker to the Remote Execution and CAS 1 servers, respectively. The runner will also connect to the specified CAS server.

  • --buildbox-run points to an executable that complies with the BuildBox runner interface. For each job the worker will launch it with the necessary arguments.

  • buildbox-worker allows specifying an arbitrary number of --runner-arg CLI options that will be passed straight to the runner binary when it is launched. Some of those options might be runner-specific, in which case the worker arguments will need adjusting when pointed to a different runner implementation.

Note

To connect to a BuildGrid instance listening on the default ports, $BOTS_URL should be set to http://buildgridIP:50051.

Warning

1

That example command makes the worker connect to a buildbox-casd instance, which is a CAS proxy that is running locally. However, casd is not required. In order to connect to a generic CAS server (for example to a BuildGrid instance), the --runner-arg=--disable-localcas option must be passed to the worker. This stops the worker and runner from attempting to invoke casd-specific function calls (see https://gitlab.com/BuildGrid/buildbox/buildbox-casd/-/blob/master/docs/casd.md#localcas-protocol for more details about them).

2. buildbox-runner

A runner program is responsible for actually executing an action. BuildBox runners follow a generic interface (see buildbox runner) that allows different implementations to be easily interchanged. Following that specification, a runner will receive a path to a file containing an Action and another path to where it will write an ActionResult file.

Note that runners are not invoked directly; they will be spawned by buildbox-worker with the right arguments.

buildbox-run-hosttools, which runs commands in the host without any sandboxing, is the quickest way to get a BuildBox environment set up locally for testing.

3. buildbox-casd

buildbox-casd is a local CAS proxy/cache. It allows to keep blobs locally on disk so that multiple actions that require a same subset of files do not need to repeatedly fetch them from a remote CAS server.

The following command launches casd in proxy mode and connects it to a remote CAS server, using /tmp/cache as its storage directory:

./buildbox-casd \
  --cas-remote=http://$CAS_IP:$CAS_PORT \
  /tmp/cache  # Directory where the blobs will be stored

casd will by default listen on an UNIX domain socket, in this case: unix:/tmp/cache/casd.sock.

Processes that require access to blobs from the remote CAS can be then transparently pointed at that casd instance to take advantage of the caching mechanism.

Note

To connect to a BuildGrid instance’s CAS running in its default port, $CAS_IP:=buildgridIP and $CAS_PORT:=50051.

Once the worker is up and running, everything is ready to start receiving work from the Remote Execution server and carrying out those actions using the specified runner.

For diagnostics and improved visibility into the status of the different parts, BuildBox components can be invoked with --log-level={DEBUG} or --log-level={TRACE}.