Installation

Installation onto host machine

How to install BuildGrid directly onto your machine.

Note

BuildGrid server currently only support Linux. macOS and Windows platforms are not supported.

Prerequisites

BuildGrid only supports python3 >= 3.5.3 but has no system requirements. Main Python dependencies, automatically handled during installation, include:

  • boto3: the Amazon Web Services (AWS) SDK for Python.

  • click: a Python composable command line library.

  • grpcio: Google’s gRPC Python interface.

  • janus: a mixed sync-async Python queue.

  • protobuf: Google’s protocol-buffers Python interface.

  • PyYAML: a YAML parser and emitter for Python.

Install from sources

BuildGrid has setuptools support. We recommend installing it in a dedicated virtual environment. In order to do so in an environment named env placed in the source tree, run:

git clone https://gitlab.com/BuildGrid/buildgrid.git
cd buildgrid
python3 -m venv env
env/bin/python -m pip install --upgrade setuptools pip wheel
env/bin/python -m pip install --editable .

Hint

Once created, the virtual environment can be activated by sourcing the env/bin/activate script. In an activated terminal session, simply run deactivate to later deactivate it.

Once completed, you can check that installation succeed by locally starting the BuildGrid server with default configuration. Simply run:

env/bin/bgd server start data/config/default.conf -vvv

Note

The setup.py script defines three extra targets, auth, docs and tests. They declare required dependency for, respectively, authentication and authorization management, generating documentation and running unit-tests. They can be use as helpers for setting up a development environment. To use them run:

env/bin/python -m pip install --editable ".[auth,docs,tests]"

Install through Docker

BuildGrid comes with Docker support for local development use-cases.

Caution

The Docker manifests are intended to be use for local development only. Do not use them in production.

Please consult the Get Started with Docker guide if you are looking for instructions on how to setup Docker on your machine.

Docker build

BuildGrid ships a Dockerfile manifest for building images from source using docker build. In order to produce a buildgrid:local base image, run:

git clone https://gitlab.com/BuildGrid/buildgrid.git
cd buildgrid
docker build --tag buildgrid:local .

Note

The image built will contain the Python sources, including example configuration files. The main endpoint is the bgd CLI tools and the default command shall run the BuildGrid server loading default configuration.

Once completed, you can check that build succeed by locally starting in a container the BuildGrid server with default configuration. Simply run:

docker run --interactive --publish 50051:50051 buildgrid:local

Hint

You can run any of the BuildGrid CLI tool using that image, simply pass extra arguments to docker run the same way you would pass them to bgd.

Bear in mind that whenever the source code or the configuration files are updated, you must re-build the image.

Docker Compose

BuildGrid ships a docker-compose.yml manifest for building and running a grid locally using docker-compose. In order to produce a buildgrid:local base image, run:

git clone https://gitlab.com/BuildGrid/buildgrid.git
cd buildgrid
docker-compose build

Once completed, you can start a minimal grid by running:

docker-compose up

Note

The grid is composed of three containers:

  • An execution and action-cache service available at http://localhost:50051.

  • An CAS service available at http://localhost:50052.

  • A single unnamed instance with one host-tools based worker bot attached.

Hint

You can spin up more bots by using docker-compose scaling capabilities:

docker-compose up --scale bots=12

Hint

The contained services configuration files are bind mounted into the container, no need to rebuild the base image on configuration update. Configuration files are read from:

  • data/config/controller.conf for the execution service.

  • data/config/storage.conf for the CAS and action-cache service.