.. _getting-started:

Getting Started
===============

Starting up a BuildGrid server requires only installation and a configuration file. 

.. warning::
    **WARNING**: The following setup is highly insecure and intended for proof-of-concept
    testing ONLY! A malicious user can have full unsandboxed access to the entire host system.
    A secure setup must be configured with authentication.

Installation
------------

Please follow the steps in the :ref:`installation guide <installation>` to install it on your host system.

Configuration File
------------------

If you'd like to get started, a sample configuration that uses your filesystem
as the CAS (Content Addressable Storage) and a SQLite-based scheduler is provided below.
Copy the following text into a file called config.yml and **change /path/to/cas/ to a suitable storage path.**

By default the scheduler's data store is written to **./example.db**. You can also change
this to a suitable storage path.

.. literalinclude:: ../../../data/config/basic-with-disk.yml
   :language: yaml

To start buildgrid with this configuration, simply run:

.. code-block:: sh

    bgd server start --verbose /path/to/config.yml

See the `Understanding the configuration file`_ section to learn more about this file.
For now, we will continue setting up BuildGrid for work. 

Setting up a bot
----------------

Now we will need a worker. The recommended worker to use with BuildGrid is `buildbox-worker`_.
This worker works best when used alongside a local CAS cache called `buildbox-casd`_. First,
build these tools following the instructions in their READMEs.

Then, start the CAS cache.

.. code-block:: sh

    buildbox-casd --cas-remote=http://localhost:50051 --bind=127.0.0.1:50011 ~/casd &

Once CASD is running we can start the worker itself, pointing it to CASD for CAS requests.

.. code-block:: sh

    buildbox-worker --buildbox-run=buildbox-run-hosttools --bots-remote=http://localhost:50051 \
        --cas-remote=http://127.0.0.1:50011 --request-timeout=30 my_bot

We should be able to see this worker connecting as log messages for ``CreateBotSession`` and
``UpdateBotSession`` requests in the server logs.

.. _buildbox-worker: https://gitlab.com/BuildGrid/buildbox/buildbox-worker
.. _buildbox-casd: https://gitlab.com/BuildGrid/buildbox/buildbox-casd

Without CASD
~~~~~~~~~~~~

.. warning::
    Whilst this approach has less moving parts, it **will** make your build slower due to
    needing to freshly fetch the input root for every Action rather than keeping a local
    cache. Production deployments should use ``buildbox-casd``.

``buildbox-worker`` supports running without ``buildbox-casd`` by pointing it to the remote CAS
rather than the local CASD, although this isn't recommended due to the additional network load
it will lead to. When running in this configuration, its important to tell the runner command
to not use the LocalCAS protocol to stage the input root.

.. code-block:: sh

    buildbox-worker --buildbox-run=buildbox-run-hosttools --bots-remote=http://localhost:50051 \
        --cas-remote=http://localhost:50051 --request-timeout=30 --runner-arg=--disable-localcas my_bot

Sending work to BuildGrid
-------------------------

We have a working BuildGrid setup! Now you can send work to it.

You can use your own remote execution compatible client, but BuildGrid also comes with a
command-line tool for execute simple commands called ``bgd execute``.

We're going to send a simple job that just cats a file. All Execute requests specify an
Action to execute, and each Action has an "input root," which is a directory that the job
is performed inside. ``bgd execute`` requires this input root as a command-line parameter,
so let's make one. 

In another terminal, create a directory in your home directory or somewhere else that
is convenient, then add a file to it:

.. code-block:: sh

    mkdir ~/my_input_root
    echo "I'm in the input root!" >> ~/my_input_root/input_root.txt

Now, let's send the action to our running BuildGrid instance and bot with "bgd execute."

.. code-block:: sh

    bgd execute --remote http://localhost:50051 --remote-cas http://localhost:50051 command ~/my_input_root /bin/cat input_root.txt

In the above request, notice that the input root (``~/my_input_root``) is specified first, before
the rest of the command. The file is specified relative to the input root.

If all goes well, the request should have been sent to BuildGrid, which will have
farmed it out to the bot. The bot will have done the work and sent it back to BuildGrid,
and bgd execute should display the response metadata. Look for the ``stdout_raw`` field,
which will contain the text of the file we catted.

If the ``stdout_raw`` field is empty, the output might be stored in CAS and need to be fetched.
The digest to fetch is in the ``stdout_digest`` field. We can fetch it using the ``bgd cas``
tool.

.. code-block:: sh

    bgd cas --remote http://localhost:50051 download-file <stdout_digest> action-stdout.log

.. _understanding-the-configuration-file:

Understanding the configuration file
------------------------------------

Looking at the config file provides insight into BuildGrid's structure.
While a full breakdown of the configuration options can be found on the 
:ref:`configuration page <configuration>`, the "instances" section deserves
special mention.

BuildGrid is not just a single service, but rather a collection of services that
work together to facilitate remote execution. The REAPI has the notion of "instances",
which effectively act as namespaces for requests since requests must contain the
instance name as a field. Each instance can have one or more services attached to it.

The services that can be attached to an instance are as follows:

- !execution
    The Execution Service is BuildGrid's primary service. Your clients' Execute() requests
    are sent here for processing. Note that it checks the ActionCache to determine whether
    it can just return a cached result. The data-store represents the backend for the scheduling
    system. In this case, we're using a sql-backed scheduler.
- !cas
    This is the interface into the Content-Addressable Storage. Note that the
    other services talk to the storage backend directly rather than make requests to this service.
- !bytestream
    This is the interface into the ByteStream service. This is used for blobs too large to be
    handled by the batch gRPC methods of the CAS service.
- !action-cache
    The ActionCache service handles requests to the ActionCache. It needs to talk to the backend
    storage to retrieve the ActionResult blobs. In this configuration, we have set the maximum
    number of cacheable actions to 256, and we allow failed builds to be cached and make the
    ActionCache writeable.

The :ref:`<anatomy-of-a-configuration-file>` section describes an example
configuration file, including this "instances" section in more detail.

.. _BuildStream proto: https://gitlab.com/BuildStream/buildstream/blob/master/src/buildstream/_protos/buildstream/v2/buildstream.proto

Next steps
----------

Now that we have a working BuildGrid, it is time to run some real Actions.
Usage of BuildGrid with various clients is documented in the :ref:`using` section,
along with information on how to use the sandboxing features of BuildBox and
some of the more advanced features of BuildGrid such as :ref:`Indexed CAS <indexed-cas>`
and :ref:`CAS cleanup <cas-cleanup>`.