Remote build execution

This feature is still experimental; some aspects may not work fully or at all just yet. Proceed with caution!

From v15 onwards, Please has the ability to submit build actions to remote executors.
By default work is still executed on your local machine, but this means parallelism is fundamentally limited by the available local compute capacity; larger teams may want to improve overall speed with access to a larger pool of compute capacity. There are also a number of other considerable benefits such as being able to share artifacts globally between all executions (essentially ensuring each build action only happens once) and executing them in a more controlled environment.

Please makes use of the Remote Execution API to distribute work. This is a generic gRPC-based API with a number of options for the server-side; see their documentation for some more information.

The API is fairly involved and there are some requirements that the server must fulfil to work with plz. Setting it up can be a reasonable amount of work.
A few things you might want to keep in mind are:

  • Please requires version 2.1 of the Remote Execution API (for the non-specific output path field). At time of writing relatively few servers identify as this, but we hope they'll upgrade soon.
  • Whatever tools you use off the system also have to be available on the server; if your build rule invokes go you have to have that on the PATH on the remote.
    This isn't needed if you use tools as other build rules though.
  • You have to have plz installed on the agent to access the tools it comes with. You might need to customise the remote.homedir property to find it.
  • One of the remote servers will need to implement the Remote Asset API. This can be a different endpoint from the others but will be needed for any remote_file rules to work.
We're actively working on a nmber of things to make the above easier and expect it to evolve quite a bit over the next few versions.

Come and chat to us on Gitter if you're interested in setting up Please for remote execution and would like some tips!