Writing build rules

Please is designed to support interleaving custom build commands as first-class citizens in the system. This describes some of the concepts to bear in mind when writing rules of your own.

The documentation for genrule may be of use as well, as that explains the available arguments and what they do.

Build location

A core concept of Please is to try to isolate compilation so we have a good concept of correctness for each action. To this end each rule builds in its own temporary directory (under plz-out/tmp) with a limited set of files in it.
If you want to have a look at what it looks like in there, plz build --shell will prepare a target for building and open up a shell into the directory, where you can look around and try running commands by hand.

Some aspects of this environment are different to normal - for example rules are given a small, deterministic set of environment variables. They shouldn't need to read anything outside this directory or not described by those variables.

Sources

Sources are the inputs to your rule - typically source code. This is defined in the srcs argument.

All sources are linked into the build directory - for performance reasons they aren't copied so you shouldn't modify them in-place.

Sources can be referred to through the $SRCS environment variable. If there's only one, a $SRC variable will be defined too.
Sources can also be named, e.g.


      srcs = {
          'srcs': ['my_source.py'],
          'resources': ['something.txt'],
      },
  
In that case they can be accessed separately through $SRCS_SRCS and $SRCS_RESOURCES.

Dependencies

Dependencies are other things you need to build - e.g. other code that your rule depends on.
These are also linked into the build directory, so the difference between sources and dependencies can seem arbitrary, but for some internal functions it's an important distinction to know how rules see the things they'll consume.

Dependencies don't have any corresponding environment variables associated with them.

Tools

Tools are the things that a rule uses to build with. They can refer either to other rules within the repo, or system-level binaries, for example:


      tools = [
          'curl',
          '//path/to:tool',
      ],
  
In this example, curl is resolved on the system using the PATH variable defined in your .plzconfig file. //path/to:tool will be built first and used from its output location.

Tools are not copied into the build directory, you can access them using the $TOOL or $TOOLS environment variables. They can also be named in a similar manner to sources.

Note that since tools aren't copied, they can be a source of nondeterminism if you make use of other outputs that happen to be located near them. In order to ensure correctness you should make sure that you only run the tool itself and don't access neighbouring outputs.

Commands

The command that you run is of course the core part of the rule. It can be passed to genrule in three formats:

  • As a string; the simplest format, since there's only one command that's run.
  • As a list, it's a sequence of commands run one after another if the preceding ones are successful (i.e. it's effectively shorthand for ' && '.join(cmd)
  • As a dict, the keys correspond to the build config to run and the values are the command to run in that config. The typical use case here is opt vs. dbg but arbitrary names can be given and specified with plz build -c name.

There are various special sequence replacements that the rule is subject to:

  • $(location //path/to:target) expands to the location of the given build rule, which must have a single output only.
  • $(locations //path/to:target) expands to the locations of the outputs of the given build rule, which can have any number of outputs.
  • $(exe //path/to:target) expands to a command to run the output of the given target. The rule must be marked as binary.
  • $(out_location //path_to:target) expands to the output of the given build rule, with the preceding plz-out/gen etc.
    Consider carefully when to use this though; it is not normally useful.
  • $(hash //path/to:target) expands to a hash of the outputs of that target.
    This can be useful to uniquely fingerprint the given rule.

The following environment variables are also set:

  • ARCH: architecture of the system, eg. amd64
  • OS: current operating system (linux, darwin, etc).
  • PATH: usual PATH environment variable as defined in your .plzconfig
  • TMP_DIR: the temporary directory you're compiling within.
  • HOME: also set to the temporary directory you're compiling within.
  • NAME: the name of the rule.
  • SRCS: the sources of your rule
  • OUTS: the outputs of your rule
  • PKG: the path to the package containing this rule
  • PKG_DIR: Similar to PKG but always contains a path (specifically . if the rule is in the root of the repo).
  • NAME: the name of this build rule
  • OUT: the output of this rule. Only present when there is only one output.
  • SRC: the source of this rule. Only present when there is only one source.
  • SRCS_<suffix>: Present when you've defined named sources on a rule. Each group creates one of these these variables with paths to those sources.
  • TOOLS: Any tools defined on the rule.
  • TOOL: Available on any rule that defines a single tool only.
  • SECRETS: If any secrets are defined on the rule, these are the paths to them.

Secrets

Rules can define a list of secrets that they want access to. These are all absolute paths (beginning with / or ~ and aren't copied to the build directory; instead they can be located using the environment variable $SECRETS.
They're useful for things like signing or access keys that you don't want to check into version control but still might be necessary for building some rules.

These don't contribute to the key used to retrieve outputs from the cache; this means it's possible for one machine to build a target with the secret and then share the output with others.