Tests in Please are run using
plz test (or
plz cover if you
want to extract coverage information).
The most obvious way of running tests is to pass a build label to
plz test //src/core/... to test all tests under a particular
Labels are also particularly useful for tests; Please accepts
--exclude flags which filter to a subset of what it was asked to run. These
are marked onto the build rules like so:
You can then invoke Please like
go_test( name = "my_test", srcs = ["my_test.go"], labels = ["slow", "integration"], )
plz test --exclude slowto run all tests except those labelled as "slow".
There is one special label,
manual which always excludes tests from
autodetection, so they will only be run if identified specifically. This is often
useful to disable tests from general usage if needed (for example, if a test starts
failing, you can disable it while investigating).
Please expects two things from tests:
RESULTS_FILE(unless the test has
no_test_output = True, in which case it is evaluated only on its exit code).
Tests can also be marked as flaky which causes them to be automatically re-run
several times until they pass. They are considered to pass if any one run passes.
This is set as follows:
By default they are run up to three times, you can alter this on a per-test basis by passing an integer instead of a boolean.
go_test( name = "my_test", srcs = ["my_test.go"], flaky = True, )
An important concept of testing that Please tries to help with is that tests should be hermetic; that is they should be isolated from local state that might cause them to unexpectedly fail. This is important to avoid nasty surprises later (for example, finding out that your test only passes if some other resources happen to have been built first, or that it fails arbitrarily if run in parallel while others are running too).
In order to help ensure this, Please runs each test in isolation from others. They
are each run within their own temporary directory which contains only the test itself
and files it's declared that it will need.
Most of the usual attributes of a build rule that refer to files are only available at build time (e.g.
deps etc). Instead
files that are needed at test runtime should be specified as
will appear within the test directory with their full repo path.
On Linux, tests can also be sandboxed, either by setting
on the test rule in question, or by setting it globally in the config:
sandbox = True,
This uses kernel namespaces to segregate it from other processes:
[test] sandbox = on
/tmp, so they can't accidentally overwrite each other's files. The temporary directory is also mounted under here to prevent tests from walking up to parent directories to find files they haven't declared.
On other platforms, we distribute the same binary alongside Please to simplify configuration, but it currently has no effect.
If you want to explore what the sandbox is doing, you can use
plz tool sandbox bash to get a shell within it; you'll observe that commands
curl no longer work.