In various places in these docs you might find reference to caches. Please can make use of several caches to speed up its performance which are described here.
Not that caches are different from the output in
plz-out. Even without caching, please will still
reuse built artifacts from
plz-out for incremental
builds. Please has strict caching and incrementality behavior. With that said,
you can still force a rebuild or a test rerun, use the
--rerun flags respectively.
In all cases artifacts are only stored in the cache after a successful build or test run.
The directory cache
This is the simplest kind of cache; it's on by default and simply is a
directory tree (by default
~/Library/Caches/please) containing various
versions of built artifacts. The main advantage of this is that it allows
extremely fast rebuilds when swapping between different versions of code
(notably git branches).
Note that the dir cache is not threadsafe or locked in any way beyond plz's normal repo lock, so sharing the same directory between multiple projects is probably a Bad Idea.
The HTTP cache
This is a more advanced cache which, as one would expect, can run on a centralised machine to share artifacts between multiple clients. It has a simple API based on PUT and GET to store and retrieve opaque blobs.
It is simply configured by setting the
httpurl property in the
cache section of the config. There are a couple more settings to configure it for readonly mode and to
set timeouts etc.
Since the API is simple there are many existing servers that can be
configured for a backend; one option is nginx using its
Alternatively some CI services (for example Cirrus) may offer a compatible cache out of the box.
Thanks to Diana Costea who implemented the original version of this as part of her internship with us, and prodded us into getting on and actually deploying it for our CI servers.
A reference implementation of the http cache can be found here however it should be possible to use any off the shelf http server with a little configuration, as described above.
Scriptable, command driven, cache
You can realize a centralised cache also by managing the remote access part in external commands or scripts. Please will simply invoke the configured command whenever it wants to store or retrieve artifacts. A good use case is AWS S3 access by simply using the aws-cli. Lite weight generic S3 support (Minio/GCS/Backblaze etc.) could be added by running e.g. MinIO mc client or similar tools.
It's somewhat slower compared to the http cache because it must spawn external commands. Benefit is that you don't need to run and maintain a proxy between Please and your storage provider.
Enable read access by setting the
RetrieveCommand property in the
cache section of the config.
The configured command must be able to stream a tar file to stdout.
Please sets the environment variable
to the cache entries filename.
[Cache] # stream from s3 object to stdout: RetrieveCommand="aws s3 cp s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY -"
Write access is controlled by the
Keep it empty if you only want to have read-only access.
The configured command must be able to read a tar file from stdin.
Use the environment variable
as a part of the filename so that Please can retrieve the
artifact by using the same key later.
[Cache] # stream stdin into a s3 object: StoreCommand="aws s3 cp - s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY"
Mind that you can configure read-only access but not write-only as RetrieveCommand is mandatory to enable custom cache support.
Raising the verbosity to
debug level will
help you to debug your script or command. Please logs each store/retrieve
operation to the custom cache and your scripts stderr will be shown as well.