Please cache

In various places in these docs you might find reference to caches. Please can make use of several caches to speed up its performance which are described here.

Note that caches are different from the output in plz-out. Even without caching, Please will still reuse built artifacts from plz-out for incremental builds. Please has strict caching and incrementality behavior. With that said, you can still force a rebuild or a test rerun, use the --rebuild and --rerun flags respectively.

In all cases artifacts are only stored in the cache after a successful build or test run.

The directory cache

This is the simplest kind of cache; it's on by default and simply is a directory tree (by default ~/.cache/please or ~/Library/Caches/please) containing various versions of built artifacts. The main advantage of this is that it allows extremely fast rebuilds when swapping between different versions of code (notably git branches).

Note that the dir cache is not threadsafe or locked in any way beyond plz's normal repo lock, so sharing the same directory between multiple projects is probably a Bad Idea.

The HTTP cache

This is a more advanced cache which, as one would expect, can run on a centralised machine to share artifacts between multiple clients. It has a simple API based on PUT and GET to store and retrieve opaque blobs.

It is simply configured by setting the httpurl property in the cache section of the config. There are a couple more settings to configure it for readonly mode and to set timeouts etc.

Since the API is simple there are many existing servers that can be configured for a backend; one option is nginx using its webDAV module.
Alternatively some CI services (for example Cirrus) may offer a compatible cache out of the box.

Thanks to Diana Costea who implemented the original version of this as part of her internship with us, and prodded us into getting on and actually deploying it for our CI servers.

A reference implementation of the http cache can be found here however it should be possible to use any off the shelf http server with a little configuration, as described above.

Scriptable, command driven, cache

You can realize a centralised cache also by managing the remote access part in external commands or scripts. Please will simply invoke the configured command whenever it wants to store or retrieve artifacts. A good use case is AWS S3 access by simply using the aws-cli. Lite weight generic S3 support (Minio/GCS/Backblaze etc.) could be added by running e.g. MinIO mc client or similar tools.

It's somewhat slower compared to the http cache because it must spawn external commands. Benefit is that you don't need to run and maintain a proxy between Please and your storage provider.

Enable read access by setting the RetrieveCommand property in the cache section of the config. The configured command must be able to stream a tar file to stdout. Please sets the environment variable CACHE_KEY to the cache entries filename.

    
    
    [Cache]
    # stream from s3 object to stdout:
    RetrieveCommand="aws s3 cp s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY -"

Write access is controlled by the StoreCommand property. Keep it empty if you only want to have read-only access. The configured command must be able to read a tar file from stdin. Use the environment variable CACHE_KEY as a part of the filename so that Please can retrieve the artifact by using the same key later.

    
    
    [Cache]
    # stream stdin into a s3 object:
    StoreCommand="aws s3 cp - s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY"

Mind that you can configure read-only access but not write-only as RetrieveCommand is mandatory to enable custom cache support.

Raising the verbosity to debug level will help you to debug your script or command. Please logs each store/retrieve operation to the custom cache and your script's stderr will be shown as well.