s3-cache/README.md

S3 Cache Action

This action allows caching dependencies and build outputs to improve workflow execution time using S3-compatible storage (like MinIO, Ceph, AWS S3, etc.).

The interface, inputs, outputs, and internal behavior of this action are intentionally designed to be identical to the official actions/cache. This ensures that it can be used as a seamless drop-in replacement in self-hosted environments where the official GitHub Actions cache might not be available or suitable.

Inputs

• path (Required): A list of files, directories, and wildcard patterns to cache and restore.
• key (Required): An explicit key for restoring and saving the cache.
• restore-keys (Optional): An ordered list of keys to use for restoring stale cache if no cache hit occurred for key. Note cache-hit returns false in this case.
• lookup-only (Optional): Check if a cache entry exists for the given input(s) (key, restore-keys) without downloading the cache. Default is false.
• fail-on-cache-miss (Optional): Fail the workflow if cache entry is not found. Default is false.
• s3-endpoint (Required): The endpoint URL of your S3-compatible storage.
• s3-bucket (Required): The name of the S3 bucket to store the cache in.
• s3-access-key (Required): The S3 access key.
• s3-secret-key (Required): The S3 secret key.
• s3-region (Optional): The S3 region. Default is us-east-1.

Outputs

• cache-hit: A boolean value to indicate an exact match was found for the primary key.

Examples

npm

steps:
  - name: Get npm cache directory
    id: npm-cache-dir
    run: echo "dir=$(npm config get cache)" >> "$GITHUB_OUTPUT"
  - name: Cache npm
    uses: https://stackit-solutions.git.onstackit.cloud/actions/s3-cache@v1
    with:
      path: ${{ steps.npm-cache-dir.outputs.dir }}
      key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
      restore-keys: |
        ${{ runner.os }}-node-
      s3-endpoint: https://object.storage.eu01.onstackit.cloud
      s3-region: eu01
      s3-bucket: ${{ vars.S3_BUCKET }}
      s3-access-key: ${{ vars.S3_ACCESS_KEY }}
      s3-secret-key: ${{ secrets.S3_SECRET_KEY }}

OpenTofu

env:
  TF_PLUGIN_CACHE_DIR: /opt/.terraform.d/plugin-cache
steps:
  - name: Install OpenTofu
    uses: opentofu/setup-opentofu@v2
    with:
      tofu_wrapper: false
  - name: Setup OpenTofu provider cache
    shell: bash
    run: mkdir -p "$TF_PLUGIN_CACHE_DIR"
  - name: Cache OpenTofu providers
    uses: https://stackit-solutions.git.onstackit.cloud/actions/s3-cache@v1
    with:
      path: ${{ env.TF_PLUGIN_CACHE_DIR }}
      key: ${{ runner.os }}-opentofu-${{ hashFiles('**/terraform.lock.hcl') }}
      restore-keys: |
        ${{ runner.os }}-opentofu-
      s3-endpoint: https://object.storage.eu01.onstackit.cloud
      s3-region: eu01
      s3-bucket: ${{ vars.S3_BUCKET }}
      s3-access-key: ${{ vars.S3_ACCESS_KEY }}
      s3-secret-key: ${{ secrets.S3_SECRET_KEY }}
  - name: Initialize OpenTofu
    shell: bash
    run: tofu init

Cache Strategies

This action supports the same caching strategies as the official cache action:

Exact Match

An exact match occurs when the primary key exactly matches a previously saved cache in the S3 bucket. When this happens, the action downloads the cache, and the cache-hit output is set to true. Because an exact match was found, the post step will skip uploading the cache again at the end of the workflow.

Prefix Match (Fallback)

If there is no exact match for the key, the action evaluates the restore-keys. It uses these keys as prefixes to search the S3 bucket. If multiple caches match the prefix, it will download the most recently created one. In this case, the cache-hit output is set to false (because there was no exact match), but the cache is still restored. Since the primary key was a miss, the post step will pack and upload a fresh cache using the new primary key at the end of the job.

S3 Store Configuration

This action relies on the underlying S3 storage to manage the lifecycle of the cache files. The action creates and reads objects but never deletes them. To prevent your bucket from growing indefinitely, it is highly recommended to configure a Lifecycle Policy on your S3 bucket to automatically remove objects after a certain period (e.g., 7 days).

Example: Setting a 7-day expiration using s3cmd

Create a file named lifecycle.xml:

<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID>ExpireOldCaches</ID>
    <Prefix>runner-cache/</Prefix>
    <Status>Enabled</Status>
    <Expiration>
      <Days>7</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

Apply the policy to your bucket:

s3cmd setlifecycle lifecycle.xml s3://your-s3-cache-bucket

Developer Notes

Setup

To set up the development environment, clone the repository and install the dependencies:

npm install

Git Hooks (Husky)

The project uses husky to manage Git hooks. Upon running npm install, the prepare script will automatically configure Husky. This ensures that formatting, linting, and other quality checks are executed before you commit your code.

Testing on macOS with Colima

The integration tests utilize testcontainers to spin up a local Adobe S3Mock instance. If you are developing on macOS and using Colima as your Docker runtime, testcontainers needs to know where to find the Docker socket inside the virtual machine.

Before running the tests, export the following environment variable:

export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock
npm run test

Local Workflow Testing with act

You can test the entire GitHub Actions workflow locally using act.

act --workflows .github/workflows/test.yaml -P stackit-docker=registry.onstackit.cloud/devex-images/ubuntu:act-latest