Skip to main content

Pattern: caching strategies

Concrete, copy-pastable caching recipes for common use cases. Each example shows a complete cache: block you can drop into your workflow.

For how caching works under the hood, see:

Strategy: Node.js / pnpm dependencies​

Cache the pnpm store so pnpm install resolves from local disk instead of the network.

deps:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull-push
    when: always
  variables:
    PNPM_STORE_DIR: .pnpm-store
  script:
    - pnpm install --frozen-lockfile

Why these choices:

  • files: [pnpm-lock.yaml] — the lockfile fully describes the dependency tree. When it changes, a new cache key is generated.
  • paths: [.pnpm-store] — only the store directory, not node_modules (which can contain platform-specific binaries that break across environments).
  • when: always — the store is valid regardless of whether tests pass or fail.
  • PNPM_STORE_DIR — tells pnpm to use a workspace-relative store path that matches the cache path.

Variant: separate caches for deps and Nx build cache​

Use the sequence form when a job needs independent caches with different invalidation lifecycles:

check:
  stage: ci
  target: linux
  cache:
    - name: pnpm
      key:
        prefix: pnpm-deps
        files: [pnpm-lock.yaml]
      paths: [.pnpm-store]
      policy: pull-push
      when: always
    - name: nx
      key:
        prefix: nx-cache
        files: [pnpm-lock.yaml, nx.json]
      paths: [.nx/cache]
      policy: pull-push
      when: on_success
  script:
    - pnpm install --frozen-lockfile
    - pnpm nx run-many -t check

The pnpm cache invalidates on lockfile changes. The Nx cache invalidates on lockfile or Nx config changes. Separating them avoids busting the pnpm cache when only the Nx graph changes.

Strategy: Go modules​

Cache the Go module download cache so go mod download skips network fetches.

go-test:
  stage: ci
  target: linux
  cache:
    key:
      prefix: go-mod
      files:
        - go.work
        - go.work.sum
        - "**/go.sum"
    paths: [.go]
    policy: pull-push
    when: always
  variables:
    GOPATH: .go
    GOMODCACHE: .go/pkg/mod
  script:
    - go mod download
    - go test ./...

Why these choices:

  • files uses glob "**/go.sum" to capture all go.sum files in a monorepo. Loom supports *, ?, and ** glob patterns in key.files.
  • GOPATH and GOMODCACHE are set to workspace-relative paths so the cached directory matches where Go writes modules.
  • go.work and go.work.sum are included because workspace-mode dependency resolution depends on them.

Strategy: build artifacts​

Cache build outputs and the build tool's own cache so it can reuse work on later runs. Loom still invokes every script command on a cache hit; in this example, Nx decides whether it can reuse a build result.

build:
  stage: ci
  target: linux
  cache:
    key:
      prefix: build-artifacts
      files: [pnpm-lock.yaml, tsconfig.base.json]
    paths: [dist, .nx/cache]
    policy: pull-push
    when: on_success
  script:
    - pnpm install --frozen-lockfile
    - pnpm nx run-many -t build

Why these choices:

  • when: on_success — only cache build outputs from successful runs. A failed build may produce partial artifacts that are unsafe to restore.
  • files includes both the lockfile (dependency changes) and tsconfig.base.json (compiler config changes). Both affect build output.
  • paths includes dist (build output) and .nx/cache (Nx computation cache).

Strategy: read-only consumers with pull policy​

When multiple jobs consume the same cache but only one should update it, use policy: pull on consumers:

install:
  stage: deps
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull-push
  script:
    - pnpm install --frozen-lockfile

lint:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull
  script:
    - pnpm run lint

test:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull
  script:
    - pnpm test

install writes the cache. lint and test read it without a redundant save — faster runs, no write contention.

Strategy: fallback keys for partial hits​

Use fallback_keys so a cache miss on the primary key can still restore a close-enough baseline:

deps:
  stage: ci
  target: linux
  cache:
    key: "pnpm-${head_sha}"
    fallback_keys:
      - "pnpm-main"
      - "pnpm-default"
    paths: [.pnpm-store]
    policy: pull-push
  script:
    - pnpm install --frozen-lockfile

On first run of a new branch, the exact ${head_sha} key misses. Loom tries pnpm-main, then pnpm-default. A partial restore from a recent baseline means pnpm install only downloads the diff.

Available template variables​

PlaceholderDescription
${job_name}Job name (graph node ID)
${job_id}Job identifier (same as job name today)
${run_id}Current run ID
${pipeline_id}Pipeline ID
${head_sha}HEAD commit SHA of the snapshot

Trust-but-verify with --cache-diff​

Run --cache-diff to compare a cache-off baseline with a cache-on variant. Both executions use independent workspaces derived from the same pinned input snapshot:

loom run --local --cache-diff --workflow .loom/workflow.yml

The versioned declared-output/v1 contract compares node presence, status, exit code, and content manifests for declared cache paths and declared artifacts. Equality covers only that mechanical surface. It does not prove undeclared workspace side effects, stdout/stderr, timing, network behavior, or universal determinism.

Loom decides whether to quarantine participating cache-hit candidate keys after collecting complete evidence for both executions. For a divergent node, Loom quarantines an unambiguous candidate key and records ambiguous candidates without quarantine. This is not a universal cache-safety guarantee.

When to use --cache-diff​

ScenarioRecommended
Introducing caching for the first timeYes
Cache key or path config just changedYes
Declared-output check (release, deploy, integrity)Yes
Job depends on external state (network, registries)Yes
Fast iteration on non-critical jobsNo — cache speed is the priority
Job is intentionally nondeterministicNo — fix nondeterminism first

What quarantine looks like​

Follow cache_diff.quarantine_path in the run receipt to the quarantine file when present. Each entry records the scope, key, reason (cache_diff_divergence), and timestamp.

To confirm a quarantine happened, check the cache system sections in the run logs:

.loom/.runtime/logs/<run_id>/jobs/<job_id>/system/cache_restore/events.jsonl
.loom/.runtime/logs/<run_id>/jobs/<job_id>/system/cache_save/events.jsonl

Quarantined entries produce log messages like:

cache restore skipped quarantined cache_name=pnpm key=pnpm-deps-abc123 scope_hash=... key_hash=...

Recovering from quarantine​

  1. Read the divergence context — start from the Diagnostics ladder and check the cache system section events.
  2. Identify the root cause:
    • Missing key inputs: add the missing file(s) to cache.key.files so the key changes when outputs should change.
    • Nondeterministic scripts: control sources of nondeterminism (timestamps, randomness, network state, ambient env vars).
    • Tool/version drift: include lockfiles and version configs in the key, or pin tool versions.
  3. Rotate the cache key: change cache.key.prefix (or the template string) to intentionally miss old entries and establish a new baseline.
  4. Clear quarantine: delete the entry from quarantine.json, or delete the file to clear all quarantines.

Cache key design checklist​

Use this before shipping any cache configuration:

  • Include all inputs that change outputs — lockfiles, build configs, tool version files. If you can't explain why a file is in the key, it probably doesn't belong.
  • Scope keys to avoid collisions — use distinct prefixes and/or ${job_name} so unrelated jobs don't overwrite each other.
  • Prefer multi-cache for different concerns — one cache entry per concern (e.g., pnpm vs go) so you can invalidate independently.
  • Use workspace-relative paths — absolute paths and paths containing .. are rejected. For Docker jobs, paths refer to directories inside the container's workspace mount at /workspace.
  • Decide on ${head_sha} — include it when correctness matters more than reuse (per-commit caches). Omit it when reuse matters more (branch/shared caches).
  • Validate with --cache-diff — review declared-output equality before trusting participating cache hits.