Skip to main content

Pattern: caching strategies

Concrete, copy-pastable caching recipes for common use cases. Each example shows a complete cache: block you can drop into your workflow.

For how caching works under the hood, see:

Strategy: Node.js / pnpm dependencies​

Cache the pnpm store so pnpm install resolves from local disk instead of the network.

deps:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull-push
    when: always
  variables:
    PNPM_STORE_DIR: .pnpm-store
  script:
    - pnpm install --frozen-lockfile

Why these choices:

  • files: [pnpm-lock.yaml] — the lockfile fully describes the dependency tree. When it changes, a new cache key is generated.
  • paths: [.pnpm-store] — only the store directory, not node_modules (which can contain platform-specific binaries that break across environments).
  • when: always — the store is valid regardless of whether tests pass or fail.
  • PNPM_STORE_DIR — tells pnpm to use a workspace-relative store path that matches the cache path.

Variant: separate caches for deps and Nx build cache​

Use the sequence form when a job needs independent caches with different invalidation lifecycles:

check:
  stage: ci
  target: linux
  cache:
    - name: pnpm
      key:
        prefix: pnpm-deps
        files: [pnpm-lock.yaml]
      paths: [.pnpm-store]
      policy: pull-push
      when: always
    - name: nx
      key:
        prefix: nx-cache
        files: [pnpm-lock.yaml, nx.json]
      paths: [.nx/cache]
      policy: pull-push
      when: on_success
  script:
    - pnpm install --frozen-lockfile
    - pnpm nx run-many -t check

The pnpm cache invalidates on lockfile changes. The Nx cache invalidates on lockfile or Nx config changes. Separating them avoids busting the pnpm cache when only the Nx graph changes.

Strategy: Go modules​

Cache the Go module download cache so go mod download skips network fetches.

go-test:
  stage: ci
  target: linux
  cache:
    key:
      prefix: go-mod
      files:
        - go.work
        - go.work.sum
        - "**/go.sum"
    paths: [.go]
    policy: pull-push
    when: always
  variables:
    GOPATH: .go
    GOMODCACHE: .go/pkg/mod
  script:
    - go mod download
    - go test ./...

Why these choices:

  • files uses glob "**/go.sum" to capture all go.sum files in a monorepo. Loom supports *, ?, and ** glob patterns in key.files.
  • GOPATH and GOMODCACHE are set to workspace-relative paths so the cached directory matches where Go writes modules.
  • go.work and go.work.sum are included because workspace-mode dependency resolution depends on them.

Strategy: build artifacts​

Cache expensive build outputs (compiled binaries, transpiled bundles) so subsequent runs skip the build step on cache hit.

build:
  stage: ci
  target: linux
  cache:
    key:
      prefix: build-artifacts
      files: [pnpm-lock.yaml, tsconfig.base.json]
    paths: [dist, .nx/cache]
    policy: pull-push
    when: on_success
  script:
    - pnpm install --frozen-lockfile
    - pnpm nx run-many -t build

Why these choices:

  • when: on_success — only cache build outputs from successful runs. A failed build may produce partial artifacts that are unsafe to restore.
  • files includes both the lockfile (dependency changes) and tsconfig.base.json (compiler config changes). Both affect build output.
  • paths includes dist (build output) and .nx/cache (Nx computation cache).

Strategy: read-only consumers with pull policy​

When multiple jobs consume the same cache but only one should update it, use policy: pull on consumers:

install:
  stage: deps
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull-push
  script:
    - pnpm install --frozen-lockfile

lint:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull
  script:
    - pnpm run lint

test:
  stage: ci
  target: linux
  cache:
    key:
      prefix: pnpm-deps
      files: [pnpm-lock.yaml]
    paths: [.pnpm-store]
    policy: pull
  script:
    - pnpm test

install writes the cache. lint and test read it without a redundant save — faster runs, no write contention.

Strategy: fallback keys for partial hits​

Use fallback_keys so a cache miss on the primary key can still restore a close-enough baseline:

deps:
  stage: ci
  target: linux
  cache:
    key: "pnpm-${head_sha}"
    fallback_keys:
      - "pnpm-main"
      - "pnpm-default"
    paths: [.pnpm-store]
    policy: pull-push
  script:
    - pnpm install --frozen-lockfile

On first run of a new branch, the exact ${head_sha} key misses. Loom tries pnpm-main, then pnpm-default. A partial restore from a recent baseline means pnpm install only downloads the diff.

Available template variables​

PlaceholderDescription
${job_name}Job name (graph node ID)
${job_id}Job identifier (same as job name today)
${run_id}Current run ID
${pipeline_id}Pipeline ID
${head_sha}HEAD commit SHA of the snapshot

Trust-but-verify with --cache-diff​

Run --cache-diff to prove that cached outputs match what the job would produce from scratch:

loom run --local --cache-diff --workflow .loom/workflow.yml

In this mode, Loom restores cached outputs and runs the job, then compares. If they diverge, the cache entry is quarantined — skipped on future restore attempts.

When to use --cache-diff​

ScenarioRecommended
Introducing caching for the first timeYes
Cache key or path config just changedYes
Correctness gate (release, deploy, integrity checks)Yes
Job depends on external state (network, registries)Yes
Fast iteration on non-critical jobsNo — cache speed is the priority
Job is intentionally nondeterministicNo — fix nondeterminism first

What quarantine looks like​

Quarantined entries are stored in .loom/.runtime/cache/<scope_hash>/quarantine.json. Each entry records the scope, key, reason (cache_diff_divergence), and timestamp.

To confirm a quarantine happened, check the cache system sections in the run logs:

.loom/.runtime/logs/<run_id>/jobs/<job_id>/system/cache_restore/events.jsonl
.loom/.runtime/logs/<run_id>/jobs/<job_id>/system/cache_save/events.jsonl

Quarantined entries produce log messages like:

cache restore skipped quarantined cache_name=pnpm key=pnpm-deps-abc123 scope_hash=... key_hash=...

Recovering from quarantine​

  1. Read the divergence context — start from the Diagnostics ladder and check the cache system section events.
  2. Identify the root cause:
    • Missing key inputs: add the missing file(s) to cache.key.files so the key changes when outputs should change.
    • Nondeterministic scripts: control sources of nondeterminism (timestamps, randomness, network state, ambient env vars).
    • Tool/version drift: include lockfiles and version configs in the key, or pin tool versions.
  3. Rotate the cache key: change cache.key.prefix (or the template string) to intentionally miss old entries and establish a new baseline.
  4. Clear quarantine: delete the entry from quarantine.json, or delete the file to clear all quarantines.

Cache key design checklist​

Use this before shipping any cache configuration:

  • Include all inputs that change outputs — lockfiles, build configs, tool version files. If you can't explain why a file is in the key, it probably doesn't belong.
  • Scope keys to avoid collisions — use distinct prefixes and/or ${job_name} so unrelated jobs don't overwrite each other.
  • Prefer multi-cache for different concerns — one cache entry per concern (e.g., pnpm vs go) so you can invalidate independently.
  • Use workspace-relative paths — absolute paths and paths containing .. are rejected. For Docker jobs, paths refer to directories inside the container's workspace mount at /workspace.
  • Decide on ${head_sha} — include it when correctness matters more than reuse (per-commit caches). Omit it when reuse matters more (branch/shared caches).
  • Validate with --cache-diff — prove determinism before trusting cache hits.