Skip to Content
Test Result Caching

Skip the Apex tests whose source and dependencies haven’t changed, and replay their last pass/fail in milliseconds instead of re-running them.

The test result cache stores one outcome per test method, keyed to the test’s source and the full set of classes it transitively depends on. When nothing that can change a test’s outcome has changed since its last run, LATdx returns the cached pass/fail in milliseconds instead of re-executing it. The cache is on by default; bypass it for one run with --no-cache.

Cache Correctness

The cache is built correctness-first. It skips a test only when it can prove the cached outcome matches what a fresh run would produce; on any uncertainty it falls through to the SF API and runs the test. The guarantee: never slower than running without the cache, and never a false pass.

Five mechanisms keep cached results correct:

  1. Phased skip rules. Skips are organized into phases 0 through 4, each adding more aggressive rules than the last. Phase 0 is the safe baseline: every test runs through the SF API (results are cached but never reused). Phase 4, the default, adds per-method body, schema-closure, and dynamic-pessimism rules. A phase becomes the default only after it passes the corpus check below; pin a lower phase at any time with LATDX_CACHE_PHASE.
  2. Empirical gating. Each phase promotion requires a fixture corpus pass rate of 100% on correctness invariants. Engineering judgment alone does not authorize a skip rule; the corpus is the gate.
  3. Three-level kill switch. LATDX_CACHE_DISABLE=1 turns the cache fully off. LATDX_CACHE_PHASE=N pins it to a known-stable phase. LATDX_CACHE_RULE_<NAME>=off disables a specific skip rule. None requires a redeploy.
  4. Explainable decisions. Every cache decision is explainable via latdx test cache explain <Class>.<method>, which prints the classes the method depends on, the hash, and the rule that authorized the decision.
  5. Shadow validation. On runs that already execute some tests, a small sample of cache hits is re-run on the org and compared against the cached result. A disagreement is reported and the fresh result wins, so a stale skip surfaces instead of passing silently. Sampling only piggybacks on runs that are already happening, never on a full cache hit, so it adds no extra org round-trip. Tune the sample with LATDX_CACHE_SHADOW_RATE (default 1%; set 0 to disable).

Overview

The cache stores one outcome per (org, test class, test method) keyed by a SHA256 hash of:

  1. The test class source.
  2. The source of every class the test transitively references (AST-derived closure).
  3. An org-level schema revision digest (covers the org schema snapshot).

If any of those inputs change, the hash changes and the next run is a miss.

Storage: one JSON file per test class at:

~/.latdx/workspaces/<id>/cache/<orgId>/test-results/<testClass>.json

Per-worktree daemons scope their cache under the workspace state dir; standalone runs fall back to ~/.latdx/cache/<orgId>/test-results/.

Disable

The cache is on by default. Skip it for a single run, a shell session, or a specific rule:

# One-shot bypass: run without the cache for this invocation. latdx test run -f MyTest.cls -o my-org --no-cache # Hard off for the shell session: every request runs through the SF API. LATDX_CACHE_DISABLE=1 latdx test run -f MyTest.cls -o my-org # Pin to a specific phase. Useful if a later phase has been rolled back. LATDX_CACHE_PHASE=1 latdx test run -f MyTest.cls -o my-org # Disable a single skip rule by name (rules are listed in `latdx test cache status`). LATDX_CACHE_RULE_CLOSURE_INVARIANCE=off latdx test run -f MyTest.cls -o my-org

Switches are documented in environment variables.

Run Summary

When the cache layer is active the run prints a trailing block in the Summary that surfaces hit ratio and saved time:

Tests PASS 19 FAIL 0 SKIP 19 (cached) Wall 357ms (saved ~2.3s) Cache ↺ 19/19
  • Wall: wall-clock duration of the run. When the cache served at least 1s of measured execution this run, a dim (saved ~Xs) suffix follows: Wall 357ms (saved ~2.3s). Sub-second saves are dropped (noise threshold). See What Saved means for the methodology and honest caveats.
  • Cache: ↺ hits/total (total = hits + misses). The glyph is amber at full intensity; the same shape in dim-amber appears inline on each cached method row so the eye threads the per-method markers to the summary roll-up. No per-outcome breakdown by default; for the underlying counters (misses, stale, written, phase) see the --json envelope’s result.cacheReport or latdx test cache status.

When --no-cache or LATDX_CACHE_DISABLE=1 is set the cache layer is bypassed entirely; the Cache row is suppressed, the Wall row drops the (saved ~Xs) suffix, and the run header echoes Cache: off.

In --json mode the full payload appears as result.cacheReport, including phase, killSwitch, hits, misses, stale, writtenOnExit, wouldHaveHit, savedNowMs, savedTodayMs, and savedOverallMs.

What Saved means

The (saved ~Xs) suffix on the Wall row reports the sum of measured durationMs from the cached methods served this run. The persistent counters (today + overall) live in <cacheHome>/cache/<orgId>/saved.json and surface via the --json envelope’s result.cacheReport.savedTodayMs / savedOverallMs. The suffix is rounded with the same precision as the Wall value (one decimal under 10s, whole seconds above); sub-second saves are dropped as noise.

Every number is a sum of real measurements: no extrapolation, no “would have taken” guesses.

Honest caveats

Sum-of-durations answers “how much measured execution would have run again if the cache were empty?” It is not the wall-clock time you actually waited less, for two reasons that pull in opposite directions:

  1. Tests run in parallel, so the wall-clock saving is smaller than the sum. Salesforce’s runTestsAsynchronous runs typically 5 concurrent test classes; LATdx’s packed-anon engine runs up to 20 (LATDX_MAX_CONCURRENCY). For a fully-parallel workload the wall-clock saving is roughly Saved / parallelism, 5x to 20x smaller than the sum.
  2. All-cached runs skip the entire test-queue submission + poll loop, so wall-clock saving can be larger than the sum. That overhead (typically 5-15s) is not in any durationMs field. A small all-cached run that reads “saved 250ms” may have spared you ~12s of waiting.

For most TDD inner-loop scenarios the two effects partially cancel and the suffix is a useful “did the cache help” signal at the right order of magnitude. Treat it as an indicator, not a stopwatch.

The methodology and the phased plan to refine these numbers live in docs/internal/saved-time-methodology.md.

Invalidation

Entries expire automatically on any of these axes:

  1. Input hash change: editing the test class or any class it depends on shifts the cached inputHash, so the next run is a miss and writes a fresh entry. This is the primary invalidation path today.
  2. TTL: 7 days since the entry was recorded.
  3. Schema bump: a CACHE_VERSION change in a CLI upgrade drops every pre-bump entry.
  4. Org mismatch: entries for one org are never served to another.
  5. Explicit clear: see the commands below.

When the daemon is running its FileWatcher translates every .cls add / change / unlink into a cache wipe for the changed class and the test classes that depend on it (via the AST-derived closure). Dependent entries drop on save, so the next run is a miss before TTL or hash rolling. With the daemon stopped, file edits still invalidate via the input-hash path: the new run is a miss, the cache writes through, and the prior entry is collapsed by the per-method rotation.

LATDX_CACHE_DISABLE=1 suppresses the FileWatcher hook (no cache state to manage when fully off). The hook runs at every other phase, including Phase 0; the resulting stale counter on the run summary reflects cache entries whose input hash no longer matches.

Cache Management Commands

# Show entry count, disk usage, and age range for the current org. latdx test cache status latdx test cache status -o my-org # Drop entries for the current org. latdx test cache clear latdx test cache clear -o my-org # Drop entries for every org in this workspace. latdx test cache clear --all-orgs # Print the dependencies used to compute the hash for one method. # Useful when a hit or miss is unexpected. latdx test cache explain MyTestClass.myTestMethod -o my-org

latdx cache clear (the existing top-level command) also drops test-results alongside other caches. Use latdx test cache clear when you only want to reset this tier.

Known Limitations

The dependency graph is AST-derived and therefore cannot follow dynamic Apex:

  • Type.forName("Foo").newInstance() touches Foo by string.
  • Database.query(String dynamicSoql) can reference objects not named in static SOQL literals.
  • Reflection through Callable or SObjectType.fromName lookups.

If a test exercises dynamic dispatch and the referenced code changes, the cache may serve a stale pass/fail. Workarounds:

  • Run with --no-cache when iterating on dynamically dispatched code.
  • Use latdx test cache clear after the relevant change lands.

Report confirmed false hits: false-hit rate is one of the gates that controls how aggressively the cache is allowed to skip tests in future releases.

Troubleshooting

  • Unexpected miss. Run latdx test cache explain <Class>.<method> to see which dependencies contribute to the hash. A newly added dependency will change the hash.
  • Unexpected hit after editing code. Confirm the daemon is running (latdx daemon status); its FileWatcher drops dependent entries on save. Without it, invalidation falls back to the input-hash path (see Invalidation).
  • Disk growth. latdx test cache status reports disk usage. Clear the tier with latdx test cache clear if it outgrows your budget.
Last updated on