Pipeline Caching Strategies for Monorepo CI Builds

Monorepos have gone from a curiosity at a couple of large tech companies to standard practice: over 63% of companies with 50 or more developers now run one. That shift has quietly broken the CI playbook most teams still use, and the fix isn't one clever trick but a layered caching strategy where dependency caching, build artifact caching, and remote distributed caching each cover a failure mode the others can't touch.

Single-repo CI is a known quantity. One change triggers one test suite, and the pipeline duration barely moves from commit to commit. Monorepo CI doesn't work that way. A one-line fix to a README can trigger the exact same pipeline load as a rewrite of a core library, because naive CI configuration doesn't distinguish between the two. Take a 30-package repo with no path filtering: every push runs all 30 test suites, and once matrix testing enters the picture (multiple runtime versions, multiple OS targets), a single pull request can spin up 60 to 90 jobs.

The dependency graph makes this worse before it makes it better. If package C changes, and packages A and B both import it, a correct pipeline has to test A and B too, or it risks shipping a downstream breakage that never got exercised. If teams miss that logic, they end up on one of two bad paths: either they test everything on every change (wasteful), or they test only what changed directly and silently miss the packages one layer upstream (dangerous). Resource limits turn this into an operational problem as well as a design one. GitHub Actions' free tier caps concurrency at 20 jobs, so a 90-job pull request just queues, and the platform's 10 GB cache quota is shared across every workflow in the repo. A poorly keyed cache doesn't just fail to help. It evicts the caches other jobs actually need.

What academic research says about CI caching effectiveness

The counterintuitive part is this: most projects that turn on caching never see their build times improve. That's an argument that caching is easy to configure badly, and the data backs it up.

An empirical study by Ghaleb et al., "The Promise and Reality of Continuous Integration Caching: An Empirical Study of Travis CI Builds" (arXiv:2601.19146), looked at 513,384 builds across 1,279 GitHub projects and found that only 30% of projects adopt CI caching. Adoption correlates with project maturity, meaning projects with more dependencies and a longer commit history are more likely to bother. But adopting caching and benefiting from it turn out to be different things: two-thirds of projects that add caching see no reduction in build time, and 80% of those don't maintain their caching configuration once it's in place. It gets set up once and then quietly rots.

Part of the reason is a cost nobody budgets for. Cache uploads happen in 97% of builds and take roughly six times longer than downloads, largely because teams end up caching things that change on every run, like installation logs, which undermines the purpose of caching. On top of that, the same study found that only one-third of projects see substantial build-time reductions from caching. A cache that's configured but never properly maintained is arguably worse than having no cache at all, since it gives a false sense that the pipeline is optimized when it's delivering little real benefit.

The most telling finding, though, is about why non-adopters don't adopt. Researchers submitted pull requests that simply enabled caching in projects that had never used it, and nearly half were accepted. That's not a story about teams deliberately avoiding caching because it's not worth the complexity. It's a story about plain unfamiliarity with what CI caching support already exists in the tools they're using.

Affected-only execution: the prerequisite that makes every cache layer count

Before any caching layer matters, the pipeline has to know what actually changed. Running four affected packages instead of 45 will outperform any caching setup, full stop. Affected-only execution is the single biggest lever available in monorepo CI, and it has to be right before caching is worth tuning.

The mechanism is fairly simple in concept. Change detection diffs against the base branch for a pull request, or against the previous commit for a push to main. That comparison decides which packages need to run. But a naive diff isn't enough: the tooling also has to walk the dependency graph upstream, because changing a shared library needs to trigger tests for every package that imports it, not just the package that changed. And certain changes should escalate to a full run regardless of the graph, changes to lockfiles, workspace configuration, or build tool config at the repo root, since those can affect everything downstream. Turborepo and Nx both handle this kind of root-change escalation, but it has to be checked in configuration, not assumed.

Several footguns break this quietly. A common footgun is a shallow clone that doesn't contain enough git history for either Turborepo or Nx to find the merge base; ensuring a full history checkout is the fix. Pull request and push events also behave differently: Turborepo's --affected flag automatically reads GITHUB_BASE_REF inside GitHub Actions, though older setups sometimes still use the manual filter syntax [origin/main...HEAD]. Nx requires explicit base-SHA configuration to get the same base-comparison behavior. Dynamic affected detection can race against merges landing on main while the pipeline runs, so pinning the base SHA at the start of the job avoids comparing against a moving target.

One practitioner account cited in coverage of this space describes cutting pipeline time from 52 minutes down to 8, using a combination of path filtering and caching together. Neither one alone gets there. Affected-only execution decides how much work needs to happen; caching decides how fast that reduced amount of work actually runs.

Layer one: dependency caching and cache key design

Dependency caching solves one specific problem: skipping the re-download of packages when the lockfile hasn't changed. The lockfile hash is almost always the right granularity for the cache key, because it's the one signal that reliably tells you whether the dependency tree actually changed.

Used well, this layer alone is worth something like a 15 to 25% reduction in build time on a mid-size project. Teams that stop here are leaving most of the possible speedup on the table.

Cache key design is where most of the damage from the research above actually happens. The classic mistake is including github.sha directly in the cache key without setting up restore keys as a fallback: every single commit then generates a brand-new cache entry, nothing ever gets reused, and the pipeline pays the full upload-and-download cost of caching without getting any of the benefit. A subtler version of the same mistake is keying on branch name instead of lockfile hash, which can quietly drive the same kind of cache waste without producing any obvious error. A badly keyed cache doesn't just fail to help, it actively evicts the caches other jobs depend on, because the quota is shared. The SHA has a place, but it belongs in the restore-keys fallback chain, not as the primary key for a dependency cache.

There's also a workspace-granularity question monorepos have to answer directly. Separate lockfiles per package are more explicit and sidestep the merge conflicts that a single monorepo-wide lockfile can produce. But the more common pattern is a shared install job: run install-dependencies once as a matrix job, and let every downstream job that needs dependencies, test-api, lint-api, whatever else, pull from that one cache instead of each running its own install. Without that shared step, two jobs on the same commit will both try to install dependencies independently, which wastes both time and cache bandwidth for no benefit.

Layer two: build artifact caching in monorepo tools

Build artifact caching operates one level up from dependency caching. If the inputs to a build task are identical to a previous run, the system reuses the output instead of rerunning the build, and it does this at the level of the task, not the individual file.

Stacked on top of dependency caching, build artifact caching produces the real gains, pushing total build-time reduction toward the 50 to 70% range. Stacked on top of dependency caching, build artifact caching pushes total build-time reduction toward the 50 to 70% range. The dependency layer buys a modest, reliable win; the artifact layer is where a 40-minute pipeline actually starts looking like an 8-minute one.

Turborepo is built around this idea by default. It automatically figures out task dependencies, orders builds correctly based on that graph, and caches the result of each task as it completes. If the pipeline runs again with no relevant changes, Turborepo reports "FULL TURBO," pulling every result straight from cache almost instantly instead of rebuilding anything. Vendor claims put the speedup at up to 10x for monorepo development through intelligent caching and remote artifact sharing, and Turborepo has become a widely adopted choice for JS monorepos going into 2026.

Local cache alone, though, doesn't solve CI. CI runners are ephemeral by nature, they spin up, run the job, and disappear, so whatever gets cached locally on that runner doesn't persist to the next run, let alone to a different workflow or a teammate's machine. That's the gap remote caching exists to close, and it's the natural bridge into the next layer.

Layer three: Docker layer caching and the traps that make it fail silently

CI runners start from nothing every time. Without a shared cache, a build using a popular containerization tool is a full rebuild from the base image up on every single run, and layer caching exists specifically to avoid that by reusing layers that haven't changed, which can turn a multi-minute build into one that finishes in seconds.

There are three real approaches to keeping that cache alive across ephemeral runners, and they trade off differently. Registry-backed caching, using BuildKit's --cache-to and --cache-from flags, pushes the cache layers to the same container registry that holds the image itself, then pulls them back down at the start of the next build. It's the most reliable option precisely because it doesn't depend on anything specific to the CI provider, it works across any set of ephemeral runners as long as they can reach the registry.

CI-native caching, set through cache-from: type=gha and cache-to: type=gha,mode=max in something like docker/build-push-action, is more convenient because BuildKit serializes the cache directly into GitHub's own cache storage with no registry step required. But it comes with a real trap: GitHub's default cache limit is 10 GB, and in a monorepo running multiple image builds, that ceiling gets hit fast. Once it does, eviction kicks in and wipes out cached layers, a failure that looks like caching is broken rather than a quota problem. Once a repo is bumping against that ceiling, registry-backed caching is the better route.

BuildKit cache mounts, using --mount=type=cache, solve a different problem: they let package manager directories, ~/.npm, a Maven local repository, whatever the language's dependency cache is, persist across build invocations. That's complementary to the other two approaches, not a replacement, since it's solving for within-build efficiency rather than cross-run persistence.

Without mode=max, only the final stage of a multi-stage build gets cached, a gotcha buried in the type=gha setup that catches teams by surprise. Every intermediate stage rebuilds from scratch on every run, silently, with no error to flag it. The build finishes and doesn't complain, which makes it look like caching is working. It's just not caching the parts that usually take the longest.

Dockerfile structure matters as much as the caching flags themselves. Splitting dependency-installation stages from build stages as aggressively as possible means the dependency layer only invalidates when the lockfile actually changes, not on every commit that touches application code. The more granular those stages are, the higher the cache hit rate, which is really the same lesson as the lockfile-hash key design from layer one, applied at the level of Dockerfile structure instead of CI cache keys.

Remote and distributed caching: sharing cache hits across the whole team

Local caching speeds up one run on one machine. Remote caching is a different category of improvement entirely: every CI run, and every developer's local machine, can share build results across the whole team, not just across runs on the same box.

That distinction has a compounding effect when it's layered on top of affected-only execution. Fewer packages need to run because the affected-detection logic already narrowed the scope, and the ones that do run are very likely already cached from a prior build somewhere in the system. A build output computed once, by one CI runner, becomes instantly reusable by a different runner, a different workflow, or a developer working locally who happens to be building the same unchanged package.

Turborepo's remote cache needs a remote endpoint to talk to. Vercel provides one, and self-hosted alternatives exist for teams that want to keep that infrastructure in-house. The Mercari result cited earlier, the reduction attributed to Turborepo's remote cache setup came from sharing previously computed results across CI runs. That's the mechanism doing the work: the second, third, and hundredth CI run all draw from the same pool of previously computed results.

Bazel and Pants take this further with remote cache servers designed to share build and test results across runs at a fine-grained leveld targets, caching both build outputs and test results rather than just build artifacts. That granularity is built for monorepos operating at a scale where even Turborepo's task-level caching isn't fine-grained enough; with enough packages and enough build targets, per-target caching is the only way the pipeline finishes in a reasonable amount of time.