Skip to content

Optimize backend CI build: skip frontend, parallelize jobs#7435

Merged
galvana merged 9 commits intomainfrom
dockerfile-build-cache-optimization
Feb 20, 2026
Merged

Optimize backend CI build: skip frontend, parallelize jobs#7435
galvana merged 9 commits intomainfrom
dockerfile-build-cache-optimization

Conversation

@galvana
Copy link
Copy Markdown
Contributor

@galvana galvana commented Feb 19, 2026

Ticket ENG-2734

Description Of Changes

Optimize the Build job in backend_checks.yml which previously took ~15 minutes. The bottleneck was building the frontend (admin-ui + privacy-center) inside Docker with no caching across CI runs.

Three key changes:

  1. Skip the real frontend build entirely for backend CI. Backend tests don't need actual admin-ui files -- the runtime code returns a placeholder when files are missing, and all UI-related tests mock the file-resolution functions. A tiny placeholder index.html satisfies the Dockerfile's COPY --from=built_frontend while eliminating ~10 min of npm/turbo/Next.js compilation.

  2. Inject the placeholder via --build-context override. The Build job uploads a placeholder artifact and passes build-contexts: built_frontend=prebuilt to docker/build-push-action. This causes BuildKit to resolve COPY --from=built_frontend from the pre-built directory and skip the frontend and built_frontend Dockerfile stages entirely. No Dockerfile changes needed for this -- local builds and publish_docker.yaml are unaffected.

  3. Run Collect-Tests in parallel with Build instead of gating it. Collect-Tests is a syntax/import sanity check (pytest --collect-only) that produces no artifacts the Build job needs. Decoupling it from Build's needs saves ~2-3 min on the critical path while still failing the workflow independently on errors.

Additionally, the Dockerfile gets BuildKit cache mounts for local build speed (uv, npm, turbo, Next.js caches) and consolidated apt-get calls.

CI Performance

Stage Build & Setup Build Step Slowest Test Critical Path
main (before) 17.5 min 14.1 min 12.5 min 30.1 min
After optimization 4.5 min 1.7 min 12.9 min 17.6 min
Delta -13.0 min (-74%) -12.4 min (-88%) +0.3 min -12.6 min (-42%)
  • Build step dropped from 14.1 min to 1.7 min by skipping the frontend build and using a placeholder
  • Total critical path reduced from 30.1 min to 17.6 min (42% reduction)
  • Test durations unchanged -- all savings come from the build phase

Code Changes

  • .github/workflows/backend_checks.yml - Add Build-Frontend-Placeholder job that creates a minimal index.html and uploads it as an artifact (~5 seconds)
  • .github/workflows/backend_checks.yml - Modify Build job to download the placeholder artifact and pass build-contexts: built_frontend=prebuilt to skip frontend Dockerfile stages
  • .github/workflows/backend_checks.yml - Remove Collect-Tests from Build's needs so they run in parallel
  • Dockerfile - Add --mount=type=cache directives for uv, npm, turbo, and Next.js caches (benefits local builds)
  • Dockerfile - Consolidate two apt-get update && apt-get install calls into one (fewer layers)

Steps to Confirm

  1. Build-Frontend-Placeholder job completes in seconds
  2. Build job completes in ~5 min (down from ~15 min)
  3. Collect-Tests runs in parallel with Build, not blocking it
  4. All downstream pytest jobs pass (container boots, tests run correctly)
  5. publish_docker.yaml and local builds (nox -s "build(slim)") are unaffected

Pre-Merge Checklist

  • All CI pipelines succeeded
  • CHANGELOG.md updated
  • UX feedback:
    • No UX review needed
  • Followup issues:
    • No followup issues
  • Database migrations:
    • No migrations
  • Documentation:
    • No documentation updates required

Add cache mounts for npm, turbo, Next.js, and uv caches so
incremental frontend and Python dependency builds can reuse
previously compiled artifacts. Also consolidate two apt-get
install calls in compile_image into one.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Feb 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated (UTC)
fides-plus-nightly Ignored Ignored Preview Feb 20, 2026 0:36am
fides-privacy-center Ignored Ignored Feb 20, 2026 0:36am

Request Review

Adrian Galvan and others added 6 commits February 19, 2026 13:23
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Move admin-ui build out of Docker into a separate Build-Frontend job
that runs in parallel with Collect-Tests. The pre-built output is
uploaded as an artifact and injected into the Docker build via
--build-context, causing BuildKit to skip the frontend stages entirely.

Co-authored-by: Cursor <cursoragent@cursor.com>
Backend tests don't need real admin-ui files -- the runtime code
returns a placeholder when files are missing, and all UI tests mock
the file-resolution functions. A single index.html satisfies the
Dockerfile COPY while eliminating the ~4 min npm ci + export step.

Co-authored-by: Cursor <cursoragent@cursor.com>
Collect-Tests is a syntax/import sanity check that produces no
artifacts Build needs. Running them in parallel shaves ~2-3 min
off the critical path while still failing the workflow on errors.

Co-authored-by: Cursor <cursoragent@cursor.com>
@galvana galvana changed the title Add BuildKit cache mounts to speed up Docker builds Optimize backend CI build: skip frontend, parallelize jobs Feb 19, 2026
@galvana galvana marked this pull request as ready for review February 19, 2026 23:33
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Feb 19, 2026

Greptile Summary

This PR optimizes the backend CI build pipeline by eliminating unnecessary frontend compilation and adding BuildKit cache mounts. The changes reduce the Build job duration from ~15 minutes to ~5 minutes through three key optimizations:

  • Frontend placeholder for backend CI: Replaces the full frontend build (admin-ui + privacy-center) with a minimal index.html placeholder in backend_checks.yml. This is safe because backend tests either don't need UI files or mock the file-resolution functions (get_admin_index_as_response returns a placeholder when files are missing, as verified in src/fides/api/ui.py:45-50).

  • BuildKit build-context override: Uses build-contexts: built_frontend=prebuilt to inject the placeholder artifact, causing BuildKit to skip the frontend and built_frontend Dockerfile stages entirely without modifying the Dockerfile itself. Production builds (publish_docker.yaml) and local builds remain unchanged.

  • Parallel job execution: Removes Collect-Tests from Build's dependencies since it's an independent syntax check that produces no artifacts needed by downstream jobs.

  • BuildKit cache mounts: Adds --mount=type=cache directives for uv, npm, turbo, and Next.js caches to speed up local and CI builds with warm caches. Also consolidates apt-get commands to reduce Docker layers.

The implementation is clean and well-architected. All changes are localized to CI configuration with no impact on production deployments.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are well-designed CI optimizations with clear separation between test and production builds. The frontend placeholder approach is validated by existing code that handles missing UI files gracefully. BuildKit cache mounts are standard Docker best practices. The parallel job restructuring is safe since Collect-Tests produces no artifacts needed by Build. All CI checks passed successfully.
  • No files require special attention

Important Files Changed

Filename Overview
.github/workflows/backend_checks.yml Added Build-Frontend-Placeholder job to create minimal index.html, modified Build to use placeholder via build-contexts override, removed Collect-Tests dependency to enable parallel execution
Dockerfile Added BuildKit cache mounts for uv, npm, turbo, and Next.js caches; consolidated apt-get commands into single RUN for fewer layers

Last reviewed commit: cf9822a

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Copy Markdown
Contributor

@JadeCara JadeCara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚤

@galvana galvana enabled auto-merge February 20, 2026 00:34
@galvana galvana added this pull request to the merge queue Feb 20, 2026
Merged via the queue into main with commit 76b37d0 Feb 20, 2026
51 checks passed
@galvana galvana deleted the dockerfile-build-cache-optimization branch February 20, 2026 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants