Skip to main content
info

This documentation is automatically synchronized from the claude-hub repository. Last updated: 2025-06-01

Docker Build Optimization Guide

This document describes the optimizations implemented in our Docker CI/CD pipeline for faster builds and better caching.

Overview​

Our optimized Docker build pipeline includes:

  • Self-hosted runner support with automatic fallback
  • Multi-stage builds for efficient layering
  • Advanced caching strategies
  • Container-based testing
  • Parallel builds for multiple images
  • Security scanning integration

Self-Hosted Runners​

Configuration​

  • Labels: self-hosted, linux, x64, docker
  • Usage: All Docker builds use self-hosted runners by default for improved performance
  • Local Cache: Self-hosted runners maintain Docker layer cache between builds
  • Fallback: Configurable via USE_SELF_HOSTED repository variable

Runner Setup​

Self-hosted runners provide:

  • Persistent Docker layer cache
  • Faster builds (no image pull overhead)
  • Better network throughput for pushing images
  • Cost savings on GitHub Actions minutes

Fallback Strategy​

The workflow implements a flexible fallback mechanism:

  1. Default behavior: Uses self-hosted runners (self-hosted, linux, x64, docker)
  2. Override option: Set repository variable USE_SELF_HOSTED=false to force GitHub-hosted runners
  3. Timeout protection: 30-minute timeout prevents hanging on unavailable runners
  4. Failure detection: build-fallback job provides instructions if self-hosted runners fail

To manually switch to GitHub-hosted runners:

# Via GitHub UI: Settings → Secrets and variables → Actions → Variables
# Add: USE_SELF_HOSTED = false

# Or via GitHub CLI:
gh variable set USE_SELF_HOSTED --body "false"

The runner selection logic:

runs-on: ${{ fromJSON(format('["{0}"]', (vars.USE_SELF_HOSTED == 'false' && 'ubuntu-latest' || 'self-hosted, linux, x64, docker'))) }}

Multi-Stage Dockerfile​

Our Dockerfile uses multiple stages for optimal caching and smaller images:

  1. Builder Stage: Compiles TypeScript
  2. Prod-deps Stage: Installs production dependencies only
  3. Test Stage: Includes dev dependencies and test files
  4. Production Stage: Minimal runtime image

Benefits​

  • Parallel builds of independent stages
  • Smaller final image (no build tools or dev dependencies)
  • Test stage can run in CI without affecting production image
  • Better layer caching between builds

Caching Strategies​

1. GitHub Actions Cache (GHA)​

cache-from: type=gha,scope=${{ matrix.image }}-prod
cache-to: type=gha,mode=max,scope=${{ matrix.image }}-prod

2. Registry Cache​

cache-from: type=registry,ref=${{ org }}/claude-hub:nightly

3. Inline Cache​

build-args: BUILDKIT_INLINE_CACHE=1
outputs: type=inline

4. Layer Ordering​

  • Package files copied first (changes less frequently)
  • Source code copied after dependencies
  • Build artifacts cached between stages

Container-Based Testing​

Tests run inside Docker containers for:

  • Consistent environment
  • Parallel test execution
  • Isolation from host system
  • Same environment as production

Test Execution​

# Unit tests in container
docker run --rm claude-hub:test npm test

# Integration tests with docker-compose
docker-compose -f docker-compose.test.yml run integration-test

# E2E tests against running services
docker-compose -f docker-compose.test.yml run e2e-test

Build Performance Optimizations​

1. BuildKit Features​

  • DOCKER_BUILDKIT=1 for improved performance
  • --mount=type=cache for package manager caches
  • Parallel stage execution

2. Docker Buildx​

  • Multi-platform builds (amd64, arm64)
  • Advanced caching backends
  • Build-only stages that don't ship to production

3. Context Optimization​

  • .dockerignore excludes unnecessary files
  • Minimal context sent to Docker daemon
  • Faster uploads and builds

4. Dependency Caching​

  • Separate stage for production dependencies
  • npm ci with --omit=dev for smaller images
  • Cache mount for npm packages

Workflow Features​

PR Builds​

  • Build and test without publishing
  • Single platform (amd64) for speed
  • Container-based test execution
  • Security scanning with Trivy

Main Branch Builds​

  • Multi-platform builds (amd64, arm64)
  • Push to registry with :nightly tag
  • Update cache images
  • Full test suite execution

Version Tag Builds​

  • Semantic versioning tags
  • :latest tag update
  • Multi-platform support
  • Production-ready images

Security Scanning​

Integrated Scanners​

  1. Trivy: Vulnerability scanning for Docker images
  2. Hadolint: Dockerfile linting
  3. npm audit: Dependency vulnerability checks
  4. SARIF uploads: Results visible in GitHub Security tab

Monitoring and Metrics​

Build Performance​

  • Build time per stage
  • Cache hit rates
  • Image size tracking
  • Test execution time

Health Checks​

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:3002/health"]
  interval: 30s
  timeout: 10s
  retries: 3

Local Development​

Building locally​

# Build with BuildKit
DOCKER_BUILDKIT=1 docker build -t claude-hub:local .

# Build specific stage
docker build --target test -t claude-hub:test .

# Run tests locally
docker-compose -f docker-compose.test.yml run test

Cache Management​

# Clear builder cache
docker builder prune

# Use local cache
docker build --cache-from claude-hub:local .

Best Practices​

  1. Order Dockerfile commands from least to most frequently changing
  2. Use specific versions for base images and dependencies
  3. Minimize layers by combining RUN commands
  4. Clean up package manager caches in the same layer
  5. Use multi-stage builds to reduce final image size
  6. Leverage BuildKit features for better performance
  7. Test in containers for consistency across environments
  8. Monitor build times and optimize bottlenecks

Troubleshooting​

Slow Builds​

  • Check cache hit rates in build logs
  • Verify .dockerignore is excluding large files
  • Use --progress=plain to see detailed timings
  • Consider parallelizing independent stages

Cache Misses​

  • Ensure consistent base image versions
  • Check for unnecessary file changes triggering rebuilds
  • Use cache mounts for package managers
  • Verify registry cache is accessible

Test Failures in Container​

  • Check environment variable differences
  • Verify volume mounts are correct
  • Ensure test dependencies are in test stage
  • Check for hardcoded paths or ports