Skip to content

Do Selective Builds for Monorepos

In large monorepo projects, building and deploying all services for every code change can be time-consuming and resource-intensive. Coherence offers a selective build feature that allows you to specify which services should be built and deployed based on the files changed in a given commit or push event.

Understanding Globs

Globs, or glob patterns, are wildcard patterns used to match file paths. They provide a powerful way to specify sets of files or directories. Here are some common glob patterns:

  • *: Matches any number of characters (except path separators)
  • **: Matches any number of characters including path separators
  • ?: Matches a single character
  • [abc]: Matches one character given in the bracket
  • [!abc]: Matches one character not given in the bracket

For example: - *.js matches all JavaScript files in the current directory - **/*.js matches all JavaScript files in any subdirectory - src/[!test]*.js matches all JavaScript files in the src directory except those starting with "test"

Configuration

Coherence uses glob settings to control which services in your monorepo will be built and deployed when a GitHub push event webhook is received. These settings are configured in the cnc.yml file.

Default Globs Based on Build Context

Important: If your service has a build context defined (builds from a specific directory in the repo and not from the repo root), Coherence will automatically use that context as the default glob pattern for the service.

For example, if your service has a build context of ./frontend, Coherence will automatically use frontend/**/* as the default included glob for that service.

Basic Structure

The build glob config is set under the x-coherence.build_config key for each service:

services:
  app:
    x-coherence:
      build_config:
        included_globs: ["pattern1", "pattern2"]
        excluded_globs: ["pattern3", "pattern4"]
    # ... other service configuration ...

included_globs

The included_globs array specifies patterns for files that, when modified, will trigger a build and deployment for this service. If any changed files match one or more of these patterns, the service will be included in the pipeline.

Example: 

included_globs: ["src/**/*", "config/*.json"]
This would include the service if any file under the src directory or any JSON file in the config directory was modified.

excluded_globs

The excluded_globs array specifies patterns for files that, when modified, will prevent the service from being built and deployed, even if they match an included_glob. This is useful for excluding certain files or directories that don't affect the service's functionality.

Example: 

excluded_globs: ["**/*.md", "src/tests/**/*"]
This would exclude the service if only Markdown files or files in the src/tests directory were modified, even if they matched an included_glob.

How Selective Builds Work

  1. When Coherence receives a GitHub push event, it analyzes the list of modified files.
  2. For each service in your cnc.yml: a. It checks if any modified files match the included_globs (or the default glob based on build context if not specified). b. If there's a match, it then checks if any of those files also match the excluded_globs. c. If there are matches in included_globs but not in excluded_globs, the service is included in the build pipeline.
  3. Only the services that pass this check are built and deployed.

Best Practices

  • Leverage the automatic build context-based globs when possible.
  • Use included_globs to expand the scope beyond the build context if necessary.
  • Use excluded_globs to fine-tune and exclude specific files or directories.
  • Consider your project structure carefully when defining globs.
  • Test your glob patterns to ensure they behave as expected.
  • Use comments in your cnc.yml to explain the reasoning behind complex glob patterns.

Example

Here's an expanded example showcasing selective builds for a monorepo with multiple services:

services:
  frontend:
    build:
      context: ./frontend
    x-coherence:
      build_config:
        included_globs: ["shared/**/*"]  # In addition to the automatic "frontend/**/*"
        excluded_globs: ["**/*.md", "**/*.test.js"]

  backend:
    build:
      context: ./backend
    x-coherence:
      build_config:
        included_globs: ["shared/**/*", "database/migrations/*"]  # In addition to the automatic "backend/**/*"
        excluded_globs: ["**/*.md", "**/*.test.js", "backend/tests/**/*"]

  api2:
    build:
      context: ./api2
    x-coherence:
      build_config:
        included_globs: ["shared/**/*"]  # In addition to the automatic "worker/**/*"
        excluded_globs: ["**/*.md", "api2/scripts/**/*"]

In this example:

  • Each service has its build context defined, so Coherence automatically includes files from those directories.
  • Additional included_globs are specified to include shared resources or other relevant files outside the build context.
  • excluded_globs are used to ignore certain file types or directories across all services.
  • The backend service will also build if database migrations are changed.
  • The api2 service ignores changes to scripts in its directory.

By utilizing these selective build settings, you can significantly optimize your CI/CD pipeline, reducing build times and resource usage while ensuring that services are only rebuilt when necessary.