Skip to content

Do Selective Builds for Monorepos

In large monorepo projects, building and deploying all services for every code change can be time-consuming and resource-intensive. Coherence offers a selective build feature that allows you to specify which services should be built and deployed based on the files changed in a given commit or push event.

Understanding globs

Globs, or glob patterns, are wildcard patterns used to identify file paths that match a given pattern. They provide a powerful means of specifying sets of files or directories.

Consider the following common glob patterns:

  • The * glob matches any number of characters, except path separators.
  • The ** glob matches any number of characters, including path separators.
  • The ? glob matches a single character.
  • The [abc] glob matches the character enclosed in the brackets.
  • The [!abc] glob negates characters that match the character enclosed in the brackets.

For example:

  • The *.js glob matches all JavaScript files in the current directory.
  • The **/*.js glob matches all JavaScript files in any subdirectory.
  • The src/[!test]*.js glob matches all JavaScript files in the src directory, except those starting with "test".

Configuration

Coherence uses glob settings to control which services in a monorepo will be built and deployed when a GitHub push event webhook is received. These settings are configured in the cnc.yml file.

Default globs based on build context

Important: If your service has a build context defined (builds from a specific directory in the repo and not from the repo root), Coherence will automatically use that context as the default glob pattern for the service.

For example, if your service has a build context of ./frontend, Coherence will automatically use frontend/**/* as the default included glob for that service.

Basic structure

In the cnc.yml, the build glob configuration is set under the x-coherence.build_config key for each service:

services:
  app:
    x-coherence:
      build_config:
        included_globs: ["pattern1", "pattern2"]
        excluded_globs: ["pattern3", "pattern4"]
    # ... other service configuration ...

Included globs

The included_globs array defines file patterns that trigger a build and deployment for this service when the files are modified. If any changed files match one or more of these patterns, the service will be included in the pipeline.

For example:

included_globs: ["src/**/*", "config/*.json"]

This configuration ensures that the service is built and deployed whenever any file in the src directory or any JSON file in the config directory is modified.

Excluded globs

The excluded_globs array specifies patterns for files that, when modified, will prevent the service from being built and deployed even if the files also match an included_glob. This is useful for excluding certain files or directories that don't affect the service's functionality.

For example:

excluded_globs: ["**/*.md", "src/tests/**/*"]

This configuration prevents the service from being built and deployed when Markdown files or files in the src/tests directory are modified, even if the files match an included_glob.

How selective builds work

When Coherence receives a GitHub push event, it analyzes the list of modified files.

For each service in the cnc.yml, Coherence checks whether any of the modified files match the included_globs (or the default glob based on build context, if no included_globs array is specified):

  • If there is a match, Coherence then checks whether any of the modified files also match the excluded_globs.
  • If there are matches in included_globs but not in excluded_globs, the service is included in the build pipeline.

Only the services that pass this check are built and deployed.

Best practices

  • Leverage the automatic build context-based globs when possible.
  • Use the included_globs array to expand the scope beyond the build context if necessary.
  • Use the excluded_globs array to fine-tune and exclude specific files or directories.
  • Consider your project structure carefully when defining globs.
  • Test your glob patterns to ensure they behave as expected.
  • Use comments in your cnc.yml to explain the reasoning behind complex glob patterns.

Example

This expanded example showcases selective builds for a monorepo with multiple services.

services:
  frontend:
    build:
      context: ./frontend
    x-coherence:
      build_config:
        included_globs: ["shared/**/*"]  # In addition to the automatic "frontend/**/*"
        excluded_globs: ["**/*.md", "**/*.test.js"]

  backend:
    build:
      context: ./backend
    x-coherence:
      build_config:
        included_globs: ["shared/**/*", "database/migrations/*"]  # In addition to the automatic "backend/**/*"
        excluded_globs: ["**/*.md", "**/*.test.js", "backend/tests/**/*"]

  api2:
    build:
      context: ./api2
    x-coherence:
      build_config:
        included_globs: ["shared/**/*"]  # In addition to the automatic "worker/**/*"
        excluded_globs: ["**/*.md", "api2/scripts/**/*"]

In this example:

  • Each service has its build context defined, so Coherence automatically includes files from those directories.
  • Additional included_globs are specified to include shared resources or other relevant files outside the build context.
  • The excluded_globs are used to ignore certain file types or directories across all services.
  • The backend service will also build if database migrations are changed.
  • The api2 service ignores changes to scripts in its directory.

Selective build settings can significantly optimize the CI/CD pipeline by ensuring that services are only rebuilt when necessary,reducing build times and resource usage.