Skip to content

Use a Pre-existing Database Instance

Introduction

By default, Coherence creates and manages database instances for your environments. However, there are scenarios where you might want to use a pre-existing database instance. This guide explains when and how to use self-managed databases with Coherence.

Default Database Management in Coherence

When you add a database service to an environment, Coherence typically:

  • Creates a new database instance for that environment
  • Manages VPC setup, including security groups and firewalls
  • Provides automatic environment variables (e.g., DATABASE_URL) to other services in the environment
  • Securely manages these variables using built-in cloud secrets support in your cloud account

Modifying Coherence-Managed Databases

You can modify Coherence-managed database instances in two ways.

  • Directly in your cloud provider:
    • Change configuration settings like instance size, backup settings, or high-availability options
    • Coherence won't treat these changes as drift or overwrite them
    • Use any tools you prefer (e.g., cloud provider web console or CLI)
  • By customizing the Terraform generated by cnc:
    • Allows for more complex modifications
    • Requires setting up custom templates in your app's settings
    • See this example for details on the process

Use Cases for Self-Managed Databases

There are scenarios where you might want to self-manage your database outside of Coherence:

  • Complex Terraform requirements
  • Configurations that don't fit the cnc architecture
  • Concerns about potential Coherence mistakes with critical database management

Some scenarios don't require special configuration:

  • Third-party DB SaaS (e.g., Planetscale, Neon, Supabase, CrunchyData, MongoDB, etc...)
    • Simply add connection info as a secret in Coherence variables
  • Database instances in the cloud with a public IP or hostname (Any provider)
    • No VPC configuration needed if access happens over a public network

For other cases, you'll need to configure Coherence for VPC configuration and secrets integration. Note that you can't directly use an existing instance from before using Coherence due to VPC creation needing to happen before the instance can be attached to it. Options include:

  • Moving the instance to the new VPC
  • Attaching it to both VPCs
  • Recreating it in the new VPC from a snapshot

These options may involve some application downtime. For more information, see:

Configuring Existing Databases

Provide the configuration (see below) you want to apply to the Coherence team, and they will set the info for your application. The instance can be specified at either the collection or environment scope.

  • If you choose collection, the instance will be shared by all environments for that service.
  • If you choose environment, the instance will only be used in that particular environment.

Configuration Options

The full set of options for an existing database can be found in the cnc code. Common options include:

# Name of the instance in the cloud
instance_name: str

# ID of the secret in secrets manager with the password
# AWS only, should be formatted like https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_secret_json_structure.html
secret_id: Optional[str] = None

# Should Coherence create databases for each environment?
manage_databases: Optional[bool] = True

# Username for database access
username: Optional[str] = None

# Logical database name to use
db_name: Optional[str] = None

# For AWS RDS: cluster mode or single instance?
cluster_mode: Optional[bool] = False

You must provide the database password. For AWS, you have 2 options.

  • Provide a secret_id in the options above, formatted correctly.
  • Provide the password directly like GCP, below.

For GCP, all you need to provide is a database password at either the collection or environment scope, as appropriate. The password should be set as the value of a secret in your cloud account, and the secret_id for that secret will be configured along with the existing instance information.

Conclusion

Using a pre-existing database instance with Coherence requires careful consideration of your specific needs and infrastructure setup. While Coherence offers robust database management by default, self-managing your database can provide additional flexibility for complex scenarios. Always ensure you understand the implications of your configuration choices, particularly regarding VPC setup and security.