Skip to content

GCP GKE (kubernetes)

Introduction

Coherence, based on the open-source cnc framework, offers you choices in the architecture your app will use to deploy to GCP. Each architecture has a different goal in terms of trade-offs between cost, compliance, scale, and complexity. Unlike a "black box" platform, usingcnc you're able to customize these deployments further based on your needs, giving you total control.

The gke flavor is designed to be the most robust and flexible deployment possible. It has best practices baked in throughout for security and reliablity. The lower bound for expected cost per collection for this deployment is approx $150 a month in GCP costs.

The gke flavor supports internal microservices that are not attached to a load balancer, using is_internal in the x-cnc yml for that service.

Resources Used

Networking

  • A unique VPC is configured for each application. Multiple services in one application share a VPC.
    • All workloads are configured to use only Private IPs by default, and all egress to the internet is via Cloud NAT Gateway
  • Regional Load Balancing is used to route traffic to the right version and k8s service in your GKE cluster
    • Internal services with no load balancer access are also supported
  • Managed SSL Certificates are used to secure connections to the application with no maintainence required.
    • For custom domains, all you need to do tell us the domain and point an A record in your DNS provider to the IP for your application (shown in the Coherence UI).
    • GCP support for custom domains with wildcard subdomains requires you to provide your own SSL certificate at this time, and upload it to the SSL Certificates interface in the GCP console

Build & Deploy

  • Coherence spins up a Google Cloud Storage bucket for is created public assets fronted by CDN for each app, as a place to put non-repo resources (e.g. fonts, PDFs, images). Eventually you may want to augment this with a CMS like Contentful.
  • Cloud Build is used to build and deploy all the configured services. Build steps are parallelized as much as possible. Up to 10 concurrent test commands can be provided to coherence.yml and will be executed in parallel.

Data Storage

Each environment in Coherence gets a distinct Cloud SQL instance. You also have the option to provide the name of an existing Cloud SQL instance and we will configure your app to use that instance. Supported resources include:

  • Cloud SQL is used to provide database resources. Backups & high availability are not configured by default and the instance type is a micro - changing these in the UI or with the CLI will not be treated as drift.
  • Memorystore is used to provde hosted redis instances. Similar to Cloud SQL, backups and availability are confiigured for effeciency by default but changes to these configurations will not be overwriten by Coherence.
  • Google Cloud Storage resources for object storage are also supported, the bucket information will be provided via environment varialbes to your application.

Container Orchestration

  • Google Kubernetes Engine for both frontend and backend.
    • A k8s namespace is created for each environment
    • A k8s service is created in each environment for each frontend or backend service
    • A managed zonal network endpoint group is created in gcp for each cnc service in each environment, with the corresponding load balancer configuration. Internal services skip this configuration.
  • Google Kubernetes Engine is used for:
    • running async workers (like a celery/sidekiq queue worker)
    • running scheduled tasks (cron tasks) if those are configured.
    • the GKE control plane incurs a monthly cost in additon to deployed workloads.

Observability & Monitoring

  • GCP Operations Suite (formerly Stackdriver) is configured to collect logs and metrics. You can adjust log filters and retention rules in the GCP console to control the cost of log ingestion and storage, if required as you scale.

Security & Configuration

  • IAM Role and Policy are used for providing permissions to the services. Distinct identies are provisioned with minimum permissions for different app components, such as building, deploying, and executing the application.
  • Secrets Manager is used to store Environment Variables.
  • Workload Identity as well as dedicated Cloud Build Service Account integrations are used to assume app service accounts as needed without ever touching a key file. The GKE workloads will use the identity federation features provided by GCP for cloud access.
  • Compute Engine is used to provide a bastion host so that external connectivity to resources in the VPC is possible, from either your local network or Coherence's toolboxes.

Default Labels

For convenience and auditability, Coherence adds default labels to all cloud resources where applicable: 

    default_labels = {
        environment = "YOUR-COLLECTION-NAME"
        managed_by = "coherence"
        application = "your-application-name"
    }

Cloud Run vs GKE

  • Google has a comparison between Cloud Run and GKE. One important difference is Cloud Run autoscaling, which can scale to zero and use no resources if there are no requests. Scaling in GKE is controlled via replicas, which you can control with the min_scale attribute in the service definition of your coherence.yml