Skip to content

AWS Elastic Beanstalk

This guide shows you how to easily migrate your Django application from AWS Elastic Beanstalk to AWS Elastic Container Service (ECS) using Coherence.

Coherence provides a streamlined path for migrating your Django application from AWS Elastic Beanstalk to Amazon Elastic Container Service (ECS). This transition offers enhanced control, scalability, and access to AWS's extensive services while maintaining deployment simplicity. Key benefits include:

  • Advanced AWS ECS deployment options with high customizability
  • Comprehensive developer portal with self-service functionality
  • Powerful CI/CD capabilities with GitHub integration
  • Multi-environment management across different cloud accounts
  • Cost optimization through efficient resource allocation
  • Full utilization of AWS cloud credits, beneficial for startups

In this guide, we will:

  1. Prepare the code from an example Django application deployed on AWS Elastic Beanstalk for deployment on Coherence with AWS ECS
  2. Deploy the application on Coherence
  3. Migrate the data from your AWS database to Coherence
  4. Transfer your domain from AWS to Coherence

Prerequisites

To follow this guide, you'll need:

1. Prepare your Django application for migration

Coherence gives you flexible options for building and deploying your application.

If you don't have a Dockerfile, Coherence will automatically use Nixpacks to build your container. This means you don't need to make any changes to your application structure.

Option B: Use an existing Dockerfile

If you've been using a Dockerfile with Elastic Beanstalk, Coherence will use that Dockerfile to deploy your application. No additional preparation is needed.

Option C: Write a Dockerfile (if necessary)

If your application doesn't have a Dockerfile and you prefer not to use Nixpacks, you'll need to write a Dockerfile for your application.

Here is an example Dockerfile for a Django application:

FROM python:3.12

WORKDIR /app

COPY requirements.txt /app/
RUN pip install --upgrade pip
RUN pip install -r requirements.txt

COPY . /app/

Update Django settings for Coherence

Now update your Django settings and remove AWS Elastic Beanstalk-specific configurations to ensure your app works correctly in the Coherence environment.

  1. Remove AWS Elastic Beanstalk-specific code:

    If you have any AWS Elastic Beanstalk-specific code or configurations in your project, remove them. This might include:

    • Elastic Beanstalk-specific environment variable references
    • Elastic Beanstalk-specific files like .ebextensions directory (if you have one)
    • Any custom scripts used for Elastic Beanstalk deployment

  2. In your Django project's settings.py file, remove the following code:

    ALLOWED_HOSTS = [
    'beanstalk-app-env.abc123.us-west-2.elasticbeanstalk.com'
    ]

    Replace it with the following Coherence-specific settings:

    ALLOWED_HOSTS = [
    os.getenv('CNC_ENVIRONMENT_DOMAIN')
    ]

    These changes set up the CNC_ENVIRONMENT_DOMAIN environment variable. This represents the domain the app will have access to once it's running in Coherence.

  3. Update your database configuration to use the dj_database_url package. Coherence uses this package to configure database connections. Ensure your settings.py file includes the following database configuration:

    import dj_database_url

DATABASES = {
    'default': dj_database_url.config(
        default=os.environ.get('DATABASE_URL'),
        conn_max_age=600
    )
}

    This configuration uses the DATABASE_URL environment variable, which Coherence will provide.

  4. Update requirements.txt and add the following packages to ensure all necessary dependencies are installed:

    psycopg2-binary==2.9.9

dj-database-url==2.1.0

gunicorn==20.1.0

    These packages are essential for database connectivity (psycopg2-binary and dj-database-url) and serving your application (gunicorn).

2. Deploy the application on Coherence

Now that your code is prepared, set up the application on Coherence.

Create a new application

  1. In the Coherence dashboard, click New application
  2. Give the application a name
  3. Select Amazon AWS as the provider and ECS as the architecture
  4. Click Continue

Create a collection

Collections in Coherence allow you to organize environments.

  1. On the application's landing page, click the Create collection button
  2. Give the collection a name (for example, "Production")
  3. Click Create Collection

Configure an environment

  1. In the collection, click New Environment
  2. Give the environment a name
  3. Select the appropriate environment type (Staging is recommended for initial testing)
  4. Click Create

Add Coherence services

Create a backend service

  1. Click the environment name to enter the environment homepage
  2. Go to the Services tab
  3. Click New service and select Django

  4. Configure the service:

    • Name: Choose a name for the service
    • Command: gunicorn mysite.wsgi (replace mysite with your Django project name)
    • Container Source: Choose Repository

  5. Connect your GitHub repository

  6. Click Create Service

Create a Postgres database service

  1. In the Services tab, click New service and select Postgres

  2. Configure the database:

    • Name: Choose a name (for example, django-db)
    • Engine: PostgreSQL
    • Version: Choose the appropriate version (for example, 14)

  3. Click Create Service.

Connect to AWS

Follow the steps in our AWS guide to connect Coherence to your cloud account.

Set up environment variables

After connecting to AWS, set up the necessary environment variables for the Django application:

  1. In your Coherence dashboard, navigate to the environment
  2. Go to the Variables tab
  3. Add the following environment variables:
  4. SECRET_KEY (if applicable): Set this to the string used for enhanced security in your Django project
  5. Any other environment-specific variables your application needs

Note: Coherence automatically provides some environment variables like DATABASE_URL. You don't need to set these manually.

Set up the migration command

Next, configure the cnc.yml file Coherence created for the app to run a migration command when the application is built.

Navigate to the cnc.yml tab in the environment and edit this section:

    x-cnc:
      type: backend
      system:
        health_check: ''
      migrate: python manage.py migrate // add this line
      url_path: ''

Start a provisioning task

Now submit a provisioning task so that Coherence can automatically set up and manage the necessary cloud resources for your collection.

  • Select the Provision Tasks tab for the collection
  • Click the Submit button to submit a provisioning task
  • Click the Submit new task button

Coherence will automatically provision infrastructure for your project based on your service requirements. Wait for this step to complete before moving on. It should take several minutes.

Start a build and deploy pipeline

  1. Go to the Deployments tab for the environment
  2. Click New deployment
  3. Select the branch and commit to build from
  4. Click Submit deployment to start the build and deploy process

View the deployed application

Once the build is complete:

  1. In your Coherence dashboard, navigate to your environment
  2. Look for the Domains section and find the provided URL
  3. Click the URL to open your deployed application in a new browser tab

Coherence app deployed

3. Migrate the data

This is a crucial step in the migration process. Don't hesitate to contact the Coherence team if you need assistance.

Export data from AWS Elastic Beanstalk

To export your data from your Elastic Beanstalk environment, follow these steps:

  1. Use the Elastic Beanstalk CLI to connect to your environment:

    eb ssh <your-environment-name>

    Replace with the name of your Elastic Beanstalk environment.

  2. Once connected, activate the virtual environment:

    source /var/app/venv/*/bin/activate

  3. Navigate to your application directory:

    cd /var/app/current

  4. Use Django's dumpdata management command to export your data:

    python manage.py dumpdata --natural-foreign --natural-primary -e contenttypes -e auth.Permission --indent 2 > /tmp/db_dump.json

    This command exports all data except for content types and permissions, which are typically not needed for migration.

  5. While in your SSH session, use the AWS CLI to copy the dump file to an S3 bucket:

    aws s3 cp /tmp/db_dump.json s3://<your-bucket-name>/

    Replace with the name of your S3 bucket.

  6. Exit the SSH session:

    exit

  7. On your local machine, download the dump file from S3:

    aws s3 cp s3://<your-bucket-name>/db_dump.json ./

  8. Now covert this SQLite JSON to PostgreSQL commands that the new database will recognize. In the folder containing the db_dump.json file, run the Python script below:

import json

def json_to_sql(json_file, output_file):
    with open(json_file, 'r') as f:
        data = json.load(f)

    with open(output_file, 'w') as f:
        for item in data:
            model = item['model']
            fields = item['fields']

            # Remove 'groups' and 'user_permissions' from auth_user only if they're empty
            if model == 'auth.user':
                if 'groups' in fields and not fields['groups']:
                    del fields['groups']
                if 'user_permissions' in fields and not fields['user_permissions']:
                    del fields['user_permissions']

            columns = ', '.join(fields.keys())
            values = []
            for v in fields.values():
                if v is None:
                    values.append('NULL')
                elif isinstance(v, (int, float)):
                    values.append(str(v))
                elif isinstance(v, bool):
                    values.append(str(v).upper())
                elif isinstance(v, list):
                    values.append(f"'{json.dumps(v)}'")
                else:
                    values.append(f"'{str(v).replace("'", "''")}'")

            values_str = ', '.join(values)

            if 'pk' in item:
                pk = item['pk']
                f.write(f"INSERT INTO {model.replace('.', '_')} (id, {columns}) VALUES ({pk}, {values_str});\n")
            else:
                f.write(f"INSERT INTO {model.replace('.', '_')} ({columns}) VALUES ({values_str});\n")

json_to_sql('db_dump.json', 'db_dump.sql')

You can use this JSON dump of your database to migrate your data to Coherence.

Choose a migration method

You can migrate your data to Coherence using one of two methods:

  • Using the Coherence cocli tool to interact with the Coherence API.
  • Accessing the AWS database locally.

The Coherence CLI tool helps you set up the tools you need to manage your applications locally.

Set up a toolbox

To interact with your Coherence environment from your local machine, first set up a toolbox:

  1. Install cocnc by following the instructions in the CNC documentation
  2. Install cocli by following the instructions in the cocli GitHub repository
  3. Authenticate the AWS CLI with this command:
aws configure

Start a toolbox

Now set up authentication.

To create an access token on Coherence, navigate to your profile and click Create Token in the API section.

Run the following command on your local machine with your access token:

export COHERENCE_ACCESS_TOKEN="your-access-token-here"

To start a toolbox, you need your app ID and collection ID. In your terminal, set your app ID and collection ID to variables with the following command, adding your app and collection names:

app_id=$(cocli apps list | jq '.[] | select(.title=="Your App Name") | .id')

collection_id=$(cocli collections list -a $app_id | jq '.data[] | select(.name=="your-collection-name") | .id')

Now start a toolbox by running the following command, replacing main with the name of your environment on Coherence:

cocli cnc -c $collection_id -- toolbox start main --proxy-only

Keep the toolbox running while you migrate the data in a new terminal.

Migrate the data to the Coherence project

Now import your data into the new Coherence-managed database:

  1. Open a new terminal in the folder containing the aws_db_dump.sql file
  2. Get the psql database connection string from the output of the toolbox. You'll find it in the list of variables, under DATABASE_URL.
  3. Run the following command:
psql -d <database-connection-string> < aws_db_dump.sql

Make sure you replace <database-connection-string> with your database connection string and path/to/aws_db_dump.sql with the path to your migration file.

To interact with your AWS database from your local machine, first authenticate with the AWS CLI:

  1. Authenticate the AWS CLI with this command:

    aws configure

  2. Run the following command to connect with your AWS database:

    aws ssm start-session --target INSTANCE_ID --document-name AWS-StartPortForwardingSessionToRemoteHost --parameters '{"host":["RDS_PROXY_ENDPOINT"],"portNumber":["5432"], "localPortNumber":["5442"]}'
    • Replace INSTANCE_ID with your own. Find it in the EC2 console for the bastion Coherence made.
    • Replace RDS_PROXY_ENDPOINT with the proxy endpoint found in the RDS console in your AWS Management console.

  3. Once connected, run this command in a separate terminal in the folder containing the db_dump.sql file`:

    psql -h localhost -p 5442 -U user_name -d database_name < db_dump.sql

    You can find the values for the user_name, database_name, and the database password in the Variables tab inside your Coherence environment.

Verify the data migration

Check the application to verify that your data has been successfully migrated.

Data successfully migrated to Coherence app

4. Migrate the domain

The final step is to update your domain settings to point to the new Coherence-hosted application.

Remove the domain record from AWS Route 53

  1. Log in to the AWS Management console
  2. Navigate to the Route 53 service
  3. In the left navigation pane, click on Hosted zones
  4. Find and click on the hosted zone for your domain
  5. Locate the record set for your Elastic Beanstalk environment. This is typically an A record or CNAME record pointing to your Elastic Beanstalk URL.
  6. Select the record and click Delete record
  7. Confirm the deletion when prompted

Change DNS settings for the domain

  1. Log in to your domain registrar's dashboard
  2. Update the DNS records to point to Coherence instead of AWS Elastic Beanstalk. You'll need to add a CNAME record pointing to the Coherence-provided domain.

Add the domain to Coherence

  1. In the Coherence environment, go to the Domains tab
  2. Add your custom domain and click Add Domain
  3. Wait for propagation (up to 48 hours)

Once DNS propagation is complete, visit your domain to verify that the website is accessible and functioning correctly with your migrated data.

Tear down the AWS Elastic Beanstalk application

After successfully migrating to Coherence, follow these steps to remove your Elastic Beanstalk environment:

  1. Use the EB CLI to terminate your environment:
eb terminate <environment-name>

  1. Delete the Elastic Beanstalk application via the AWS Management console:

    • Navigate to Elastic Beanstalk > Applications
    • Select your application
    • Click Actions > Delete application

  2. Check for and manually delete any remaining resources.

Conclusion

Congratulations! You've successfully migrated your Django application from AWS Elastic Beanstalk to Coherence with AWS ECS. Your application is now running in a more flexible and scalable environment, with the added benefits of Coherence's powerful development tools.

If you encounter any issues or have questions during the migration process, don't hesitate to reach out to the Coherence support team. We're here to ensure your migration is smooth and successful.