Run Integration/E2E Tests

Running integration or end-to-end (e2e) tests against your deployed environments is a crucial step in ensuring the quality and reliability of your application. This guide will show you how to set up and run integration tests on Coherence environments using CI/CD pipelines, with a focus on GitHub Actions.

Overview

The general process for running integration tests on Coherence environments involves:

1. Deploying your application to a Coherence environment
2. Triggering a test workflow when the deployment is successful. Coherence will post a check run against each commit that it deploys for a service. The check run will be called Coherence deploy for SERVICE_NAME
3. Running your integration tests against the deployed environment
4. Reporting the test results

While we'll use GitHub Actions in our example, you can apply similar principles to other CI/CD providers or test runners.

GitHub Actions Example

Here's an example of a GitHub Actions workflow that triggers integration tests after a successful deployment to a Coherence service called web in my environment:

name: Staging Integration Tests

on:
  check_run:
    types:
      - completed

jobs:
  e2e_testing:
    if: ${{ github.event.check_run.name == 'Coherence deploy for web' }}
    uses: ./.github/workflows/run-integration-tests.yml
    secrets: inherit
    with:
      test_search_query: MY_SEARCH_QUERY (e.g. ${{ startsWith(github.ref, 'refs/tags/release') && 'tag:production' || 'tag:staging' }})

Let's break down this workflow:

1. The workflow is triggered on the check_run event, specifically when a check run is completed.
2. It checks if the completed check run is named 'Coherence deploy for web', which corresponds to a successful deployment in Coherence.
3. If the condition is met, it runs a reusable workflow defined in .github/workflows/run-integration-tests.yml.
4. The test_search_query input is set based on whether the trigger is a release tag or not, allowing you to run different tests for production and staging environments.

Setting Up Your Test Workflow

Your run-integration-tests.yml file should contain the actual test execution steps. Here's an example of what it might look like:

name: Run E2E Tests

on:
  workflow_call:
    inputs:
      test_search_query:
        required: true
        type: string

jobs:
  run_tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '14'
      - name: Install dependencies
        run: npm ci
      - name: Run E2E tests
        run: npm run test:e2e
        env:
          TEST_URL: ${{ secrets.COHERENCE_ENVIRONMENT_URL }}
          TEST_SEARCH_QUERY: ${{ inputs.test_search_query }}

This workflow:

1. Sets up a Node.js environment
2. Installs dependencies
3. Runs your E2E tests, passing in the Coherence environment URL and the test search query

Coherence Integration

To make this work with Coherence:

1. Ensure your Coherence deployments are set up to create GitHub check runs. This is typically done automatically when you connect your GitHub repository to Coherence.

2. Store your Coherence environment URL as a secret in your GitHub repository settings. Name it COHERENCE_ENVIRONMENT_URL.

3. In your test code, use the TEST_URL environment variable to know which URL to test against.

Best Practices

1. Parameterize your tests: Use environment variables or configuration files to make your tests flexible enough to run against different environments.

2. Use test tagging: Implement a tagging system in your tests to easily run subsets of tests for different environments or scenarios.

3. Handle authentication: If your app requires authentication, consider setting up test users or implementing a way to generate authentication tokens for your tests.

4. Clean up test data: Ensure your tests clean up any data they create to prevent test pollution in shared environments.

5. Parallel testing: For large test suites, consider running tests in parallel to speed up the process.

Conclusion

By integrating your integration tests with your Coherence deployments, you can ensure that every successful deployment is immediately verified. This approach helps catch issues early and provides confidence in the quality of your deployments.

Remember to adjust the example workflows and code to fit your specific project structure and testing framework.