Playwright

Overview

This document guides you through the steps required to run Playwright tests using Signadot's Continuous Testing (CT) feature.

Running Tests as a Signadot Job

Running a test as a Signadot Job provides context awareness, meaning that if the sandbox referenced in the test is terminated, closed, or in any other failed state, the test will automatically fail. This makes the tests more reliable.

Create Job Runner Group

To start with Signadot CT, we need to create a Job Runner Group

your_project/.signadot/testing/playwright-runner.yaml

name: playwright
spec:
  cluster: "@{cluster}"
  labels:
    env: "@{env}"
  namespace: <namespace where run tests>
  jobTimeout: 30m
  image: mcr.microsoft.com/playwright:v1.45.1-jammy
  scaling:
    manual:
      desiredPods: 1

Create Job Specification

your_project/.signadot/testing/e2e-demo-playwright.yaml

spec:
  namePrefix: playwright-e2e
  runnerGroup: playwright
  script: |
    #!/bin/bash
    set -e
    
    # Clone the git repo
    echo "Cloning repo"
    git clone --single-branch -b "@{branch}" \
      https://github.com/<org>/<repo>.git

    # Run all playwright tests
    cd <repo>
    
    export CI=true
    
    npm ci
    set +e
    npm run e2e:playwright # Change with your corresponding npm command or npx playwright test
    E2E_EXIT_CODE=$?
    set -e

    # Compress the playwright-report directory so that it can be uploaded as a single artifact.
    tar czf playwright-report.tar.gz playwright-report

    exit $E2E_EXIT_CODE
  routingContext:
    sandbox: "@{sandbox}"
  uploadArtifact:
    - path: <repo>/playwright-report.tar.gz

There a few things to note in that code snippet

1. Exit code handling

    set +e
    npm run e2e:playwright # Change with your corresponding npm command or npx playwright test
    E2E_EXIT_CODE=$?
    set -e
    
    ... more script
    
    exit $E2E_EXIT_CODE

If you want to make sure the job doesn't terminate when the tests fail, we are saving the exit code and then returning it at the very end.

2. Report

As of now, directory upload is not supported. So we are compressing the playwright-report directory so that all the artifacts are available as a single file.

Set Up Routing Context

There are some use cases where we need to run tests against specific Sandboxes, so we need to rely on some routing context. We could use the preview url, but there are some situations where it would be better to use simply the routing key. For the Playwright integration, we have two options:

Using Playwright config

We can use the Playwright config, and set the extraHTTPHeaders in the browsers or test settings.

For example:

extraHTTPHeaders: {
    baggage: `sd-routing-key=${process.env.SIGNADOT_ROUTING_KEY}`
}

The resulting config will be something similar to:

your_project/plawyright.config.ts

import { defineConfig, devices } from "@playwright/test";

const BASE_URL = 'https://yourwebsite.com';

export default defineConfig({
    testDir: "./playwright-tests",
    fullyParallel: true,
    forbidOnly: !!process.env.CI,
    retries: process.env.CI ? 0 : 0,
    workers: process.env.CI ? 1 : undefined,
    reporter: [["html", { open: "never" }]],
    use: {
        baseURL: BASE_URL,
        trace: "on",
        video: "on",
        extraHTTPHeaders: {
            baggage: `sd-routing-key=${process.env.SIGNADOT_ROUTING_KEY}`,
        },
    },
    projects: [
        {
            name: "chromium",
            use: {
                ...devices["Desktop Chrome"],
                baseURL: BASE_URL,
            },
        },
    ],
});

Using Playwright route

In others cases we may need to apply the Routing Key to specific routes, we can do this by using page.route. Which allows to mutate the request before is executed.

For example

your_project/tests/basePage.ts

await page.route(
    (url: URL) => {
        // Only add the routing key if the url contains mydomain.com
        return url.href.includes("mydomain.com");
    },
    async (route, request) => {
        const headers = await request.allHeaders();

        // You can also use previous headers.baggage if need to
        await route.continue({
            headers: {
                ...headers,
                baggage: `sd-routing-key=${routingKey}`,
            },
        });
    }
);