Skip to main content

Set up Tests using Jobs

Overview​

Maintaining service reliability during rapid development requires automated testing infrastructure that can validate changes early in the development cycle. Signadot's Jobs framework provides a managed test execution environment that integrates directly with your sandboxes.

This guide outlines how to set up and manage test infrastructure using Signadot Jobs.

Implementation Steps​

1. Set up Sandbox Infrastructure​

Before configuring test automation, ensure you have sandbox infrastructure in place following the Pull Request Sandboxes guide. The test infrastructure will integrate with these sandboxes.

2. Configure Job Runner Infrastructure​

Job Runner Groups manage the test execution environment. They provide:

  • Automated pod provisioning for test execution
  • Pre-warming for faster test startup
  • Resource isolation between test runs

Example configuration:

name: my-jrg
spec:
  cluster: "my-cluster"
  labels:
    env: "test-env"
  namespace: my-tests-ns      # dedicated namespace for test executors
  image: runner/image:latest  # base image for test execution
  jobTimeout: 30m            # execution time limit
  scaling:
    manual:
      desiredPods: 2         # concurrent test capacity

This specification creates a dedicated test execution environment named my-jrg. Configure it through the UI or CLI. See a complete example for Cypress tests here.

3. Define Test Execution Templates​

Create standardized job templates that define test execution parameters. These templates ensure consistent test execution across your environments:

spec:
  namePrefix: my-job
  runnerGroup: my-jrg        # links to runner infrastructure
  script: |
    #!/bin/bash
    set -e

    # Source code checkout
    echo "Clone repo"
    git clone --single-branch -b "@{branch}" \
      https://github.com/org/repo.git

    # Test execution
    cd tests/
    ./run-tests.sh
    
  routingContext:         
    sandbox: "@{sandbox}"    # enables sandbox targeting
  uploadArtifact:           # artifact collection
    - path: path/output.mp4

The template supports variables for dynamic configuration. Customize pod templates to manage secrets, credentials, and other resources needed during test execution.

Execute jobs using the template:

signadot job submit -f ./job-template.yaml --set branch=main --set sandbox=my-sandbox --attach

4. Configure Test Environment Variables​

Jobs automatically receive sandbox routing information through environment variables:

  • SIGNADOT_ROUTING_KEY: For request routing to sandboxes
  • SIGNADOT_SANDBOX_NAME: For sandbox identification

Use these variables in your test code to ensure proper sandbox targeting.

5. Integrate with CI Workflows​

Incorporate job execution into your CI pipeline. See the CI integration guide for implementation details.

Infrastructure Best Practices​

Resource Organization​

Store infrastructure configurations in version control:

  • Job Runner Groups: Define in central infrastructure repository or manage via UI
  • Job Templates: Store in source control, typically alongside other infrastructure code: 
    - signadot/
      ├── job-integration-routeservice-mocha.yaml
      └── job-e2e-routeservice-cypress.yaml

Runner Image Strategy​

Choose an approach that balances maintenance overhead with execution speed:

  1. Generic Base Image

    • Use standard images (e.g., ubuntu)
    • Install dependencies during execution
    • Maximizes flexibility, slower startup
    • Ideal for diverse test requirements

  2. Optimized Custom Images

    • Pre-bake test dependencies
    • Faster test startup
    • Requires image maintenance
    • Best for standardized test suites

Start with generic images and optimize with custom images as testing patterns stabilize.

Conclusion​

A well-configured test infrastructure using Signadot Jobs enables reliable, automated testing of sandbox environments. This approach provides:

  • Consistent test execution environments
  • Automated resource management
  • Integration with existing CI workflows
  • Scalable test automation

Focus on establishing standardized patterns that teams can easily adopt and maintain.