Skip to main content

Set up Contract Tests in Git

Overview​

This guide outlines how to set up, organize, and manage contract tests in Git. Contract testing helps ensure services work together correctly by validating that each service adheres to its expected contract. By the end of this guide, you'll have a complete setup where contract tests run automatically and post results directly to your pull requests. We'll cover:

  • Setting up a contract testing infrastructure
  • Organizing tests in Git
  • Automating test execution with CI/CD
  • Surfacing test results in pull requests
  • Best practices for test organization and management

Setup Requirements​

Before you begin:

  1. Install Signadot Operator v0.19+ in your cluster
  2. Enable Managed Runner Groups in your Signadot Dashboard
  3. Verify at least one runner pod is active in your cluster

Implementation Steps​

1. Enable Test Infrastructure​

Before writing tests, enable managed runner groups in your Signadot account:

  1. Go to Settings in your Signadot dashboard
  2. Enable "Managed Runner Groups"

This setting allows Signadot to automatically manage test runners in your cluster.

2. Create Test Structure in your Git Repository​

Set up your repository with this structure:

.
├── .signadot/
│   └── config.yaml          # Test discovery configuration
└── contract-tests/
    ├── frontend/           
    │   ├── .labels         # Frontend test labels
    │   └── api.star        # Frontend's API tests
    └── backend/
        ├── .labels         # Backend test labels
        └── api.star        # Backend's API tests

Example configuration files:

# .signadot/config.yaml
smart_tests:
- contract-tests/    # Root directory for test discovery
# contract-tests/frontend/.labels
service: frontend
type: consumer
team: frontend-team
criticality: high
owner: frontend@company.com

# contract-tests/backend/.labels
service: backend
type: provider
team: backend-team
criticality: high
owner: backend@company.com

Example test that you can run immediately:

# contract-tests/frontend/api.star
# Test external API contract (using httpbin.org as example)

def test_get_api():
    res = http.get(
        url="https://httpbin.org/get",
        capture=True,
        name="getExample"
    )

    ck = smart_test.check("api-contract")
    if res.status_code != 200:
        ck.error("expected 200 status code, got {}", res.status_code)

This structure enables:

  • Clear service ownership through labels
  • Easy test discovery via config
  • Consistent test organization
  • Simple dependency tracking

3. Run Tests​

During development, run tests from within your Git repository:

# Test against baseline
signadot smart-test run --cluster=staging

# Test against a sandbox and publish results
signadot smart-test run --sandbox=my-sandbox --publish

When run from within the Git repository, the CLI automatically discovers and runs all tests based on your .signadot/config.yaml and test labels. No additional arguments are needed to specify test files.

Use the --publish flag only in CI environments to avoid polluting sandbox results with local development runs.

For running individual test files outside the repository structure, see signadot smart-test run --help for additional options.

Viewing Test Results​

After running tests, you can view results in multiple ways:

  • CLI: Use signadot smart-test execution list and signadot smart-test execution get to see detailed results
  • UI Dashboard: View results under the Executions page or directly in the sandbox details if you used --publish
  • Pull Requests: As we'll see in the next sections, test results can be automatically posted to your PRs.

4. Automate with CI/CD​

Integrate contract testing into your CI/CD pipeline. Example GitHub Actions workflow:

...
jobs:
  contract-tests:
    runs-on: ubuntu-latest
    env:
      SIGNADOT_ORG: ${{ secrets.SIGNADOT_ORG }}
      SIGNADOT_API_KEY: ${{ secrets.SIGNADOT_API_KEY }}
      SANDBOX_NAME: pr-${{ github.event.number }}-test
    steps:
      # ... other steps     
      - name: Set up Signadot CLI
        run: |
          curl -sSLf https://raw.githubusercontent.com/signadot/cli/main/scripts/install.sh | sh

      - name: Create Sandbox
        run: |
          signadot sandbox apply \
            --set name=${SANDBOX_NAME} \
            --set github-pr=${{ github.event.number }} \
            --set image=myrepo/image:${{ github.sha }} \
            -f .signadot/sandbox-template.yaml

      - name: Run Contract Tests
        run: |
          signadot st run \
            --sandbox=${SANDBOX_NAME} \
            --set-label foo=bar \
            --publish

The --set-label argument allows you to add dynamic labels at runtime that complement the static labels defined in .labels files. These runtime labels are useful for:

  • Tracking test runs by environment
  • Adding CI-specific metadata
  • Grouping related test runs
  • Filtering test results

Labels set via --set-label are additive to the labels defined in .labels files, giving you both static service-level labels and dynamic execution-level labels.

5. Developer Feedback​

Test results can be integrated directly into your pull request workflow. If you're using GitHub, you can set up native integration to automatically post test results as PR comments:

Successful GitHub PR Comment

For other version control systems, you can use the sandbox API to fetch test results and integrate them into your existing PR workflow.

Infrastructure Best Practices​

Test Organization​

Contract tests can be organized in different ways, each with its own advantages. Many teams find value in combining these approaches based on their testing needs.

Centralized Repository Use a single repository with labels to categorize tests by service, team, or functionality. This approach:

  • Enables shared ownership and collaboration across teams
  • Makes it easier to maintain consistent testing practices
  • Facilitates cross-team review of contract changes
  • Works well for tests that span multiple services or teams

Per-Service Repository Keep tests alongside each microservice's code. This approach:

  • Maintains clear ownership by service teams
  • Keeps tests close to the service implementation
  • Simplifies versioning with service changes
  • Works well for service-specific contract validation

Many organizations benefit from using both strategies:

  • Centralized repo for cross-team contracts and shared infrastructure tests
  • Service repos for specific API validation and team-owned contracts

Label Management​

Set up comprehensive labels to enable future test selection capabilities:

# Example label structure
service: payment-service
type: provider          # provider/consumer
team: platform
criticality: high
owner: platform@company.com

Key label categories to consider:

  • Service identification and type
  • Team ownership and contacts
  • Test priority and criticality
  • Service dependencies
  • Test categories (e.g., API, events, data)

Conclusion​

A well-organized contract testing infrastructure helps teams catch integration issues early and maintain service compatibility. By following these patterns and practices, you can create a scalable and maintainable contract testing system that improves your overall development workflow.

To learn more: