Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.testdriver.ai/llms.txt

Use this file to discover all available pages before exploring further.

Auto-healing tests automatically fix themselves when your application changes. By integrating GitHub Copilot with TestDriver in your CI pipeline, you can have AI investigate test failures and propose fixes.

​
How It Works

When a test fails in CI:
  1. GitHub Actions detects the failure
  2. Copilot spawns the TestDriver agent with access to the MCP server
  3. The agent investigates by running the failing test and analyzing what changed
  4. Copilot creates a fix by updating the test code
  5. A pull request is opened with the proposed changes for review

​
Setting Up Auto-Healing

1

Add Repository Secrets

Add your TestDriver API key to your repository secrets:
  1. Go to Settings → Secrets and variables → Actions
  2. Click New repository secret
  3. Add TD_API_KEY with your API key value
2

Create the Auto-Heal Workflow

Add a GitHub Actions workflow that triggers on test failures:
.github/workflows/auto-heal.yml
name: Auto-Heal Tests

on:
  workflow_run:
    workflows: ["Tests"]  # Your main test workflow
    types: [completed]
    branches: [main]

jobs:
  auto-heal:
    if: ${{ github.event.workflow_run.conclusion == 'failure' }}
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm ci

      - name: Run failing tests and capture output
        id: tests
        continue-on-error: true
        run: |
          vitest run 2>&1 | tee test-output.txt
          echo "output<<EOF" >> $GITHUB_OUTPUT
          cat test-output.txt >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}

      - name: Invoke Copilot to fix tests
        uses: github/copilot-action@v1
        with:
          prompt: |
            @testdriver The following tests failed:

            ${{ steps.tests.outputs.output }}

            Please investigate each failure by:
            1. Running the failing test
            2. Analyzing what changed in the UI or behavior
            3. Updating the test code to fix the issue

            Create a commit with your changes.
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}

      - name: Create Pull Request
        uses: peter-evans/create-pull-request@v5
        with:
          title: "Auto-heal: Fix failing tests"
          body: |
            This PR was automatically generated by the auto-heal workflow.

            ## Changes
            The following tests were updated to fix failures detected in CI.

            ## Review
            Please review the changes carefully before merging.
          branch: auto-heal/${{ github.run_id }}
          commit-message: "fix: auto-heal failing tests"
3

Configure Test Workflow

Make sure your main test workflow has a name that matches the workflows trigger:
.github/workflows/tests.yml
name: Tests  # Must match the workflow_run trigger

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
      - run: npm ci
      - run: vitest run
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}

​
Example: Button Text Change

Here’s what auto-healing looks like when a button’s text changes:
1

Application Changes

A developer changes a button’s text from “Submit” to “Send”:
<!-- Before -->
<button>Submit</button>

<!-- After -->
<button>Send</button>
2

Test Fails

The test fails because it’s looking for the old text:
// This now fails
const submitButton = await testdriver.find("Submit button");
3

Auto-Heal Triggers

The auto-heal workflow runs, and Copilot investigates:
The test is looking for a "Submit button" but I see a "Send button"
on the page. The button functionality is the same, just the text changed.
I'll update the test to use the new text.
4

PR Created

A pull request is opened with the fix:
// Updated by auto-heal
const submitButton = await testdriver.find("Send button");

​
Configuration Options

​
Selective Auto-Healing

You can limit auto-healing to specific test files or patterns: 
- name: Run failing tests
  run: |
    # Only heal tests in the e2e directory
    vitest run tests/e2e/ 2>&1 | tee test-output.txt

​
Manual Approval

For safety, you can require manual approval before auto-heal runs: 
jobs:
  auto-heal:
    environment: auto-heal  # Requires approval
Configure the environment in Settings → Environments with required reviewers.

​
Limiting Changes

Add instructions to constrain what the AI can change: 
- name: Invoke Copilot to fix tests
  uses: github/copilot-action@v1
  with:
    prompt: |
      @testdriver Fix the failing tests.

      Rules:
      - Only update element selectors and text matching
      - Do not change test logic or assertions
      - Do not add or remove tests
      - Keep changes minimal

​
Best Practices

Auto-heal is a tool, not a replacement for human judgment. Review all changes before merging to ensure the test still validates what you intended.
Tests with clear, semantic descriptions are easier for the AI to heal:
// ✅ Good - describes purpose
await testdriver.find("primary call-to-action button in the hero section");

// ❌ Bad - too vague
await testdriver.find("button");
Configure GitHub notifications or Slack integration to be alerted when auto-heal PRs are created.
Monitor how often tests need healing. High heal rates may indicate:
  • Tests are too brittle
  • Application is changing rapidly
  • Element descriptions need improvement

​
Limitations

Auto-healing works best for:
  • Element text changes
  • Layout and styling updates
  • Minor UI restructuring
It may struggle with:
  • Major workflow changes
  • New features requiring new assertions
  • Complex multi-step interactions
For significant changes, create new tests manually using the TestDriver agent.