AI Test Generation

TestDriver is designed from the ground up to be used by AI agents, enabling autonomous test creation, execution, and debugging.

AI Agent Support

Complete guide for AI agents with autonomous operation patterns:

agents.md - AI Agent Guide

Comprehensive documentation specifically for AI agents, including:

Quick setup and authentication
Provision methods (chrome, vscode, electron)
Core API methods with examples
Sandbox reconnection patterns
Debugging failed finds
Iterative selector refinement
Best practices for autonomous operation

// AI agents can autonomously write and run tests
import { test } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('ai-generated test', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://example.com'
  });

  await testdriver.find('More information link').click();
  await testdriver.assert('IANA page is visible');
});

Natural Language Everything

AI-powered element location, assertions, and instructions:

Element Location
Assertions
Instructions

// Describe elements naturally
await testdriver.find('submit button');
await testdriver.find('email input in the login form');
await testdriver.find('delete button in the top right corner');
await testdriver.find('first product card in the grid');

// Assert application state naturally
await testdriver.assert('the login page is displayed');
await testdriver.assert('success message says "Welcome back"');
await testdriver.assert('the form is empty');
await testdriver.assert('the button is blue and enabled');
await testdriver.assert('loading spinner is not visible');

// Complex interactions described naturally
await testdriver.find('Products menu').hover();
await testdriver.find('Laptops submenu item in the Products dropdown').click();
await testdriver.find('Sort by dropdown').click();
await testdriver.find('Price: Low to High option').click();

Test Generation with Claude Code

Feed natural language requirements directly to Claude Code to generate complete test suites:

Ask Claude Code to Generate Tests

Provide the requirements to Claude Code with context about your testing setup:

Generate a TestDriver test using Vitest for these requirements:

Test the login flow:
1. Navigate to login page
2. Enter valid credentials
3. Click submit
4. Verify dashboard appears

Use the chrome preset and target https://myapp.com/login

Claude Generates Complete Tests

Claude Code will create production-ready test files:

import { test } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('user can login', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://myapp.com/login'
  });

  await testdriver.find('email input').type('[email protected]');
  await testdriver.find('password input').type('password', { secret: true });
  await testdriver.find('Login button').click();

  await testdriver.assert('Dashboard is visible');
});

Run Tests and Iterate

Ask Claude Code to run the tests and refine based on results:

Run these tests and fix any failures. Then add tests for:
- Invalid credentials
- Empty form submission
- Password visibility toggle

Claude will execute the tests, analyze failures, and automatically refine the test code.

Claude Code understands TestDriver’s natural language API and will automatically use appropriate selectors, assertions, and best practices like secret: true for passwords.

Sandbox Reconnection

AI agents can reconnect to sandboxes for debugging:

// Initial test attempt
test('initial attempt', async () => {
  const testdriver = new TestDriver({
    apiKey: process.env.TD_API_KEY
  });

  const instance = await testdriver.connect();
  console.log('Sandbox ID:', instance.instanceId);
  // Save this ID for reconnection

  try {
    await testdriver.find('difficult selector').click();
  } catch (error) {
    console.error('Failed:', error);
    // Keep sandbox alive for debugging
  }
});

// Reconnect to same sandbox
test('reconnect attempt', async () => {
  const testdriver = new TestDriver({
    apiKey: process.env.TD_API_KEY
  });

  // Manually specify the sandbox ID from previous test
  await testdriver.connect({
    sandboxId: 'your-sandbox-id-here',
    newSandbox: false
  });

  // Try alternative selector
  await testdriver.find('alternative selector').click();

  await testdriver.disconnect();
});

Automatic Sandbox Tracking

TestDriver automatically saves the last sandbox ID to ~/.testdriverai-last-sandbox. By default, newSandbox: true, so each connection creates a new sandbox. Set newSandbox: false and provide a sandboxId to reconnect to an existing sandbox.

Error Analysis

Rich error information for AI agents to analyze and adapt:

try {
  await testdriver.find('non-existent button').click();
} catch (error) {
  // Rich error information for AI analysis
  console.log('Error type:', error.name); // 'ElementNotFoundError'
  console.log('Message:', error.message);
  console.log('Similarity score:', error.similarity); // 0.72
  console.log('Confidence:', error.confidence); // 0.81
  console.log('Debug screenshot:', error.debugScreenshot); // Base64
  console.log('Cache info:', error.cacheInfo);
  console.log('Attempted prompt:', error.prompt);

  // AI can analyze and adapt
  if (error.similarity > 0.7) {
    // Close match, try more specific selector
    await testdriver.find('blue submit button in bottom right').click();
  } else {
    // No close match, try completely different approach
    await testdriver.find('primary action button').click();
  }
}

MCP Integration (Coming Soon)

Query test infrastructure using natural language:

// Future MCP interactivity
await testdriver.mcp.query("What is the CPU usage of my app?");
await testdriver.mcp.query("Show me memory leaks in the last test run");
await testdriver.mcp.query("What tests are flakiest this week?");
await testdriver.mcp.query("Compare performance between branches");

MCP (Model Context Protocol) integration is planned for Q1 2025.

AI-Powered Debugging

TestDriver provides intelligent debugging assistance:

Similarity Scoring

// AI analyzes how close the match was
try {
  await testdriver.find('submit button');
} catch (error) {
  if (error.similarity > 0.8) {
    console.log('Very close match - likely wrong selector specificity');
  } else if (error.similarity > 0.5) {
    console.log('Partial match - element might be present but different');
  } else {
    console.log('No match - element likely not on screen');
  }
}

Visual Debugging

// Get debug screenshot for AI analysis
try {
  await testdriver.find('button');
} catch (error) {
  // AI can analyze the debug screenshot
  const screenshot = error.debugScreenshot; // Base64 image
  // Send to vision model for analysis
  const analysis = await visionModel.analyze(screenshot,
    "Why can't we find the submit button?"
  );
}

Cache Diagnostics

// Understand cache behavior
try {
  await testdriver.find('element');
} catch (error) {
  console.log('Cache info:', error.cacheInfo);
  // {
  //   cacheKey: 'abc123',
  //   cacheHit: false,
  //   similarEntries: 3,
  //   bestSimilarity: 0.87,
  //   reason: 'threshold_not_met'
  // }
}

Programmatic Control

AI agents have full programmatic control:

// Dynamic test generation
const testCases = [
  { email: '[email protected]', password: 'pass1', shouldSucceed: true },
  { email: 'invalid', password: 'pass2', shouldSucceed: false },
  { email: '[email protected]', password: 'wrong', shouldSucceed: false },
];

for (const testCase of testCases) {
  test(`login with ${testCase.email}`, async (context) => {
    const { testdriver } = await chrome(context, { url });

    await testdriver.find('email input').type(testCase.email);
    await testdriver.find('password input').type(testCase.password, { secret: true });
    await testdriver.find('submit').click();

    if (testCase.shouldSucceed) {
      await testdriver.assert('dashboard is visible');
    } else {
      await testdriver.assert('error message is displayed');
    }
  });
}

Learning & Adaptation

TestDriver learns from test executions:

AI Understanding of Your App

The cache system builds a visual understanding over time:

Element location patterns
UI component relationships
Common interaction flows
Stability characteristics
Performance baselines

This enables:

Faster subsequent executions
Better error messages
Intelligent suggestions
Anomaly detection

Best Practices for AI Agents

1. Use Descriptive Selectors

// ✅ Good - specific and contextual
await testdriver.find('blue submit button below the login form');

// ❌ Bad - too vague
await testdriver.find('button');

2. Always Use secret: true for Passwords

// ✅ Protected from logging
await testdriver.type('MyPassword123', { secret: true });

// ❌ Password will be logged
await testdriver.type('MyPassword123');

3. Handle Errors Gracefully

// ✅ Try alternative approaches
try {
  await testdriver.find('optional element').click();
} catch (error) {
  console.log('Element not found, trying alternative...');
  await testdriver.find('alternative element').click();
}

4. Save Sandbox IDs

// ✅ Enable reconnection
const instance = await testdriver.connect();
console.log('Sandbox ID:', instance.instanceId);
// Save for debugging later

Learn More

AI Agent Guide

Complete guide for AI agents

API Reference

Full API documentation

Error Handling

Handle errors effectively

Debugging Guide

Debug failing tests

Getting Started

Examples

Guides

Reference

AI Agent Support

agents.md - AI Agent Guide

Natural Language Everything

Test Generation with Claude Code

Sandbox Reconnection

Automatic Sandbox Tracking

Error Analysis

MCP Integration (Coming Soon)

AI-Powered Debugging

Programmatic Control

Learning & Adaptation

AI Understanding of Your App

Best Practices for AI Agents

Learn More

AI Agent Guide

API Reference

Error Handling

Debugging Guide

Getting Started

Examples

Guides

Reference

​AI Agent Support

agents.md - AI Agent Guide

​Natural Language Everything

​Test Generation with Claude Code

​Sandbox Reconnection

Automatic Sandbox Tracking

​Error Analysis

​MCP Integration (Coming Soon)

​AI-Powered Debugging

​Programmatic Control

​Learning & Adaptation

AI Understanding of Your App

​Best Practices for AI Agents

​Learn More

AI Agent Guide

API Reference

Error Handling

Debugging Guide

AI Agent Support

Natural Language Everything

Test Generation with Claude Code

Sandbox Reconnection

Error Analysis

MCP Integration (Coming Soon)

AI-Powered Debugging

Programmatic Control

Learning & Adaptation

Best Practices for AI Agents

Learn More