Running & Debugging Tests

Learn how to run TestDriver tests efficiently and debug failures using video replays, logs, and error analysis.

Running Tests

TestDriver works with Vitest’s powerful test runner:

Run All Tests
Run Specific Tests
Watch Mode
Parallel Execution

# Run all tests once
npx vitest run

# Run in watch mode
npx vitest

# Run with coverage
npx vitest run --coverage

Test Output

Understanding test output:

$ npx vitest run

 ✓ login.test.js (3) 23.4s
   ✓ user can login with valid credentials 12.3s
   ✓ shows error for invalid email 5.6s
   ✓ shows error for wrong password 5.5s

 ✓ checkout.test.js (2) 34.7s
   ✓ user can complete checkout 28.9s
   ✓ validates credit card format 5.8s

Test Files  2 passed (2)
     Tests  5 passed (5)
  Duration  58.12s

📹 Dashcam Replays:
  - https://console.testdriver.ai/dashcam/abc123
  - https://console.testdriver.ai/dashcam/def456

Click the Dashcam URLs to watch video replays of your tests!

Debugging with Dashcam

Every test automatically records a video replay:

Get Replay URL

test('my test', async (context) => {
  const testdriver = TestDriver(context, { headless: true });
  await testdriver.provision.chrome({ url });

  // Your test code

  console.log('Replay:', testdriver.dashcam.url);
});

Or find URLs in test output.

Open Replay

Visit console.testdriver.ai or click the URL directly.

Analyze Replay

Dashcam shows:

Full video of test execution
Timeline of all actions
Console logs synchronized
Network requests
Element highlights
Screenshots at key moments

Dashcam Features

Play/pause, speed control
Jump to any action via timeline
Frame-by-frame stepping
Download as MP4
Share with team
View network waterfall
See console logs

Complete observability guide

Understanding Failures

When a test fails, TestDriver provides rich debugging information:

try {
  await testdriver.find('submit button').click();
} catch (error) {
  console.log('Error:', error.message);
  console.log('Type:', error.name); // 'ElementNotFoundError'
  console.log('Similarity:', error.similarity); // 0.72 (how close it got)
  console.log('Screenshot:', error.debugScreenshot); // Base64 image
  console.log('Cache info:', error.cacheInfo);
}

Typical Error Output

ElementNotFoundError: Could not find 'submit button'

Debug information:
- Similarity: 0.72 (partial match found)
- Timeout: 30000ms
- Retry attempts: 60
- Cache hit: false

Screenshot saved: .testdriver/debug-screenshots/test-001.png
Dashcam: https://console.testdriver.ai/dashcam/abc123

Recent actions:
  1. ✓ navigate to https://example.com (2.1s)
  2. ✓ find('email input') (1.3s)
  3. ✓ type('[email protected]') (0.8s)
  4. ✗ find('submit button') (30.0s) FAILED

Suggestions:
- Try more specific selector: 'blue submit button in bottom right'
- Check if button is visible on screen
- Verify button loaded (check network tab)
- View Dashcam replay to see what happened

Common Debugging Scenarios

Element Not Found

Error: ElementNotFoundError: Could not find 'button'Debug steps:

Watch Dashcam replay - See what’s on screen
Check similarity score:
- > 0.8 - Close match, be more specific
- 0.5-0.8 - Partial match, adjust selector
- < 0.5 - No match, element might not be present

Try variations:

const selectors = [
  'submit button',
  'blue submit button',
  'button in bottom right',
  'primary action button'
];

for (const selector of selectors) {
  try {
    await testdriver.find(selector).click();
    break;
  } catch (error) {
    console.log(`Failed: ${selector} (${error.similarity})`);
  }
}

Check timing - Element might load later:

await testdriver.find('submit button', { timeout: 60000 });

Assertion Failures

Error: AssertionError: 'welcome message is visible' failedDebug steps:

Watch Dashcam - Verify what’s actually displayed

Simplify assertion:

// Instead of:
await testdriver.assert('welcome message says "Hello John"');

// Try:
await testdriver.assert('welcome message is visible');

Check timing:

// Wait longer for async content
await testdriver.assert('message is visible', { timeout: 60000 });

Use find() instead:

const element = await testdriver.find('welcome message');
console.log('Message text:', element.text);

Test Timeout

Error: Test timed out after 120000msDebug steps:

Increase timeout:

vitest.config.mjs

export default defineConfig({
  test: {
    testTimeout: 180000, // 3 minutes
  },
});

Check sandbox logs - Look for network issues
Watch Dashcam - See where test got stuck

Add checkpoints:

await testdriver.find('button').click();
console.log('Clicked button');

await testdriver.assert('page loaded');
console.log('Page loaded');

Flaky Tests

Issue: Test passes sometimes, fails other timesSolutions:

Check Dashcam replays - Compare passing vs failing runs

Increase stability delay:

await testdriver.find('element', {
  stabilityDelay: 500 // Wait 500ms for stability
});

Wait for network:

await testdriver.find('button').click();
// TestDriver automatically waits for network requests
await testdriver.assert('content loaded');

Disable animations in test mode:

await testdriver.exec('js', `
  document.querySelectorAll('*').forEach(el => {
    el.style.transition = 'none';
    el.style.animation = 'none';
  });
`, 5000);

Learn more about stability

Debugging Tools

1. Console Logging

test('debug test', async (context) => {
  const { testdriver } = await chrome(context, { url });

  console.log('Starting test...');

  const element = await testdriver.find('button');
  console.log('Found element:', {
    text: element.text,
    coordinates: element.coordinates,
    confidence: element.confidence,
    cacheHit: element.cacheHit
  });

  await element.click();
  console.log('Clicked element');
});

2. Screenshots

// Capture debug screenshot
try {
  await testdriver.find('element');
} catch (error) {
  // Save debug screenshot
  const fs = require('fs');
  const buffer = Buffer.from(error.debugScreenshot, 'base64');
  fs.writeFileSync('debug.png', buffer);
  console.log('Screenshot saved to debug.png');
}

3. Step-by-Step Execution

test('step by step', async (context) => {
  const { testdriver } = await chrome(context, { url });

  console.log('Step 1: Find email input');
  const email = await testdriver.find('email input');
  console.log('✓ Found:', email);

  console.log('Step 2: Type email');
  await email.type('[email protected]');
  console.log('✓ Typed');

  console.log('Step 3: Find button');
  const button = await testdriver.find('submit button');
  console.log('✓ Found:', button);

  console.log('Step 4: Click button');
  await button.click();
  console.log('✓ Clicked');
});

4. Network Inspection

// View network activity in Dashcam replay
test('check network', async (context) => {
  const { testdriver } = await chrome(context, {
    url,
    captureNetwork: true // Enable network capture
  });

  await testdriver.find('load data button').click();

  // Network requests appear in Dashcam replay
  // Filter by XHR, images, scripts, etc.
});

Performance Profiling

Identify slow tests:

# Run with reporter showing timing
npx vitest run --reporter=verbose

# Output shows duration per test:
✓ slow-test.test.js > user can checkout (34.7s)
  → find('product') (2.1s)
  → click() (0.4s)
  → find('add to cart') (1.8s)
  → click() (0.3s)
  → find('checkout') (28.9s) ← SLOW!
  → click() (0.5s)

Optimize slow operations:

Enable caching - First run is slow, subsequent runs are fast
Use parallel execution - Run tests concurrently
Check network delays - Look for slow API calls in Dashcam
Optimize selectors - More specific = faster matching

Performance optimization guide

Debugging in CI/CD

When tests fail in CI:

Get Dashcam URL

CI output includes replay URLs:

Tests failed!
Dashcam replay: https://console.testdriver.ai/dashcam/abc123

Download Artifacts

Save debug screenshots and logs:

.github/workflows/test.yml

- uses: actions/upload-artifact@v3
  if: failure()
  with:
    name: test-artifacts
    path: |
      .testdriver/debug-screenshots/
      .testdriver/dashcam-urls.txt

Reconnect to Sandbox

If test failed, reconnect to same sandbox to debug:

# Get sandbox ID from CI logs
export TESTDRIVER_SANDBOX_ID=i-abc123def

# Run tests locally against same sandbox
npx vitest run

Sandbox management guide

Vitest UI

Use Vitest UI for interactive debugging:

# Start Vitest UI
npx vitest --ui

Open http://localhost:51204 to see:

Test file tree
Test status and duration
Console output
Re-run individual tests
Filter and search tests

Next Steps

Setting Up in CI

Run tests in CI/CD pipelines

Debugging Guide

Advanced debugging techniques

Observable Features

Complete observability guide

Troubleshooting

Common issues and solutions

Dashcam Features

Overview

Getting Started

Examples

Actions

API Reference

Running & Debugging Tests

Running Tests

Test Output

Debugging with Dashcam

Understanding Failures

Typical Error Output

Common Debugging Scenarios

Debugging Tools

1. Console Logging

2. Screenshots

3. Step-by-Step Execution

4. Network Inspection

Performance Profiling

Debugging in CI/CD

Vitest UI

Next Steps

Setting Up in CI

Debugging Guide

Observable Features

Troubleshooting

Overview

Getting Started

Examples

Actions

API Reference

​Running Tests

​Test Output

​Debugging with Dashcam

Dashcam Features

​Understanding Failures

​Typical Error Output

​Common Debugging Scenarios

​Debugging Tools

​1. Console Logging

​2. Screenshots

​3. Step-by-Step Execution

​4. Network Inspection

​Performance Profiling

​Debugging in CI/CD

​Vitest UI

​Next Steps

Setting Up in CI

Debugging Guide

Observable Features

Troubleshooting

Running Tests

Test Output

Debugging with Dashcam

Understanding Failures

Typical Error Output

Common Debugging Scenarios

Debugging Tools

1. Console Logging

2. Screenshots

3. Step-by-Step Execution

4. Network Inspection

Performance Profiling

Debugging in CI/CD

Vitest UI

Next Steps