Skip to main content
Speed matters when you’re running hundreds or thousands of tests. TestDriver is engineered for performance with intelligent caching that delivers up to 1.7x faster test execution by skipping redundant AI vision analysis.

​
Test Caching Performance

Selector caching dramatically reduces test execution time by skipping redundant AI vision analysis: 
// First run: builds cache
await testdriver.find('submit button');

// Second run: exact match
await testdriver.find('submit button');
Performance Improvement: 1.7x faster – The more you run your tests, the faster they get. Cache hits are nearly instant, turning slow AI analysis into millisecond lookups.
  • First run: 45.2s with 12 AI vision calls
  • Cached run: 26.6s with 11/12 cache hits

​
Intelligent Cache Strategy

TestDriver uses a sophisticated three-tier matching system to maximize cache hits while maintaining accuracy:
1

Exact Hash Match (Fastest)

Perceptual hash comparison for identical screenshots.
  • Speed: Instant (< 1ms)
  • Accuracy: 100%
  • Use case: Repeat test runs
2

Pixel Diff Match (Fast)

Pixel-by-pixel comparison for similar screenshots.
  • Speed: Very fast (< 50ms)
  • Accuracy: 95%+ similarity threshold
  • Use case: Minor UI changes (hover states, animations)
3

Template Match (Fallback)

Edge detection and template matching for structural similarity.
  • Speed: Fast (< 200ms)
  • Accuracy: 75%+ confidence
  • Use case: Layout changes, responsive design

​
Automatic Caching

Caching is enabled automatically with zero configuration: 
import { test } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('auto-cached test', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://example.com'
  });

  // First call: AI analyzes screen, saves to cache
  await testdriver.find('More information link'); // 2.1s

  // Second call: cache hit, instant response
  await testdriver.find('More information link'); // 12ms ⚡
});
File-Based Cache Keys: TestDriver automatically generates cache keys from a hash of the test file calling the TestDriver methods. This means each test file has its own isolated cache namespace, preventing cache pollution between different tests while enabling fast lookups within the same test file.

​
How Cache Keys Work

The cache key is computed from:
  • File hash: SHA-256 hash of the test file contents
  • Selector prompt: The exact text description passed to find()
  • Screenshot context: Perceptual hash of the current screen state
  • Platform: Operating system and browser version
When you modify your test file, the hash changes automatically, invalidating stale cache entries and ensuring fresh AI analysis with your updated test logic.

​
Cache Hit Monitoring

Track cache performance in real-time: 
test('monitor cache performance', async (context) => {
  const { testdriver } = await chrome(context, { url });

  const element = await testdriver.find('submit button');

  if (element.cacheHit) {
    console.log('✅ Cache hit - instant response');
    console.log('Strategy:', element.cacheStrategy); // 'exact', 'pixeldiff', or 'template'
    console.log('Similarity:', `${(element.similarity * 100).toFixed(1)}%`);
    console.log('Cache age:', element.cacheCreatedAt);
  } else {
    console.log('⏱️  Cache miss - AI analysis performed');
    console.log('New cache entry created');
  }
});
Example output: 
✅ Cache hit - instant response
Strategy: pixeldiff
Similarity: 97.3%
Cache age: 2024-12-02T10:15:30Z

​
Configurable Thresholds

Fine-tune cache behavior for your application: 
// Default: 95% similarity required
await testdriver.find('submit button');

// Explicit strict threshold
await testdriver.find('submit button', {
  cacheThreshold: 0.01 // 99% similarity
});
Best for:
  • Static content
  • Stable UI elements
  • Critical interactions

​
Cache Persistence

Cache entries are stored server-side with intelligent expiration:
PropertyValue
LocationServer (MongoDB + S3)
Persistence7 days rolling window
ScopePer-team, per-test-file
MatchingScreenshot similarity + prompt + OS
InvalidationAutomatic on file changes
The cache automatically invalidates when your test code changes, ensuring accuracy while maximizing performance.

​
Cache Analytics

View cache performance in the TestDriver console:
1

Access Console

Visit console.testdriver.ai
2

View Cache Metrics

See detailed cache statistics:
  • Cache hit rate per test
  • Most frequently cached selectors
  • Cache age distribution
  • Cost savings from caching
3

Manage Cache

Delete stale entries or clear cache for specific tests.

​
Performance Best Practices

// ✅ Good - consistent wording
await testdriver.find('submit button');
await testdriver.find('submit button'); // Cache hit

// ❌ Bad - different wording
await testdriver.find('submit button');
await testdriver.find('the submit button'); // Cache miss
Cache matching is prompt-specific. Use the exact same description for maximum cache hits.
// Stable elements: strict threshold
await testdriver.find('navigation logo', {
  cacheThreshold: 0.01
});

// Dynamic elements: lenient threshold
await testdriver.find('live feed item', {
  cacheThreshold: 0.10
});
Match your threshold to element stability.
// ✅ Recommended - per-file automatic caching
await testdriver.find('button');

// ⚠️  Advanced - custom cache key for shared cache
await testdriver.find('button', {
  cacheKey: 'global-buttons'
});
Automatic file-based cache keys prevent cross-test issues while enabling caching.
// ❌ Without cache key - creates new cache for each variable value
const email = '[email protected]';
await testdriver.find(`input for ${email}`); // Cache miss every time

// ✅ With cache key - reuses cache regardless of variable
const email = '[email protected]';
await testdriver.find(`input for ${email}`, {
  cacheKey: 'email-input'
});

// Also useful for dynamic IDs, names, or other changing data
const orderId = generateOrderId();
await testdriver.find(`order ${orderId} status`, {
  cacheKey: 'order-status'  // Same cache for all orders
});
Custom cache keys prevent cache pollution when using variables in prompts, dramatically improving cache hit rates.
test('track cache performance', async (context) => {
  const { testdriver } = await chrome(context, { url });

  let cacheHits = 0;
  let cacheMisses = 0;

  const selectors = ['button 1', 'button 2', 'button 3'];

  for (const selector of selectors) {
    const element = await testdriver.find(selector);
    if (element.cacheHit) cacheHits++;
    else cacheMisses++;
  }

  console.log(`Cache hit rate: ${(cacheHits / selectors.length * 100).toFixed(0)}%`);
});
Track your cache hit rates to optimize threshold settings.

​
Real-World Performance

Here’s a real test suite performance comparison: 
$ npx vitest run

 ✓ test/login.test.js (3 tests) 87.4s
 ✓ test/checkout.test.js (5 tests) 142.6s
 ✓ test/profile.test.js (4 tests) 98.2s

 Test Files: 3 passed (3)
 Tests: 12 passed (12)
 Duration: 328.2s
Performance improvement: 2.6x faster with an average 91% cache hit rate across all tests.

​
Advanced: Custom Cache Configuration

For advanced use cases, configure caching globally: 
import { TestDriver } from 'testdriverai';

const testdriver = new TestDriver({
  apiKey: process.env.TD_API_KEY,
  cacheKey: 'my-test-suite',
  cacheDefaults: {
    threshold: 0.05,      // 95% similarity
    enabled: true,
    ttl: 604800,          // 7 days (in seconds)
  }
});

​
Learn More

Parallel Execution

Run tests concurrently for maximum speed

Selector Caching Guide

Deep dive into caching strategies

AI Prompt Caching

Cache AI-generated test code

Performance Guide

Optimize test suite performance

Console Dashboard

View cache analytics