find() - TestDriver

Overview

Find UI elements on the screen with natural language descriptions and AI. This returns an Element object. You can interact with the object.

Syntax

const element = await testdriver.find(description)
const element = await testdriver.find(description, options)

Parameters

string

required

Natural language description of the element to find

object | number

Optional configuration for finding and caching

Show properties

string

A custom cache key to store the element location. Use this to keep the cache clean when you use dynamic variables in prompts. You can also use it to share the cache between tests.

number

The similarity threshold (0-1) for a cache match. A lower value needs more similarity. Set it to -1 to disable the cache.

number

default:10000

The maximum time in milliseconds to poll for the element. TestDriver tries again each 5 seconds until it finds the element or the timeout ends. The default is 10000 (10 seconds). Set it to 0 to disable the poll and try one time.

number

The minimum confidence threshold (0-1). If the confidence score of the found element is less than this value, the find is a failure (element.found() returns false). Use this to make sure of good matches in critical test steps.

string

Element type hint that wraps the description for better matching. Accepted values:

"text" — Wraps the prompt as The text "..."
"image" — Wraps the prompt as The image "..."
"ui" — Wraps the prompt as The UI element "..."
"any" — No wrapping, uses the description as-is (default behavior)

boolean

default:false

A two-phase zoom mode for more precision in full UIs that have many similar elements. It is disabled by default.

boolean

default:false

This enables AI verification of the found element. When true, a second AI call makes sure that the coordinates agree with the correct element. This catches incorrect positions. It is disabled by default for less latency. When you do not set it for each call, it uses the global verify option from the SDK options.

object

AI sampling configuration for this find call (overrides global ai config from constructor).

Show properties

number

Controls randomness. 0 = deterministic. Default: 0 for find verification.

object

Sampling parameters

Show properties

number

Top-P (nucleus sampling). Range: 0-1.

number

Top-K sampling. 1 = most deterministic.

Returns

Promise<Element> - The Element instance that TestDriver found automatically.

Examples

Basic Element Finding

// Find by role
const button = await testdriver.find('submit button');
const input = await testdriver.find('email input field');

// Find by text content
const link = await testdriver.find('Contact Us link');
const heading = await testdriver.find('Welcome heading');

// Find by visual appearance
const icon = await testdriver.find('red warning icon');
const image = await testdriver.find('company logo image');

Finding with Context

// Provide location context
const field = await testdriver.find('username input in the login form');
const button = await testdriver.find('delete button in the top right corner');

// Describe nearby elements
const input = await testdriver.find('input field below the email label');
const checkbox = await testdriver.find('checkbox next to "Remember me"');

// Describe visual position
const menu = await testdriver.find('hamburger menu icon in the top left');

Interacting with Found Elements

// Find and click
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// Find and verify
const message = await testdriver.find('success message');
if (message.found()) {
  console.log('Success message appeared');
}

// Find and extract info
const price = await testdriver.find('product price');
console.log('Price location:', price.coordinates);
console.log('Price text:', price.text);

Element Object

The Element object that TestDriver returns gives these:

Methods

found() - Make a check if TestDriver found the element
click(action) - Click the element
hover() - Put the cursor on the element
doubleClick() - Double-click the element
rightClick() - Right-click the element
find(newDescription) - Find the element again with an optional new description

Properties

coordinates - Element position {x, y, centerX, centerY}
x, y - Top-left coordinates
centerX, centerY - Center coordinates
text - Text content (if available)
screenshot - Base64 screenshot (if available)
confidence - AI confidence score
width, height - Element dimensions
boundingBox - Complete bounding box

See Elements Reference for complete details.

JSON Serialization

You can serialize elements safely with JSON.stringify() for logs and for debug. TestDriver removes circular references automatically:

const element = await testdriver.find('login button');

// Safe to stringify - no circular reference errors
console.log(JSON.stringify(element, null, 2));

// Output includes useful debugging info:
// {
//   "description": "login button",
//   "coordinates": { "x": 100, "y": 200, "centerX": 150, "centerY": 225 },
//   "found": true,
//   "threshold": 0.01,
//   "x": 100,
//   "y": 200,
//   "cache": {
//     "hit": true,
//     "strategy": "pixel-diff",
//     "createdAt": "2025-12-09T10:30:00Z",
//     "diffPercent": 0.0023,
//     "imageUrl": "https://..."
//   },
//   "similarity": 0.98,
//   "confidence": 0.95,
//   "selector": "button#login",
//   "aiResponse": "Found the blue login button..."
// }

Use this for these:

To debug problems with element detection
To log the details of the test
To share element data between processes
To examine the cache performance

Best Practices

Be specific in descriptionsMore specific descriptions make the accuracy better:

// ✅ Good
await testdriver.find('blue submit button below the email field');

// ❌ Too vague
await testdriver.find('button');

Always check if foundMake sure that TestDriver found the elements before you interact with them:

const element = await testdriver.find('login button');
if (!element.found()) {
  throw new Error('Login button not found');
}
await element.click();

Include visual or positional context

// Include color
await testdriver.find('red error icon');

// Include position
await testdriver.find('search button in the top navigation bar');

// Include nearby text
await testdriver.find('checkbox next to "I agree to terms"');

Confidence Threshold

Set a minimum confidence score for element matches. If the confidence is less than the threshold, find() makes the result “not found”:

// Require at least 90% confidence
const element = await testdriver.find('submit button', { confidence: 0.9 });

if (!element.found()) {
  // AI found something but wasn't confident enough
  throw new Error('Could not confidently locate submit button');
}

await element.click();

Use this for these:

Critical test steps. An incorrect click can cause more failures.
To tell the difference between similar elements (for example, many buttons)
To fail quickly when the UI changed

// Combine with timeout for robust polling with confidence gate
const element = await testdriver.find('success notification', {
  confidence: 0.85,
  timeout: 15000,
});

The confidence value is a float between 0 and 1 (for example, 0.9 = 90%). The AI returns its confidence with each find result. You can also read this from element.confidence after a good find.

Element Type

Use the type option to show which kind of element you look for. This puts your description into a more specific prompt for the AI. It makes the match accuracy better, primarily when a description is short or not clear.

// Find text on the page
const label = await testdriver.find('Sign In', { type: 'text' });
// AI prompt becomes: The text "Sign In"

// Find an image
const logo = await testdriver.find('company logo', { type: 'image' });
// AI prompt becomes: The image "company logo"

// Find a UI element (button, input, checkbox, etc.)
const btn = await testdriver.find('Submit', { type: 'ui' });
// AI prompt becomes: The UI element "Submit"

// No wrapping — same as omitting the option
const el = await testdriver.find('the blue submit button', { type: 'any' });

Type	Prompt sent to AI
`"text"`	`The text "..."`
`"image"`	`The image "..."`
`"ui"`	`The UI element "..."`
`"any"`	Original description (no wrapping)

Use this primarily for short descriptions such as "Submit" or "Login". In these, the AI can be not sure to look for a button, a link, or visible text. When you give the type, the description becomes clear.

Polling for Dynamic Elements

By default, find() polls for a maximum of 10 seconds (it tries again each 5 seconds) until it finds the element. You can change this with the timeout option:

// Uses default 10s timeout - polls every 5 seconds
const element = await testdriver.find('login button');
await element.click();

// Custom timeout - wait up to 30 seconds
const element = await testdriver.find('login button', { timeout: 30000 });
await element.click();

// Disable polling - single attempt only
const element = await testdriver.find('login button', { timeout: 0 });

The timeout option:

Has a default of 10000 (10 seconds)
Tries to find the element again each 5 seconds
Stops when it finds the element or the timeout ends
Logs the progress during the poll
Returns the element (make a check with element.found() if it does not throw an error on a failure)
Set it to 0 to disable the poll and try one time

Zoom Mode

Zoom mode is disabled by default. It uses a two-phase method for more precision when it finds elements, primarily in full UIs that have many similar elements. To enable zoom for a specific find call, give zoom: true:

// Enable zoom for better precision in crowded UIs
const extensionsBtn = await testdriver.find('extensions puzzle icon in Chrome toolbar', { zoom: true });
await extensionsBtn.click();

// Without zoom (default)
const largeButton = await testdriver.find('big hero button');

How Zoom Mode Works

Phase 1: The AI finds the approximate location of the element.
Phase 2: TestDriver makes a 30% crop of the screen around that location.
Phase 3: The AI does the precise location on the cropped image.
Result: TestDriver changes the coordinates back to the absolute screen position.

This two-phase method gives the AI a higher-resolution view of the target area. It makes the accuracy better when many similar elements are near together.

Enable zoom with zoom: true when:

You target small elements in full UIs that have many similar elements
You need more precision for UI elements that are near together

Verify Mode

Verify mode is disabled by default. When it is enabled, a second AI call makes sure that the coordinates from find() agree with the correct element. This catches incorrect positions.

// Enable verification for critical interactions
const deleteBtn = await testdriver.find('delete account button', { verify: true });
await deleteBtn.click();

How Verify Mode Works

Phase 1: The AI finds the element and returns coordinates.
Phase 2: A second AI call looks at the screenshot at those coordinates. It makes sure that the element agrees with the description.
Result: If the verification fails, TestDriver tries the find again or marks it “not found”.

Combining Zoom and Verify

For the maximum accuracy, enable zoom and verify together. Use this for critical interactions. A click on the wrong element can cause more failures:

// Maximum accuracy: zoom for precision + verify to catch hallucinations
const element = await testdriver.find('small cancel icon next to the subscription', {
  zoom: true,
  verify: true,
});
await element.click();

Cache Options

When a test completes correctly, TestDriver caches the result of each find(). On later runs, TestDriver uses the cached match again. It does not make a new AI call. This finds the same element much more quickly. The cache is in your dashboard. TestDriver shares it between runs. Read the Cache page to see how the match, the thresholds, and the invalidation work. Control the cache to make the performance better, primarily when you use dynamic variables in prompts.

Custom Cache Key

Use cacheKey to keep the cache clean when prompts have variables:

// ❌ Without cacheKey - creates new cache entry for each email value
const email = 'user@example.com';
await testdriver.find(`input for ${email}`); // Cache miss every time

// ✅ With cacheKey - reuses cache regardless of variable
const email = 'user@example.com';
await testdriver.find(`input for ${email}`, {
  cacheKey: 'email-input'
});

// Also useful for dynamic IDs, names, or other changing data
const orderId = generateOrderId();
await testdriver.find(`order ${orderId} status`, {
  cacheKey: 'order-status'  // Same cache for all orders
});

Cache Threshold

Control how similar a cached result must be before TestDriver uses it again:

// Default: 95% similarity required
await testdriver.find('submit button');

// Strict threshold - 99% similarity required
await testdriver.find('submit button', {
  cacheThreshold: 0.01
});

// Disable cache entirely for this call
await testdriver.find('submit button', {
  cacheThreshold: -1
});

// Combine cacheKey with threshold
await testdriver.find('submit button', {
  cacheKey: 'submit-btn',
  cacheThreshold: 0.01
});

By default, TestDriver makes a cache key automatically from the SHA-256 hash of your test file. When you change your test file, the hash changes automatically. This makes the old cache entries not valid.

Manual Polling (Alternative)

If you need custom poll logic:

async function waitForElement(testdriver, description, timeout = 30000) {
  const startTime = Date.now();
  
  while (Date.now() - startTime < timeout) {
    const element = await testdriver.find(description);
    if (element.found()) return element;
    await new Promise(r => setTimeout(r, 1000));
  }
  
  throw new Error(`Element "${description}" not found after ${timeout}ms`);
}

// Usage
const button = await waitForElement(testdriver, 'submit button', 10000);
await button.click();

Use Cases

Form Fields

const emailField = await testdriver.find('email input field');
await emailField.click();
await testdriver.type('user@example.com');

const passwordField = await testdriver.find('password input');
await passwordField.click();
await testdriver.type('MyP@ssw0rd');

Buttons and Links

const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

const cancelLink = await testdriver.find('cancel link');
await cancelLink.click();

const menuIcon = await testdriver.find('hamburger menu icon');
await menuIcon.click();

Dynamic Content

// Wait for loading to complete
let content;
for (let i = 0; i < 30; i++) {
  content = await testdriver.find('results table');
  if (content.found()) break;
  await new Promise(r => setTimeout(r, 1000));
}

// Interact with loaded content
const firstRow = await testdriver.find('first row in the results table');
await firstRow.click();

Complex UI Elements

// Modals and dialogs
const modal = await testdriver.find('confirmation dialog');
if (modal.found()) {
  const confirmBtn = await testdriver.find('confirm button in the dialog');
  await confirmBtn.click();
}

// Dropdown menus
const dropdown = await testdriver.find('country dropdown');
await dropdown.click();

const option = await testdriver.find('United States option');
await option.click();

Complete Example

import { beforeAll, afterAll, describe, it, expect } from 'vitest';
import TestDriver from 'testdriverai';

describe('Element Finding', () => {
  let testdriver;

  beforeAll(async () => {
    client = new TestDriver(process.env.TD_API_KEY);
    await testdriver.auth();
    await testdriver.connect();
  });

  afterAll(async () => {
    await testdriver.disconnect();
  });

  it('should find and interact with elements', async () => {
    await testdriver.focusApplication('Google Chrome');
    
    // Find login form elements
    const usernameField = await testdriver.find('username input field');
    expect(usernameField.found()).toBe(true);
    
    await usernameField.click();
    await testdriver.type('testuser');
    
    // Find with context
    const passwordField = await testdriver.find('password input below username');
    await passwordField.click();
    await testdriver.type('password123');
    
    // Find button
    const submitBtn = await testdriver.find('green submit button');
    expect(submitBtn.found()).toBe(true);
    
    console.log('Button location:', submitBtn.centerX, submitBtn.centerY);
    
    await submitBtn.click();
    
    // Wait for success message
    let successMsg;
    for (let i = 0; i < 10; i++) {
      successMsg = await testdriver.find('success notification');
      if (successMsg.found()) break;
      await new Promise(r => setTimeout(r, 1000));
    }
    
    expect(successMsg.found()).toBe(true);
  });
});

click() - Click on found elements
hover() - Hover over elements
assert() - Verify element states
Elements Reference - Complete Element API

findAll()

Locate all elements matching a description, rather than just one.

Syntax

const elements = await testdriver.findAll(description, options)

Parameters

string

required

Natural language description of elements to find

object | number

Optional cache options (same as find())

Show properties

string

Cache key for storing element location

number

Similarity threshold (0-1) for cache matching. Set to -1 to disable cache.

Returns

Promise<Element[]> - Array of Element instances

Examples

Basic Usage

// Find all matching elements
const buttons = await testdriver.findAll('button');
console.log(`Found ${buttons.length} buttons`);

// Interact with specific element
if (buttons.length > 0) {
  await buttons[0].click(); // Click first button
}

// Iterate over all
for (const button of buttons) {
  console.log(`Button at (${button.x}, ${button.y})`);
}

Finding Multiple Items

// Find all list items
const items = await testdriver.findAll('list item');

// Find specific item by index
const thirdItem = items[2];
await thirdItem.click();

// Check all items
for (let i = 0; i < items.length; i++) {
  console.log(`Item ${i + 1}: ${items[i].text || 'No text'}`);
}

With Caching

// Cache element locations for faster subsequent runs
const menuItems = await testdriver.findAll('menu item', {
  cacheKey: 'main-menu-items'
});

// First run: ~2-3 seconds (AI call)
// Subsequent runs: ~100ms (cache hit)

Empty Results

// Returns empty array if nothing found (doesn't throw error)
const errors = await testdriver.findAll('error message');

if (errors.length === 0) {
  console.log('No errors found - test passed!');
} else {
  console.log(`Found ${errors.length} errors`);
}

Differences from find()

Feature	find()	findAll()
Return type	Single `Element`	Array of `Element[]`
If nothing found	Throws `ElementNotFoundError`	Returns empty array `[]`
Chainable	✅ Yes: `await find('button').click()`	❌ No (returns array)
Use case	One specific element	Multiple similar elements
Cache support	✅ Yes	✅ Yes

Use Cases

Table Rows

// Find all rows in a table
const rows = await testdriver.findAll('table row');

// Click every row
for (const row of rows) {
  await row.click();
  await new Promise(r => setTimeout(r, 500)); // Wait between clicks
}

// Or click specific row
await rows[2].click(); // Click third row

Checkboxes/Radio Buttons

// Find all checkboxes
const checkboxes = await testdriver.findAll('checkbox');

// Check all boxes
for (const checkbox of checkboxes) {
  await checkbox.click();
}

// Or select first unchecked
const unchecked = checkboxes[0];
await unchecked.click();

Navigation Links

// Find all navigation links
const navLinks = await testdriver.findAll('navigation link');

// Validate all are present
expect(navLinks.length).toBeGreaterThan(0);

// Click specific link by text
const homeLink = navLinks.find(link => 
  link.text?.toLowerCase().includes('home')
);

if (homeLink) {
  await homeLink.click();
}

Conditional Interactions

// Check if any error messages exist
const errors = await testdriver.findAll('error message');

if (errors.length > 0) {
  console.log(`Found ${errors.length} validation errors`);
  
  // Log each error location
  errors.forEach((error, i) => {
    console.log(`Error ${i + 1} at (${error.x}, ${error.y})`);
  });
} else {
  console.log('Form validation passed!');
}

Complete Example

import { test, expect } from 'vitest';
import { chrome } from 'testdriverai/presets';

test('select multiple items from list', async (context) => {
  const { testdriver } = await chrome(context, {
    url: 'https://example.com/products'
  });
  
  // Find all product cards
  const products = await testdriver.findAll('product card');
  
  expect(products.length).toBeGreaterThan(0);
  console.log(`Found ${products.length} products`);
  
  // Click first 3 products
  const productsToSelect = Math.min(3, products.length);
  
  for (let i = 0; i < productsToSelect; i++) {
    await products[i].click();
    console.log(`Selected product ${i + 1}`);
    await new Promise(r => setTimeout(r, 500)); // Brief pause
  }
  
  // Verify selections
  const selectedBadges = await testdriver.findAll('selected badge');
  expect(selectedBadges.length).toBe(productsToSelect);
});

Best Practices

Handle empty arrays gracefully

// ✅ Good - check length first
const items = await testdriver.findAll('list item');
if (items.length > 0) {
  await items[0].click();
}

// ❌ Bad - may throw error
const items = await testdriver.findAll('list item');
await items[0].click(); // Error if array is empty!

Use find() for single elements

// ✅ Use find() when you need exactly one
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// ❌ Unnecessary - findAll() returns array
const buttons = await testdriver.findAll('submit button');
await buttons[0].click(); // Extra array handling

Cache for performance

// First run - slow (AI call)
const items = await testdriver.findAll('menu item', {
  cacheKey: 'menu-items'
});

// Subsequent runs - fast (cache hit)
// ~10-20x faster than without cache

​Overview

​Syntax

​Parameters

​Returns

​Examples

​Basic Element Finding

​Finding with Context

​Interacting with Found Elements

​Element Object

​Methods

​Properties

​JSON Serialization

​Best Practices

​Confidence Threshold

​Element Type

​Polling for Dynamic Elements

​Zoom Mode

​How Zoom Mode Works

​Verify Mode

​How Verify Mode Works

​Combining Zoom and Verify

​Cache Options

​Custom Cache Key

​Cache Threshold

​Manual Polling (Alternative)

​Use Cases

​Complete Example

​Related Methods

​findAll()

​Syntax

​Parameters

​Returns

​Examples

​Basic Usage

​Finding Multiple Items

​With Caching

​Empty Results

​Differences from find()

​Use Cases

​Complete Example

​Best Practices

Overview

Syntax

Parameters

Returns

Examples

Basic Element Finding

Finding with Context

Interacting with Found Elements

Element Object

Methods

Properties

JSON Serialization

Best Practices

Confidence Threshold

Element Type

Polling for Dynamic Elements

Zoom Mode

How Zoom Mode Works

Verify Mode

How Verify Mode Works

Combining Zoom and Verify

Cache Options

Custom Cache Key

Cache Threshold

Manual Polling (Alternative)

Use Cases

Complete Example

Related Methods

findAll()

Syntax

Parameters

Returns

Examples

Basic Usage

Finding Multiple Items

With Caching

Empty Results

Differences from find()

Use Cases

Complete Example

Best Practices