Assertions & Testing

Overview

TestDriver provides AI-powered assertions that can verify complex visual states and conditions on screen without requiring explicit element locators or coordinates.

AI Assertions

assert()

Make an AI-powered assertion about the current screen state.

await testdriver.assert(assertion)

Parameters:

assertion (string) - Natural language description of what should be true

Returns: Promise<boolean> - true if assertion passes, throws error if fails

Basic Assertions

// Verify page elements
await testdriver.assert('the login page is displayed');
await testdriver.assert('submit button is visible');
await testdriver.assert('error message is shown');

// Verify text content
await testdriver.assert('the page title is "Welcome"');
await testdriver.assert('username field contains "john.doe"');
await testdriver.assert('success message says "Account created"');

// Verify states
await testdriver.assert('the form is empty');
await testdriver.assert('the checkbox is checked');
await testdriver.assert('the dropdown shows "United States"');

// Verify visual appearance
await testdriver.assert('the button is blue');
await testdriver.assert('the loading spinner is displayed');
await testdriver.assert('the modal dialog is open');

Pattern: Polling Assertions

For conditions that may take time to become true, use polling:

// Poll until condition is true
async function waitForAssertion(testdriver, assertion, timeout = 30000) {
  const startTime = Date.now();
  
  while (Date.now() - startTime < timeout) {
    try {
      await testdriver.assert(assertion);
      return true; // Assertion passed
    } catch (error) {
      // Assertion failed, wait and retry
      await new Promise(resolve => setTimeout(resolve, 1000));
    }
  }
  
  throw new Error(`Assertion timeout: "${assertion}"`);
}

// Usage
await waitForAssertion(testdriver, 'the page has finished loading', 30000);
await waitForAssertion(testdriver, 'results are displayed', 10000);

Extracting Information

extract()

Extract information from the screen using AI.

await testdriver.extract(description)

Parameters:

description (string) - What information to extract from the screen

Returns: Promise<string> - Extracted information

Examples

// Extract text from screen
const password = await testdriver.extract('the password displayed on screen');
const total = await testdriver.extract('the order total amount');
const errorMessage = await testdriver.extract('the error message text');

// Extract structured data
const email = await testdriver.extract('the email address in the confirmation');
const orderId = await testdriver.extract('the order ID number');
const phoneNumber = await testdriver.extract('the phone number');

// Use extracted data
const password = await testdriver.extract('the password for standard_user');
const passwordField = await testdriver.find('password input');
await passwordField.click();
await testdriver.type(password);

Use Cases

Dynamic Content:

// Extract generated values
const confirmationCode = await testdriver.extract('the 6-digit confirmation code');
console.log('Code:', confirmationCode);

// Use it later in the test
await testdriver.type(confirmationCode);

Verification:

// Extract and verify
const displayedTotal = await testdriver.extract('the cart total');
const expectedTotal = '$99.99';

expect(displayedTotal).toBe(expectedTotal);

Conditional Logic:

// Extract state and make decisions
const status = await testdriver.extract('the order status');

if (status.includes('Pending')) {
  // Wait for processing
  await new Promise(resolve => setTimeout(resolve, 5000));
} else if (status.includes('Completed')) {
  // Continue with test
  const downloadButton = await testdriver.find('download receipt button');
  await downloadButton.click();
}

Testing Patterns

Test-Driven Assertions

Structure tests with clear arrange-act-assert pattern:

it('should display validation error for empty email', async () => {
  // Arrange
  await testdriver.focusApplication('Google Chrome');
  
  // Act
  const submitButton = await testdriver.find('submit button');
  await submitButton.click();
  
  // Assert
  const result = await testdriver.assert('validation error says "Email is required"');
  expect(result).toBeTruthy();
});

Multi-Step Assertions

Verify state at each step of a workflow:

it('should complete checkout flow', async () => {
  // Add item to cart
  const addButton = await testdriver.find('add to cart button');
  await addButton.click();
  
  // Verify item added
  await testdriver.assert('cart shows 1 item');
  
  // Go to checkout
  const cartIcon = await testdriver.find('cart icon');
  await cartIcon.click();
  
  // Verify cart page
  await testdriver.assert('shopping cart page is displayed');
  await testdriver.assert('cart contains the selected item');
  
  // Proceed to checkout
  const checkoutButton = await testdriver.find('proceed to checkout button');
  await checkoutButton.click();
  
  // Verify checkout page
  await testdriver.assert('checkout form is displayed');
});

Soft Assertions

Collect multiple assertion failures without stopping:

it('should validate all form fields', async () => {
  const errors = [];
  
  // Try multiple assertions
  try {
    await testdriver.assert('username field has error border');
  } catch (e) {
    errors.push('Username validation failed');
  }
  
  try {
    await testdriver.assert('email field has error border');
  } catch (e) {
    errors.push('Email validation failed');
  }
  
  try {
    await testdriver.assert('password field has error border');
  } catch (e) {
    errors.push('Password validation failed');
  }
  
  // Report all errors
  if (errors.length > 0) {
    throw new Error(`Validation errors:\n${errors.join('\n')}`);
  }
});

Complete Example

import { beforeAll, afterAll, describe, it, expect } from 'vitest';
import TestDriver from 'testdriverai';

describe('E2E Shopping Flow', () => {
  let testdriver;

  beforeAll(async () => {
    client = new TestDriver(process.env.TD_API_KEY);
    await testdriver.auth();
    await testdriver.connect({ newSandbox: true });
  });

  afterAll(async () => {
    await testdriver.disconnect();
  });

  it('should complete a purchase', async () => {
    await testdriver.focusApplication('Google Chrome');
    
    // Verify initial state
    await testdriver.assert('the product listing page is displayed');
    await testdriver.assert('cart is empty');
    
    // Add product to cart
    const addButton = await testdriver.find('add to cart button for first product');
    await addButton.click();
    
    // Verify cart updated
    await testdriver.assert('cart badge shows 1 item');
    
    // Extract product details
    const productName = await testdriver.extract('the name of the product just added');
    const price = await testdriver.extract('the price of the product');
    
    console.log(`Added ${productName} at ${price}`);
    
    // Go to cart
    const cartIcon = await testdriver.find('shopping cart icon');
    await cartIcon.click();
    
    // Verify cart contents
    await testdriver.assert(`cart contains ${productName}`);
    await testdriver.assert(`cart total is ${price}`);
    
    // Proceed to checkout
    const checkoutButton = await testdriver.find('checkout button');
    await checkoutButton.click();
    
    // Fill shipping info
    const emailField = await testdriver.find('email input');
    await emailField.click();
    await testdriver.type('[email protected]');
    
    await testdriver.pressKeys(['tab']);
    await testdriver.type('John Doe');
    
    await testdriver.pressKeys(['tab']);
    await testdriver.type('123 Main St');
    
    // Submit order
    const submitButton = await testdriver.find('place order button');
    await submitButton.click();
    
    // Verify success
    await testdriver.assert('order confirmation page is displayed');
    await testdriver.assert('success message is shown');
    
    // Extract order number
    const orderNumber = await testdriver.extract('the order number');
    console.log('Order placed:', orderNumber);
    expect(orderNumber).toBeTruthy();
  });

  it('should show validation errors', async () => {
    // Try to submit empty form
    const submitButton = await testdriver.find('submit button');
    await submitButton.click();
    
    // Verify multiple validation errors
    await testdriver.assert('email field shows "required" error');
    await testdriver.assert('name field shows "required" error');
    await testdriver.assert('address field shows "required" error');
    
    // Verify form not submitted
    await testdriver.assert('still on checkout page', false, false);
    await testdriver.assert('order confirmation page is shown', false, true); // Inverted
  });
});

Best Practices

Be specific in assertions

More specific assertions are more reliable:

// ❌ Too vague
await testdriver.assert('button is visible');

// ✅ Specific
await testdriver.assert('blue submit button is visible below the form');

Assert state changes

Verify state before and after actions:

// Before
await testdriver.assert('cart is empty');

// Action
const addButton = await testdriver.find('add to cart');
await addButton.click();

// After
await testdriver.assert('cart contains 1 item');

Use polling for dynamic content

Some assertions may need time to become true:

async function waitFor(assertion, timeout = 10000) {
  const start = Date.now();
  while (Date.now() - start < timeout) {
    try {
      await testdriver.assert(assertion);
      return;
    } catch {
      await new Promise(r => setTimeout(r, 1000));
    }
  }
  throw new Error(`Timeout: ${assertion}`);
}

await waitFor('loading complete');

Extract before comparing

Use extract() to extract values for detailed comparisons:

const actualTotal = await testdriver.extract('the cart total');
const expectedTotal = '$99.99';

expect(actualTotal).toBe(expectedTotal);

Combine with traditional assertions

Mix AI assertions with framework assertions for comprehensive testing:

// AI assertion for UI state
await testdriver.assert('success message is displayed');

// Extract and use traditional assertion for exact values
const message = await testdriver.extract('the success message text');
expect(message).toContain('successfully');
expect(message).toMatch(/order #\d+/i);

Getting Started

Examples

Guides

Reference

Assertions & Testing

Overview

AI Assertions

assert()

Basic Assertions

Pattern: Polling Assertions

Extracting Information

extract()

Examples

Use Cases

Testing Patterns

Test-Driven Assertions

Multi-Step Assertions

Soft Assertions

Complete Example

Best Practices

Getting Started

Examples

Guides

Reference

​Overview

​AI Assertions

​assert()

​Basic Assertions

​Pattern: Polling Assertions

​Extracting Information

​extract()

​Examples

​Use Cases

​Testing Patterns

​Test-Driven Assertions

​Multi-Step Assertions

​Soft Assertions

​Complete Example

​Best Practices

Overview

AI Assertions

assert()

Basic Assertions

Pattern: Polling Assertions

Extracting Information

extract()

Examples

Use Cases

Testing Patterns

Test-Driven Assertions

Multi-Step Assertions

Soft Assertions

Complete Example

Best Practices