LogoLogo
  • Overview
    • Quickstart
    • FAQ
    • Overview
    • Pricing
    • Comparison
    • Discord
  • Pro Setup
    • Book a Demo
    • 30x30 Promotion
  • Guides
    • Generate a Test Suite
    • Local Agent Setup
    • Prompting
    • Getting an API Key
    • GitHub Actions
      • GitHub Action Setup
      • Prerun Scripts
      • Environment Config
      • Parallel Testing
      • Storing Secrets
      • Optimizing Performance
      • Action Output
      • Examples
        • Test Generation
        • Parallel Testing
        • Importing Tests
        • Desktop Apps
        • Secure Log In
    • Debugging Test Runs
    • Monitoring Performance
  • Reference
    • Test Steps
      • assert
      • exec
      • focus-application
      • hover-image
      • match-image
      • hover-text
      • if
      • press-keys
      • remember
      • run
      • scroll
      • scroll-until-image
      • scroll-until-text
      • type
      • wait
      • wait-for-image
      • wait-for-text
    • Interactive Commands
      • /assert
      • /undo
      • /save
      • /run
      • /generate
    • CLI
      • testdriverai init
      • testdriverai [file]
      • testdriverai run [file]
  • Security & Privacy
    • Agent
    • Action
    • Dashboard
  • FAQ
    • Screen Recording Permissions (Mac Only)
    • Status Page
Powered by GitBook
On this page
  • 🔧 Product Capabilities
  • 🤖 Test Creation and Generation
  • 🛠️ Test Maintenance and Resilience
  • 🚨 Failures, Debugging, and Feedback
  • 🚀 Performance and Parallelism
  • 🔍 Advanced Testing Features
  • 🧩 Integration and Setup
  • 📊 Test Coverage and Effectiveness
  • 🔒 Security and Compliance
  • 🧠 AI Behavior and Prompting
  • 📦 Use Cases and Scenarios

Was this helpful?

Export as PDF
  1. Overview

FAQ

🔧 Product Capabilities

  • What is TestDriver? TestDriver is an AI-powered testing platform that simulates user interactions to automate end-to-end testing for web, desktop, and mobile applications.

  • How does TestDriver work? It interprets high-level prompts, interacts with interfaces like a user would, and verifies expected outcomes using assertions and visual validation.

  • What platforms does TestDriver support? TestDriver supports Windows, Mac, Linux desktop apps, web browsers, and mobile interfaces (via emulator or device farm).

  • Can it be used for exploratory testing? Yes. TestDriver can autonomously navigate the application to discover potential issues or generate new test cases.

  • Can it test desktop applications? Yes. It supports testing native desktop applications by simulating mouse and keyboard input and identifying UI elements.

  • Can it test mobile apps? Yes, via mobile emulators or integration with device farms.

🤖 Test Creation and Generation

  • Can TestDriver generate tests automatically? Yes, it explores the app and creates test cases based on UI flows and user interactions.

  • Can I create tests from natural language prompts? Yes. You can write high-level instructions in plain language, and TestDriver will interpret and build tests from them.

  • Can it generate tests from user stories or documentation? Yes. It can use minimal descriptions to produce complete test cases.

  • Can it turn recorded user sessions into tests? Yes, in supported environments, TestDriver can generate test steps from interaction logs or screen recordings.

🛠️ Test Maintenance and Resilience

  • What happens when the UI changes? TestDriver adapts using AI—if a button or label changes, it can often infer the correct action without breaking.

  • Do I need to rewrite tests often? No. TestDriver reduces maintenance by handling common UI changes automatically.

  • How does it handle flaky tests? It retries failed actions, assigns confidence scores, and logs inconsistencies so you can investigate root causes.

  • How are tests updated over time? You can regenerate them using updated prompts or manually edit the test specs.

🚨 Failures, Debugging, and Feedback

  • How does TestDriver report test failures? It provides detailed logs, screenshots, console output, and visual diffs.

  • What happens when a test fails? It stops execution, flags the failing step, and provides context for debugging.

  • Can I view why a test failed? Yes. You can view step-by-step logs, network traffic, DOM state, and video playback of the test run.

  • Can it automatically retry failed actions? Yes. You can configure retry behavior for individual steps or full tests.

🚀 Performance and Parallelism

  • Can I run tests in parallel? Yes. TestDriver supports parallel execution using multiple VMs or containers.

  • Can I track performance metrics during testing? Yes. It can log CPU, memory, load times, and frame rates to help catch performance regressions.

🔍 Advanced Testing Features

  • Can it validate non-deterministic output? Yes. It uses AI assertions to verify outcomes even when outputs vary (e.g., generated text or dynamic UIs).

  • Can it test workflows with variable inputs? Yes. It supports data-driven tests using parameterized inputs.

  • Can it test file uploads and downloads? Yes. TestDriver can interact with file pickers and validate uploaded/downloaded content.

  • Can it generate tests for PDFs or document output? Yes. It can open and verify generated files for expected text or formatting.

  • Can I trigger tests based on pull requests or merges? Yes. You can integrate TestDriver with your CI to trigger runs via GitHub Actions or other CI/CD tools.

🧩 Integration and Setup

  • Does it integrate with CI/CD tools? Yes. TestDriver integrates with pipelines like GitHub Actions, GitLab CI, and CircleCI.

  • Can I integrate TestDriver with Jira, Slack, etc.? Yes. You can receive alerts and sync test results with third-party tools via API/webhooks.

  • Does it support cloud and local environments? Yes. You can run tests locally or in the cloud using ephemeral VMs for clean state testing.

  • Does it work with existing test frameworks? It can complement or convert some existing test cases into its format, though full conversion depends on compatibility.

📊 Test Coverage and Effectiveness

  • How does TestDriver measure test coverage? It tracks UI paths, element interaction frequency, and application state changes to infer coverage.

  • Can it suggest missing test scenarios? Yes. Based on interaction patterns and user behavior, it can propose additional test cases.

  • Can it analyze test stability over time? Yes. You can view trends in pass/fail rates and test execution consistency.

🔒 Security and Compliance

  • Is it safe to test sensitive data? Yes. TestDriver supports variable obfuscation, secure containers, and test data isolation.

  • Can I avoid using production data in tests? Yes. You can configure mock data, sanitize logs, and use test-specific environments.

🧠 AI Behavior and Prompting

  • How does the AI understand what to test? It uses language models to interpret your goals, element names, and interface cues to perform tasks.

  • Can I adjust how the AI interprets my prompt? Yes. You can rewrite prompts, add constraints, or review and tweak auto-generated steps.

  • Can I see what the AI is doing behind the scenes? Yes. You can inspect the resolved steps, see element matches, and adjust test flows before execution.

📦 Use Cases and Scenarios

  • Can I use TestDriver to test new features? Yes. It’s great for validating changes, ensuring no regressions, and verifying rollout configurations.

  • Can it test seasonal or time-based behaviors? Yes. You can schedule tests or run them under specific date/time settings to verify time-sensitive logic.

PreviousQuickstartNextOverview

Last updated 1 month ago

Was this helpful?