Test Runners

TestDriver's Test Runners spawn our Computer-Use agent on Ephemeral Windows VMs

The TestDriver GitHub action will execute tests when you mention it in a comment, on a schedule, or anywhere else in your CI/CD workflow.

Try the GitHub Bot

Try TestDriver in GitHub now by tagging @testdriverbot, or follow the steps below to add the TestDriver agent to GitHub.

Ask @testdriverbot to test your app within any issue or pull request! TestDriver is an agentic user tester, it has access to the entire OS and uses computer vision with mouse and keyboard control to simulate a real user.

Just tag @testdriverbot and include a markdown list of tasks. TestDriver will attempt your tasks and generate a regression test.

Installing the GitHub action

TestDriver Cloud Testing is performed via our GitHub action. You can learn more by visiting the marketplace page.

TestDriver.ai - GitHub MarketplaceGitHub

Click here to install the GitHub action in your repository.

Get your API Key

You'll need a paid account to use the TestDriver GitHub action

In order to execute your TestDriver actions on our VMs you'll need to add your API key as a GitHub secret. If you don't see an API key, you'll need ot upgrade your account.

1. Log in the team page in and copy your API key from there.

2. Paste the API key as a new GitHub secret named TESTDRIVER_API_KEY

Create your workflow

Now it's time to create your first TestDriver workflow.

In .github/workflows/testdriver.yml add the following code.

If you used testdriverai init to create your TestDriver project, these files will already be in your repository.

name: TestDriver.ai

on:
  pull_request: # run on every PR event
  schedule:
    - cron: '0 * * * *' # run every hour
  push:
    branches:
      - main # run on merge to the main branch
  workflow_dispatch:

jobs:
  test:
    name: "TestDriver"
    runs-on: ubuntu-latest
    steps:
      - uses: dashcamio/testdriver@main
        version: v4.0.0
        key: ${{secrets.TESTDRIVER_API_KEY}}
        with:
          prompt: |
            1. /run /Users/ec2-user/actions-runner/_work/testdriver/testdriver/.testdriver/test.yml
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          FORCE_COLOR: "3"

Notice that on line 21-22 we're invoking the Computer-Use Agent. In this case we're using the /run command to execute our file from our local directory.

You can also use commands like /explore, use variables, or supply your prompts dynamically from earlier steps. A common workflow is wait for staging to deploy before executing the test.

Output

See Action Output for output definitions and code samples.

How it works

1. The GitHub action is triggered via the conditions supplied via on

2. The key value is used to authenticate you

3. An ephemeral virtual machine is spawn on our infrastructure

4. The code from the current branch will be cloned on to the VM

5. Dashcam begins recording

6. If supplied, the prerun shell script runs

7. prompt is parsed as a markdown list.

8. Each list item from promp is fed into TestDriver one by one

9. TestDriver summarizes the test and sets it's exit code depending on it's pass or failed state

10. Dashcam ends recording

11. The VM is destroyed and all data is wiped

Deploy the test

Save the file, make a new branch, push to your repository, and create a new pull request.

git checkout -b testdriver
git commit -am 'add testdriver github action'
git push origin testdriver
gh pr create --web

This will trigger a new TestDriver execution.