Setting Up in CI

TestDriver integrates seamlessly with all major CI/CD platforms. This guide covers the essential setup for running tests in continuous integration.

Quick Setup

Running TestDriver in CI requires just three things:

Install Dependencies

- run: npm install

Set API Key

env:
  TD_API_KEY: ${{ secrets.TD_API_KEY }}

Run Tests

- run: npx vitest run

Platform-Specific Setup

GitHub Actions
GitLab CI
CircleCI
Jenkins

.github/workflows/test.yml

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm install

      - name: Run tests
        run: npx vitest run
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}

      - name: Upload artifacts
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: test-artifacts
          path: |
            .testdriver/debug-screenshots/
            .testdriver/dashcam-urls.txt

Complete GitHub Actions guide

.gitlab-ci.yml

test:
  image: node:20
  stage: test
  script:
    - npm install
    - npx vitest run
  variables:
    TD_API_KEY: $TD_API_KEY
  artifacts:
    when: on_failure
    paths:
      - .testdriver/debug-screenshots/
      - .testdriver/dashcam-urls.txt

Complete GitLab CI guide

.circleci/config.yml

version: 2.1

jobs:
  test:
    docker:
      - image: cimg/node:20.0
    steps:
      - checkout
      - run: npm install
      - run:
          name: Run tests
          command: npx vitest run
          environment:
            TD_API_KEY: << pipeline.parameters.td-api-key >>
      - store_artifacts:
          path: .testdriver/debug-screenshots/
          when: on_fail

Complete CircleCI guide

Jenkinsfile

pipeline {
  agent any

  environment {
    TD_API_KEY = credentials('testdriver-api-key')
  }

  stages {
    stage('Install') {
      steps {
        sh 'npm install'
      }
    }

    stage('Test') {
      steps {
        sh 'npx vitest run'
      }
    }
  }

  post {
    failure {
      archiveArtifacts artifacts: '.testdriver/**/*', allowEmptyArchive: true
    }
  }
}

Complete Jenkins guide

Storing API Keys

Store your TestDriver API key securely in CI secrets:

GitHub Actions

Go to repository Settings → Secrets
Click “New repository secret”
Name: TD_API_KEY
Value: Your API key

GitLab CI

Go to Settings → CI/CD → Variables
Click “Add variable”
Key: TD_API_KEY
Value: Your API key
Check “Mask variable”

CircleCI

Project Settings → Environment Variables
Click “Add Environment Variable”
Name: TD_API_KEY
Value: Your API key

Jenkins

Manage Jenkins → Credentials
Add Credentials → Secret text
ID: testdriver-api-key
Secret: Your API key

Parallel Execution

Run tests in parallel for faster CI builds:

GitHub Actions Matrix
GitLab CI Parallel
CircleCI Parallelism

strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx vitest run --shard=${{ matrix.shard }}/4

Runs 4 parallel jobs, each handling 1/4 of tests.

test:
  parallel: 4
  script:
    - npx vitest run --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL

test:
  parallelism: 4
  steps:
    - run: |
        SHARD_INDEX=$CIRCLE_NODE_INDEX
        TOTAL_SHARDS=$CIRCLE_NODE_TOTAL
        npx vitest run --shard=$((SHARD_INDEX+1))/$TOTAL_SHARDS

Generating Reports

Create test reports for CI dashboards:

JUnit XML
HTML Report
JSON Report

npx vitest run --reporter=junit --outputFile=test-results.xml

- name: Publish Test Results
  uses: EnricoMi/publish-unit-test-result-action@v2
  if: always()
  with:
    files: test-results.xml

npx vitest run --reporter=html --outputFile=test-report.html

- name: Upload HTML Report
  uses: actions/upload-artifact@v3
  with:
    name: test-report
    path: test-report.html

npx vitest run --reporter=json --outputFile=test-results.json

Parse JSON for custom reporting.

Handling Test Failures

Capture debug information when tests fail:

.github/workflows/test.yml

- name: Run tests
  id: tests
  run: npx vitest run
  continue-on-error: true
  env:
    TD_API_KEY: ${{ secrets.TD_API_KEY }}

- name: Upload debug artifacts
  if: steps.tests.outcome == 'failure'
  uses: actions/upload-artifact@v3
  with:
    name: test-failures
    path: |
      .testdriver/debug-screenshots/
      .testdriver/dashcam-urls.txt

- name: Comment on PR with failures
  if: steps.tests.outcome == 'failure' && github.event_name == 'pull_request'
  uses: actions/github-script@v6
  with:
    script: |
      const fs = require('fs');
      const dashcamUrls = fs.readFileSync('.testdriver/dashcam-urls.txt', 'utf8');

      await github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: `## ❌ Tests Failed\n\nView Dashcam replays:\n${dashcamUrls}`
      });

- name: Fail workflow if tests failed
  if: steps.tests.outcome == 'failure'
  run: exit 1

Caching for Faster Builds

Cache node_modules for faster CI runs:

GitHub Actions
GitLab CI
CircleCI

- name: Cache node modules
  uses: actions/cache@v3
  with:
    path: ~/.npm
    key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-node-

cache:
  paths:
    - node_modules/
  key:
    files:
      - package-lock.json

- restore_cache:
    keys:
      - node-deps-{{ checksum "package-lock.json" }}
      - node-deps-

- save_cache:
    key: node-deps-{{ checksum "package-lock.json" }}
    paths:
      - node_modules

TestDriver’s selector cache is stored server-side, so you automatically get faster test runs without any CI configuration!

Notifications

Send notifications on test failures:

Slack
Email
Teams

- name: Notify Slack
  if: failure()
  uses: slackapi/slack-github-action@v1
  with:
    payload: |
      {
        "text": "Tests failed on ${{ github.repository }}",
        "blocks": [
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "❌ Tests failed\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
            }
          }
        ]
      }
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

- name: Send email
  if: failure()
  uses: dawidd6/action-send-mail@v3
  with:
    server_address: smtp.gmail.com
    server_port: 465
    username: ${{ secrets.EMAIL_USERNAME }}
    password: ${{ secrets.EMAIL_PASSWORD }}
    subject: Tests failed in ${{ github.repository }}
    body: Tests failed. View results at ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
    to: [email protected]
    from: CI/CD

- name: Notify Teams
  if: failure()
  run: |
    curl -H 'Content-Type: application/json' \
      -d '{"text": "Tests failed in ${{ github.repository }}"}' \
      ${{ secrets.TEAMS_WEBHOOK_URL }}

Branch Protection

Require tests to pass before merging:

GitHub

Go to Settings → Branches
Add rule for main branch
Check “Require status checks to pass”
Select “test” job

GitLab

Settings → Repository → Protected branches
Select main branch
Set “Allowed to merge” to “Developers + Maintainers”
Enable “Pipelines must succeed”

Cost Optimization

Reduce CI costs with TestDriver:

1. Leverage Test Caching

First CI run: Slow (builds cache) Subsequent runs: 1.7x faster (uses cache)No configuration needed - automatic!

2. Run Only Changed Tests

# GitHub Actions
- run: npx vitest run --changed HEAD~1

# Only runs tests affected by PR changes

3. Use Parallel Execution

strategy:
  matrix:
    shard: [1, 2, 3, 4]

4x parallel = 4x faster, but same total cost.

4. Skip Tests on Documentation Changes

on:
  push:
    paths-ignore:
      - '**.md'
      - 'docs/**'

Monitoring CI Performance

Track test performance over time:

- name: Upload timing data
  if: always()
  run: |
    npx vitest run --reporter=json > test-results.json

- name: Analyze performance
  run: |
    # Send to your analytics service
    curl -X POST https://analytics.company.com/test-metrics \
      -H "Content-Type: application/json" \
      -d @test-results.json

View performance trends at console.testdriver.ai.

Troubleshooting CI

Tests Pass Locally, Fail in CI

Common causes:

Environment differences - Check Node version matches
Missing environment variables - Verify TD_API_KEY is set
Timeout too short - Increase test timeout
Network restrictions - Check firewall allows TestDriver API

Debug:

- run: node --version
- run: npm --version
- run: env | grep TD_
- run: npx vitest run --reporter=verbose

Intermittent Failures

Solutions:

Increase timeout:

vitest.config.mjs

export default defineConfig({
  test: {
    testTimeout: 180000, // 3 minutes in CI
  },
});

Add retries:

vitest.config.mjs

export default defineConfig({
  test: {
    retry: 2, // Retry failed tests twice
  },
});

Check Dashcam replays for both passing and failing runs

Rate Limiting

If running many tests concurrently:

vitest.config.mjs

export default defineConfig({
  test: {
    maxConcurrency: 5, // Limit concurrent tests
  },
});

Contact sales for higher rate limits.

Next Steps

GitHub Actions

Complete GitHub Actions setup

GitLab CI

Complete GitLab CI setup

CircleCI

Complete CircleCI setup

Jenkins

Complete Jenkins setup

All CI/CD Guides

View all platform guides

Performance Optimization

Optimize test performance

Getting Started

Examples

Guides

Reference

Quick Setup

Platform-Specific Setup

Storing API Keys

GitHub Actions

GitLab CI

CircleCI

Jenkins

Parallel Execution

Generating Reports

Handling Test Failures

Caching for Faster Builds

Notifications

Branch Protection

Cost Optimization

Monitoring CI Performance

Troubleshooting CI

Next Steps

GitHub Actions

GitLab CI

CircleCI

Jenkins

All CI/CD Guides

Performance Optimization

Getting Started

Examples

Guides

Reference

​Quick Setup

​Platform-Specific Setup

​Storing API Keys

GitHub Actions

GitLab CI

CircleCI

Jenkins

​Parallel Execution

​Generating Reports

​Handling Test Failures

​Caching for Faster Builds

​Notifications

​Branch Protection

​Cost Optimization

​Monitoring CI Performance

​Troubleshooting CI

​Next Steps

GitHub Actions

GitLab CI

CircleCI

Jenkins

All CI/CD Guides

Performance Optimization

Quick Setup

Platform-Specific Setup

Storing API Keys

Parallel Execution

Generating Reports

Handling Test Failures

Caching for Faster Builds

Notifications

Branch Protection

Cost Optimization

Monitoring CI Performance

Troubleshooting CI

Next Steps