​
Description

The match-image command is used to locate an image on the screen by matching it with a reference image file and performing an action (For example, click or hover) at its center. This command is particularly useful for interacting with elements that the AI has difficulty locating using descriptions or other methods.

​
Arguments

ArgumentTypeDescription
pathstringThe relative path to the image file that needs to be matched on the screen. don’t include testdriver/screenshots/*/ in the path.
actionstringThe action to perform when the image is found. Available actions are: click or hover. The action will be performed at the center of the matched image.

​
Example usage

command: match-image
path: button.png
action: click

​
How it works

  • The match-image command takes a screenshot of the desktop and searches for the location of the reference image within the screenshot.
  • The matching logic looks for the most similar image within the screenshot, not an exact match. If the similarity is below ~80%, it will search additional scales. If no match is found, the command will fail.
  • Screenshots should be stored in the testdriver/screenshots/(mac/linux/windows)/ directory. TestDriver dynamically resolves the correct image based on the current platform.

​
Protips

  • To create high-quality screenshots for matching:
    • Download the video of the test and open it at “full” or “actual” size on your computer.
    • Use a screenshot tool (like Cleanshot X) to capture the target element.
    • Center the clickable element as much as possible within the screenshot.
  • Ensure the image file is clear and free of unnecessary visual noise to improve matching accuracy.

​
Gotchas

  • If the image match is below ~80% similarity, the command will fail.
  • Variations in screen resolution, scaling settings, or platform-specific UI differences may affect matching accuracy.
  • Ensure the image file is stored in the correct directory structure (testdriver/screenshots/(mac/linux/windows)/) for dynamic resolution.

​
Notes

  • The match-image command is ideal for interacting with visual elements that can’t be reliably described or located using other commands like hover-image.
  • This command supports flexible scaling to account for minor differences in image size or resolution.
  • Use this command as a fallback when other methods fail to locate the desired element.