--heal
to enable self-healing in your test runs. This feature allows TestDriver to automatically recover when minor changes occur, such as text updates or small UI adjustments, without requiring manual intervention.
Why use self-healing?
In modern software development, applications frequently undergo changes, whether it’s UI updates, text modifications, or layout adjustments. These changes can lead to test failures, requiring developers to manually update test scripts to reflect the new state of the application. This means that if a test fails due to a change in the application, TestDriver will attempt to recover by finding the updated element or text and retrying the action. If successful, it will update the test script accordingly. This feature is particularly useful in continuous integration (CI) environments, where tests need to adapt to frequent changes in the application.How to enable self-healing
To enable self-healing in TestDriver, simply add the--heal
and --write
flags when running your tests. The --heal
flag instructs TestDriver to recover when test fail due to minor changes in the application, while --write
saves newly generated steps to the test file.
How self-healing works
When TestDriver steps fail, the AI will progressively fall back to attempt to complete the test. When running with--heal
and --write
, TestDriver will update the test files locally with any changes made during self-healing. You can then use GitHub Actions to commit these changes and create a pull request for review.
Benefits of self-healing
- Reduced Maintenance: Eliminates the need for manual updates to tests when minor UI changes occur.
- Increased Test Resilience: Ensures tests adapt to changes without breaking.
- Continuous Integration: Keeps your test suite aligned with the latest application changes.
- Developer Collaboration: Automatically opens a PR, allowing developers to review and approve updates.
How self-healing works
- Test Execution: TestDriver runs your test suite as usual.
- Failure Detection: If a test fails due to a change in the application (for example, text or UI updates), the AI identifies the failure point.
- Recovery Attempt: The AI uses its adaptive capabilities to locate the updated element or text and retries the action.
- Test Update: If the recovery is successful, TestDriver updates the test script with the new element or text.
- Commit and PR: GitHub Actions can detect the changes and automatically commit them and create a pull request for review.
Example: Button text changes from “Submit” to “Send”
1
Initial Test
2
Application Update
- The button text is changed from “Submit” to “Send” in the application.
3
Test Execution
- TestDriver runs the test and fails to find the “Submit” button.
4
Self-Healing
- The AI detects the failure and searches for a similar element based on the
description
. - It identifies the “Send” button as the updated element and retries the action.
5
Test Update
- The test script is updated to reflect the new button text:
6
Commit and PR Creation
- GitHub Actions detects the changes and commits the updated test to a new branch and opens a PR:
- Branch Name:
auto-heal-update-submit-to-send
- PR Title:
Auto-Healed Test: Updated "Submit" to "Send"
- Branch Name:
Example GitHub Action for self-healing
Here’s how you can configure a GitHub Action to enable self-healing and automatically create PRs for any test updates:Limitations of Self-Healing
- Major UI Overhauls: Self-healing is best suited for minor changes (for example, text updates, small layout adjustments). Significant UI changes may still require manual intervention.
- Ambiguous Changes: If multiple elements match the updated criteria, the AI may require additional context to make the correct decision.
- Step Deletion: If UI or wizard pages are removed, the AI may not be able to recover. In such cases, manual updates are necessary.