LLM-Assisted Test Automation¶
Coverage Expansion Modes¶
| Mode | Purpose | Representative Prompt | Metrics | Caution |
|---|---|---|---|---|
| Unit Completion | Routine test generation | "Function spec → test case table" | Branch coverage | Spec error propagation |
| Spec → Test | Specification-driven | "Spec Markdown → pytest" | Spec diff detection rate | Ambiguous language interpretation |
| Regression Summary | Failure log summarization | "Failure log → failure cause clusters" | Cause cluster accuracy | Sensitive log leakage |
| Property Discovery | Invariant extraction | "Code → invariants to enforce" | Invariant detection count | Over-generation |
Guardrails¶
Evaluation criteria:
- Execution success rate of generated tests
- PII contamination rate in dummy data
- Flake ratio (failures/successes over N runs)
Workflow Example¶
- Collect function docstrings
- LLM: Extract boundary values + error conditions
- Convert to property sets (e.g., Hypothesis)
- Initial run of generated tests & summarize failure clusters
- Manual review confirmation → commit to repository
Next Actions¶
- Integrate test diff auto-review with GitHub Actions
- Flake detection scoring
Back to: index.md