2023-05-16 · Software Testing
Run a triage workflow for flaky tests
Quarantine, diagnose, delete: a repeatable routine for tests that lie.
Introduction
Flaky tests train teams to ignore red builds. This checklist covers detecting flakes automatically, a quarantine lane with a named owner, and the four root causes behind most of them.
Field notes
A few patterns we kept seeing while puttering around Software Testing problems:
- The simple version survived every refactor; the clever version didn’t.
- Documentation written during the work aged far better than documentation written after.
- The second time a problem shows up, it is already a pattern — name it.
A checklist you can steal
- The failure is reproducible with one command.
- The fix has a test that fails without it.
- Someone other than the author has run it on their machine.
Closing
Write the short version future-you will actually read. That is the whole trick.