flaky test
A test produces inconsistent outcomes without an intentional relevant change.
Use this when A check intermittently fails under the same expected conditions.
without the term
“This test fails maybe one in five runs for no reason. Just rerunning it makes it pass. Can you find out why instead of adding retries?”
with the term · How do you ask a model for flaky test?
Investigate this flaky test using its failure evidence. Isolate timing or shared state and verify the fix under the conditions that previously failed.
What is flaky test?
A test passes and fails on the same code. The cause may be timing, shared state, the test infrastructure, or a real race condition in the product.
Why does flaky test matter when vibe coding?
Retries can hide both test defects and intermittent application failures.
How do you do flaky test?
Capture the failure conditions, control suspected inputs, and verify the underlying cause before changing expectations.
What goes wrong with flaky test?
Deleting the assertion or increasing retries while leaving the cause unexplained.