What is a flaky test, and how do you deal with it?
Quick Answer
A flaky test passes and fails without the code changing. Common causes: dependence on timing/Thread.Sleep, shared mutable state or database rows between tests, test-order dependence, real network/clock/random, and race conditions in async code. Fix the root cause; quarantine (don't just retry) while you investigate.
A flaky test is one that fails intermittently even though nothing changed. The usual culprits are timing — a Thread.Sleep that's sometimes too short — tests sharing state, like leftover database rows or a static field, tests that only pass in a certain order, and race conditions in async code. The temptation is to add a retry in CI, but that just papers over a real intermittent bug and people stop trusting the suite. I'd quarantine the test so it's not blocking, then track down the actual cause — usually it's shared state or an await I missed.
- Timing: replace sleeps with proper synchronisation or awaiting a signal; use a fake clock.
- Shared state: ensure each test sets up and tears down its own data; run tests in random order to expose ordering bugs.
- Async: await everything; don't fire-and-forget in tests.
- Retrying flaky tests in CI hides real intermittent bugs and erodes trust in the suite — quarantine and fix.