Tests That Passed Alone and Failed Together

TL;DR

• Rate limit middleware tests passed in isolation and failed when run as a batch.
• Suspected environment variables first; logging confirmed they were being read correctly.
• The real cause was an uninitialised boolean for whether the rate limiter is enabled.
• Tests expected limiting to be active; nothing ever set the flag to true.
• Setting it properly made the tests pass both individually and together.

The rate limit middleware tests fail when run en masse and pass when run alone. That’s a specific and useful symptom — it almost always means shared state rather than broken logic.

Chasing the wrong thing first

My first theory was environment variables, since the tests passed in CI. So I console-logged the values: block codes, max attempts.

Max attempts stayed at four despite my attempts to change it temporarily, which briefly looked like a smoking gun. It wasn’t. The values were being read from the environment correctly the whole time — the variables were fine and I’d spent a while proving it.

Not wasted effort exactly. Ruling out the environment is what left only one place to look.

The actual cause

A boolean flag controlling whether the rate limiter is enabled was never initialised.

The tests expected limits to be in force. The flag was never set to true. So the tests asserted against rate limiting behaviour from middleware that wasn’t rate limiting anything.

Why does that pass in isolation? Because a single test run hits its own attempt counts and thresholds in ways that happen to satisfy the assertions. Run the whole suite and the accumulated state across tests exposes the fact that nothing is actually limiting.

Setting the enabled switch properly, and checking the surrounding variables lined up with their intended logic, made everything pass — individually and collectively.

Worth remembering

An uninitialised boolean defaults to falsy and produces no error anywhere. Nothing crashes, nothing logs, and the tests fail somewhere unrelated to the omission.

“Passes alone, fails together” is worth treating as a category rather than a mystery. It means state that outlives a single test — and the state doesn’t have to be data. Here it was configuration nobody set.