The Perils of Bugmaxxing

In 2014, I was managing a new digital banking product. We were gearing up for our initial launch and made our first QA hire to help with manual acceptance testing. We’ll call her Monica. She was a machine — she found edge cases we had never dreamed of, let alone specified, surfacing hundreds of defects in her first few weeks.

Awesome, right? Well, sort of. The problem was, a lot of the bugs Monica found were along the lines of: “the acceptance criteria say that this short text field should have a limit of 50 characters. Its limit is actually 60 characters.” Often, her reports elicited judgement calls from PMs on topics that weren’t actually meaningful for the product — the type of implementation detail that belongs nowhere near a healthy behavioural spec. In short, she found a lot of real correctness issues that were still ungrounded in the question that matters most: does a user experience this as a problem, and do they care?

And trivial issue reports aren’t harmless. A team’s focus is a real resource, and picking nits spends it. To mix entomological metaphors: it’s much less valuable to point out a few dozen (harmless, annoying) flies than it is to call attention to the open window they came in through. (Our unlaunched banking platform still had a few rooms with no windows installed at all!)

Claude will happily be your Monica

I’ve been thinking of Monica as I’ve been using the latest generation of agentic code review tools. Claude will happily be your Monica now, if you let it. Dynamic workflows and Fable will team up to produce a nearly limitless font of correctness issues; one of my test drives surfaced 50 verified bugs in about 15 minutes. What they won’t tell you, especially without prompting, is whether actual users experience those issues as a problem, and whether they care — which is how you end up with a backlog full of 200 defects you don’t understand, representing zero actual feedback on your product.

The bottleneck is decisions

To address this problem, let’s think about the constraints: where’s the bottleneck? I’d suggest that in most agentic work, the bottleneck is decision-making — given a high-enough token budget, agents will work at whatever pace you can give them decisions to work from. This is especially true when information is scarce, like it often is early in a product’s lifecycle, when fewer decisions have clear, data-driven answers. That’s why being flooded with trivial bugs is such a problem: it consumes your decision-making bandwidth at a high rate and generates relatively little product value in return.

A spec is a record of decisions

We particularly care about this at dot•requirements  because specifications are, among other things, durable records of decision-making. Every acceptance criterion is a falsifiable choice about what the product should do; in many ways, dot•requirements exists to make it easier to create, maintain, share, and trust those decisions. And a specification is also meant to illustrate taste about what qualifies as a bug. “The Continue button is prominent” is a better requirement than “the Continue button is 50 px by 150 px” precisely because it describes the thing we care about (prominence), and not the thing we trust designers, engineers, and agents to get right (pixel-perfect dimensions). Well-factored tests do the same thing.

But a spec can only illustrate taste; it can’t force a reviewer to share it. A reviewer who accepts “prominent” on its own terms validates the decision we actually made. A reviewer who demands pixels — as a bug report, as a clarifying question, fifty times a week — will pull decisions out of you that you never wanted to make, no matter how well-factored the spec is.

What I’ve actually changed

So what do we do? One option would be to turn off multi-agent code review to force focus, but I haven’t done that. I’m finding there’s real signal amid the noise — dynamic workflows are doing a better job of finding actual issues than a single-agent reviewer does. What I’ve done instead are a few concrete changes to how AI agents interact with bug reports:

1. Relentlessly prompt agents to orient on user experience. This is easier when you have an agent-facing behavioural spec, but you can do it no matter what. Any agent with the ability to report a defect needs to be admonished to describe how a user experiences the issue, what the expected behaviour is, and what the actual behaviour is.
2. Require that agents consult you before reporting a bug. “Just make a Linear issue” feels efficient in the moment, but it’s actually just moving the decision bottleneck away from the moment you have the most context. Paired with #1, you should get a legible report, at which point it’s on you to choose whether to fix, file, or ignore.
3. Use a dynamic workflow to refactor your backlog. I’m using “refactor” intentionally here — the goal isn’t to change the substance of your backlog; it’s to get rid of cruft and ensure everything that’s left meets your standards. Use the Jira or Linear MCP, and ask Claude to have a subagent review each bug for whether it still reproduces, whether it’s described in user-centric language, and whether it seems like it matters. Anything that meets the first two criteria but not the third becomes a short list for you to review.

Finding bugs is cheap now

Every bug Monica filed was real. What made them expensive wasn’t inaccuracy — it was that they kept forcing product decisions at a granularity the product didn’t need, and no spec, however well-written, was going to fix that on its own. The fix is in how the reviewer engages: what they treat as a defect, what they ask before filing, what they accept as an answer. Agentic review gives you a reviewer whose engagement you can actually dictate.

Finding bugs is now effectively free. Deciding what counts as a bug is not — it’s the scarcest resource in the room, and it can’t be delegated, because it is the product judgement your team exists to exercise. Fifty verified bugs in fifteen minutes is a lot of flies. Knowing which window to close is still your job.

If your backlog is filling up faster than your understanding of it — that’s part of what we’re building dot•requirements  for: requirements that record what the product should do, in language that keeps humans and agents oriented on what users actually care about, so when a bug report arrives, you can tell whether it matters.