Forget coverage; focus on risk

In a rush? You can find a summary of the key leadership takeaways at the bottom of this post.

It’s easy to fall into the trap of thinking that more test coverage equals higher quality. Dashboards glow green, percentages look impressive, and managers nod in approval. But here’s the catch: coverage is a metric of quantity, not effectiveness. More tests can actually increase maintenance overhead, result in brittle suites, and give a false sense of security, or worse, a loss of faith in the test suite altogether.

For engineering leaders, the goal shouldn’t be more tests. It should be less risk. Quality isn’t about measuring lines of code, it’s about understanding where things can go wrong and ensuring we catch critical issues early.

The Misleading Comfort of Coverage

We’ve all seen it: teams chasing coverage thresholds, celebrating when they hit 80% or 90%, without asking what those tests are actually doing. Coverage tells us how much code is exercised, it says nothing about whether the right behaviour is validated.

High coverage can often be a hiding ground for poor quality tests that breeze past real failure scenarios. Worse, they can break at the worst possible moment, slowing development and creating frustration. It’s the difference between feeling safe and being safe.

When incidents happen despite “good” metrics, leaders are left answering uncomfortable questions.

Acknowledging the floor

I might be advocating that coverage is a bad metric, but that doesn’t mean I think we should have no coverage. Coverage can be a useful insight, but it is a terrible KPI. Low coverage still tells us risk is high, it is just that high coverage doesn’t tell us risk is low.

Good Tests > More Tests

It’s tempting to assume that more passing tests should increase confidence. That only holds if the tests themselves are high quality, which is not always the case.

When teams are targeted to reach specific coverage goals, it is not uncommon for the quality of the tests to slip. This causes two problems, false confidence and increased maintenance cost.

Common anti-patterns that sneak into tests when teams are targeted for coverage include:

• Trivial execution tests – Tests that exist purely to execute lines of code—getters, setters, or simple conditionals—without asserting meaningful behaviour. They boost percentages without reducing risk.
• Assertion-light tests – Tests that only verify “no exception was thrown” or check superficial outputs. They pass easily, but fail to catch real regressions when behaviour subtly changes.
• Over-reliance on mocks – Tests so heavily mocked that they only verify the implementation’s assumptions, not reality. They remain green while integrations fail in production.
• Implementation-coupled tests – Tests that mirror the internal structure of the code rather than the behaviour users or downstream systems rely on. These tests break during refactors, even when behaviour remains correct.

These patterns inflate coverage while actively undermining confidence. Real defects slip through despite a sea of green, while fragile tests fail for the wrong reasons. Over time, teams learn that “passing” doesn’t mean safe, and that failures are something to retry or bypass rather than investigate.

Coverage hasn’t reduced risk here, it has obscured it.

Reframing the Conversation: Risk-First Questions

Leaders shouldn’t lead with coverage metrics. Instead, ask:

• What are the most critical business risks?
• Which failures would we be embarrassed to explain to the business?

and then follow up with

• Do we have tests that mitigate those risks early in the development lifecycle?
• If not, how can we increase confidence in that area?

Risk thinking aligns quality with business impact, not just technical metrics. It’s about prioritising effort where it matters most. A holistic testing model, as I’ve explored in previous posts, supports this by combining functional testing, exploratory testing, risk-focused regression suites, and high-quality monitoring and observability.

Right Tests at the Right Time

A risk-led model doesn’t mean ignoring coverage entirely. It means being intentional. Look at failure modes: security, compliance, scalability, performance, data integrity. Coverage becomes contextual, tests matter when they mitigate the biggest risks.

For example, a critical payment processing module may only have 60% coverage, but if those tests verify key transaction flows, high-impact edge cases, and business-critical failure modes, the risk is far lower than a system showing 90% coverage built from low-value tests.

Pragmatic Trade-Offs

Not all code deserves equal attention. Focusing on exhaustive unit test coverage can slow delivery and frustrate developers. Risk-based prioritisation balances confidence with agility: reducing rework and enabling faster iteration where it counts.

But I really like metrics

We all love a good metric, but coverage is a terrible KPI. So what would I recommend instead?

Well I am a big advocate of the DORA metrics. They may not relate directly to testing, but they give an excellent overview of the health and maturity of a modern software engineering team. I also believe that to improve any of them, you need to improve your quality practices, so they are a good indirect measure of quality.

Summary

Shift your leadership conversations from code coverage to risk mitigation. Start by mapping your top business risks, reviewing test suites through that lens, and making deliberate decisions about where to invest testing effort.

Quality isn’t about metrics that make dashboards pretty. It’s about outcomes that matter, to users and the business.

TL;DR

• Coverage is a floor, not a ceiling: Use it to find untested gaps, but never as a target KPI.
• Focus on Risk: Prioritize tests that validate critical business paths and “embarrassing” failure modes.
• Measure Outcomes: Use DORA metrics (like Change Failure Rate and Lead Time To Deployment) to gauge real engineering health.

The Bottom Line: Don’t aim for more tests; aim for the right tests.

Further reading

If you enjoyed this post then be sure to check out my other posts on Engineering Leadership.

Join the flock

Subscribe to get the latest posts on the messy reality of software delivery sent directly to your mailbox. I regularly share new thoughts on leadership and strategy —zero spam, just value.

Leave This Blank:Leave This Blank Too:Do Not Change This:

Your email: