Testing as Measurement—Why Bug-Hunting Misses the Point With Tom Gilb and Simon Holzapfel

Scrum Master Toolbox Podcast: Agile storytelling from the trenches

BONUS: Testing as Measurement—Why Bug-Hunting Misses the Point With Tom Gilb and Simon Holzapfel The Revelation That Almost Caused a Car Crash

"Tom said like 10 sentences in a row, kind of like a geometric proof, that just so blew my mind I almost drove off the road. I realized I had wasted hundreds of hours in boardrooms arguing about errors of which we were aware of perhaps 10%."

Simon shares the moment Tom's framework clicked for him. The insight? Traditional testing—finding bugs and defects—is the wrong focus entirely. It's a programmer's view of the world. Managers don't care about bugs; they care about results, about improvements in their business. Tom calls this shift moving from "testing" to "measurement of enhanced or increased value at every cycle."

The American Toast Problem

"How do we make toast in America? We burn the toast, and then we pay someone to scrape off the black bits off the bread."

Vasco invokes Deming's classic analogy to describe traditional software testing. The entire testing-at-the-end approach is fundamentally wasteful. Instead, Tom advocates for continuous measurement against quantified values. If you expected 3% progress toward your goals this week and didn't get it, you've learned something critical: your strategy needs to change. If you did get it, keep going with confidence.

Four Questions at Every Checkpoint

"Where are we going? Where are we now? Where should we have been at this point? And why is there a gap?"

Drawing from fighter pilot doctrine, these four questions should be asked at every micro-cycle—not just at quarterly reviews. Fighter pilots ask these questions every minute during critical missions, with clear abort criteria if answers are unacceptable. Most organizations have no abort criteria for their strategies at all, guaranteeing they'll discover failures far too late.

About Tom Gilb and Simon Holzapfel

Tom Gilb, born in the US, lived in London, and then moved to Norway in 1958. An independent teacher, consultant, and writer, he has worked in software engineering, corporate top management, and large-scale systems engineering. As the saying goes, Tom was writing about Agile before Agile was named. In 1976, Tom introduced the term "evolutionary" in his book Software Metrics, advocating for development in small, measurable steps. Today, we talk about Evo, the name Tom uses to describe his approach. Tom has worked with Dr. Deming and holds a certificate personally signed by him.

You can listen to Tom Gilb's previous episodes here.

You can link with Tom Gilb on LinkedIn

Simon Holzapfel is an educator, coach, and learning innovator who helps teams work with greater clarity, speed, and purpose. He specializes in separating strategy from tactics, enabling short-cycle decision-making and higher-value workflows. Simon has spent his career coaching individuals and teams to achieve performance with deeper meaning and joy. Simon is also the author of the Equonomist newsletter on Substack.

And you can listen to Simon's previous episodes on the podcast here.

You can link with Simon Holzapfel on LinkedIn.