Cost of Defects in Requirements

     Tom Gilb, Jeff Sutherland, Kai Gilb

Requirement defects cause rework. Every defect is a bug that causes extra work. So it is important to check your requirements for defects.

Tom Gilb has an agile one hour workshop that will take a random page of your specifications and identify the number of defects. You can then multiple the defects discovered on this page by the total number of pages of requirements to estimate the total number of requirement defects that you are planning to implement.

Defects are things like:

1. The requirement is not clear to the developer implementing it.
2. The requirement specifies technical details instead of the user requirement.
3. The requirement is not testable.

In a recent paper, Tom describes a one hour workshop with management in a large company. They identified 33 defects in an hour. He has data from many decades of work showing that the first time through you will only identify 1/3 of the defects. If you spend another hour you will find many more defects.

So in one page, many companies have 100 defects. Any major defect has been shown to take about 10 hours of extra work. In one of the examples in Tom's paper he tells the management they have 2000 hours of rework ahead of them. They are surprised and ask him how he knows that. The project team has been working for a year and was supposed to be finished after about 2000 hours of project time. However, they still have another year of work.

So take a look. 100 defects for every page of product backlog items may be slowing your projects down. Tom Gilb. Agile Specification Quality Control: Shifting Emphasis from Cleaning to Sampling Defects. TE Testing Experience. March, 2009. Here is a link to Tom's paper.