Points vs. Hours

Traditional estimation results in what is quite brilliantly termed "The Cone of Uncertainty."  The graph shows that beginning estimates of work can range from 400 percent beyond the time actually taken to 25 percent of the time taken. The low and high estimates

 

differ by a factor of sixteen. As the project progresses and more and more gets settled, the estimates fall more and more into line with reality until there are no more estimates, only reality.

It is often framed as a debate of Points vs Hours, but in reality there is no debate. Estimating in Points is still estimation, an educated guess, but it gives far more accurate, predictable, and stable estimates. Estimating in hours will simply result in estimates that are so far off the mark as to be useless. There are reasons why traditional software projects so often come in late and over-budget. The insistence on estimating in hours instead of Points is a major factor.

Reason 1: Humans are Bad at Estimating in Hours

Humans have been able to measure time for millennia but until about 150 years ago (when clocks became more accurate and widely available) measuring time in anything more precise than days was all but unheard of. Essentially, humanity has evolved over hundreds of thousands of years not using hours and, as a result, is bad at estimating in them.

What humans are good at is estimating relative size. Research backs this up. At the outset of the Cold War, the Department of Defense, asked the RAND Corporation to forecast how modern technology would change warfare. (How long will it take for a country to develop a nuclear weapon?) RAND discovered that time-based estimates had large error rates and wide variability, and that humans were much better at estimating relative sizes, particularly those that follow the Fibonacci sequence. (The sequence is effective in estimation because the sum of the quantities of the larger number is equal to the ratio of the larger number to the smaller one. And, as the video to the left so aptly demonstrates, it is a pattern the human race has seen everywhere in the natural world since the beginning of time.)

A recent Microsoft study analyzing two Scrum teams (one estimating in hours, the other in points) showed similar results. The Team estimating in Points had an error rate of +/- 25%, whereas the team estimating in hours had an error rate as large as +/- 400% (denoted as the light grey line in the slide to your right.)

Another advantage of estimating relative size is that it is faster than trying to plan how many hours it will take to complete a job.

Reason 2: Using Hours Undermines Velocity

In Scrum, Velocity is used both to quickly project how much effort it takes to complete a job and as a metric to determine how a change in process effects productivity. (If output goes up, keep the process change, if not revert to the previous process.) If a Team measures in hours, it will be impossible to gauge output or measure productivity.

This is because hours measures input and Scrum is based on measuring output. For example, if it takes one team member an hour to complete a job and another team member two hours to complete the same job, the output (the job being completed) is the same despite the different amount of time put into the job. Estimation of output needs to be the same regardless of who is doing the work. Velocity measures the collective effort of the Team and not the input of an individual team member.

Reason 3: Time is Finite

There are only 40 hours in a typical workweek. If effort is measured in hours, a 5 member Team can never exceed 200 hours of work in a week. Hyper-Productive Scrum Teams increase output by 400% or more. In order to realize these gains when measuring in hours, every team member would have a 160-hour workweek. Not possible and definitely not a Sustainable Pace.

Improvement comes when the overall team becomes more efficient through improving both process and practices. If it once took a team member two hours to complete a job but now she can do it in one, her output has doubled but the time she put into the job has reduced by half. Measuring in hours would never capture her improvement.

As long as it is conceptually tied to time, Velocity will have no value.

Often, organizations are resistant to using points because management is concerned about Teams artificially inflating the amount of points it takes to get a job done. Why not just estimate a job at five points of effort rather than the three that it is actually worth, creating two points of inflation and artificially increasing Velocity?

Regardless of the validity of this concern, Teams are more likely to deflate point value.

For example, the team member who can now complete her job in one hour instead of two might be inclined to estimate that same job at a lower point value in a future Sprint Planning meeting because she feels like it takes her less effort now that she has improved productivity. This is point deflation. Her output is the same so the job should be estimated at the same point value.

However, both inflation and deflation can be measured and controlled using Scrum Management Metrics.

(Hours also suffer from inflationary pressures because they are often used to compare employee effort. Using a technique called Value Stream Mapping shows that few companies achieve process efficiency above 15% without focused effort. No employee wants to admit that out of a 40 hour work week they only accomplished 6 hours of actual work.)

Points

The Scrum Pattern Language of Programing : The PLoP movement codifies well know Agile practices that have been successfully implemented many times.