Is Your Scrum Training Paying Off?
Like any other upskilling or reskilling efforts, companies need a way to determine whether the Scrum training they bought provided the ‘bang for the buck’ they expected.
The answer: metrics, metrics, metrics.
But which ones?
A recent article titled Make Sure Your Company’s Reskilling Efforts Pay Off, penned by Anand Chopra-McGowan and published in the Harvard Business Review, highlights areas that can help clients better understand the value created by training either as part of an Agile Transformation or a stand-alone effort.
Over a decade, Chopra-McGowan has compiled four main categories of metrics to look at: cost, productivity, people, and sponsor satisfaction.
Let’s take a look at each and I’ll add my view on what has been said.
Cost Metrics
This seems simple enough – what did it cost to get everyone trained?
But that single number is not an accurate picture; you also have to consider opportunity costs.
To do so, add any associated costs (ex. administrative, travel, etc.) and the cost of the employee’s time off (including what they could NOT do while getting trained).
According to an article in CFO Magazine by Perry D. Wiggins:
“Among the top 25% of organizations that are considered the best performers on this metric, annual revenue per business entity employee is $562,500 or more. Among the bottom 25%, revenue generated per employee is $188,889 or less. At the median are the organizations bringing in $322,835 in revenue per employee each year.”
Using the high number of $562,500, that means over a 5-day work week and 48-week work year, the opportunity cost of someone being in a Scrum Master by Scrum Inc. course is the price of the class ($1,999) plus lost revenue ($4,688) for a total investment of $6,687. [Note: this assumes a virtual class and does not account for applicable tax deduction opportunities.]
So, how do you justify that expense?
You have to look at the results down the road. And not just for the individual, but the team they are on or support.
A simple way is just looking at return on investment (ROI) during the following months.
For example, Chopra-McGowan cites a report commissioned by General Assembly and produced by Whiteboard Advisors where the ROI for upskilling in expensive roles (ex. software engineer, data scientist) is six times over. And while that’s compelling data, the best picture can be seen when you take productivity metrics into account.
Productivity Metrics
There are a ton of productivity metrics out there in the land of Agile.
Dr. Sutherland, a co-creator of Scrum and my mentor, loves velocity. It’s a measure of the "throughput" of work a Scrum Team generates in a given Sprint.
Most productivity metrics are looking at the speed or effectiveness of a person or team at getting something done.
Great examples of increasing these metrics can be found in figuring out how to automate things completely (ex. software testing, sorting bolts, etc.) or helping people to do what cannot be done fully automated faster and/or with higher quality.
An example Chopra-McGowan quotes is from the beauty products company, L’Oreal, who after putting their marketing team through training on search engine optimization saw a spike in web traffic.
Now, let’s add these first two metrics categories together.
Consider a single Scrum Team of 5 people who thought they were “Agile” that you’ve decided to send to training.
Let’s use the numbers from above which would bring the total investment to $33,453.
To get an accurate picture, let’s determine the cost of the team itself – an important number for Product Owners to know if they want to be able to understand the ROI for any effort the team makes.
If these 5 people, on average, earn $100,000 per person per year including benefits (average salary for an engineer in the US is $69,000/year), then the cost of the team is $500,000 annually.
If you divide that by 52 weeks in a year, then the cost of the team is $9,615 per week. If the team will be doing 2-week Sprints (pretty much industry standard), then the cost is $19,231 per Sprint.
Let’s say this team’s historical average past velocity was 50. This would give an expenditure of $385 per point of team effort.
If after training, all that happened was the team’s average velocity went up 20% (a meager result we see beaten in the field consistently), then the new cost per point (CPP) would be $321.
That puts the new output of the team at a value of $23,100, or an increase in the value of output at $3,869 per Sprint. Divide the $33,435 investment by that number and you get to a break-even point of about 9 Sprints or 4.5 months. After that, it’s pure profit. Because, on average, US workers stay at an employer for 4.1 years, you can really get a lot of value out of training your people properly.
People Metrics
This class of metrics has seen a tremendous amount of increase in use and popularity in recent years. The basics of this category are, “how do we measure the satisfaction and stability of the body of employees”.
Again, there are a lot of these out there. Retention numbers and engagement scores are very popular and there are a lot of ways to assess them.
Chopra-McGowan quotes a really interesting study by IBM where they found that new employees are 42% more likely to stay on if they receive training that enables them to do their current job better.
For me, focusing on these metrics is the most important part because they aren’t “fluffy” - they affect bottom-line results.
When people become disengaged employees, their productivity falls in ways that affect the whole organization. One simple bottom-line number where the opposite is reflected is that companies with highly engaged workforces. Consider this section from Gallup’s 2021 State Of The Global Workforce report:
“Gallup’s engagement measures have proven through 10 meta-analyses that employee engagement correlates strongly with positive performance outcomes, such as profitability, productivity, sales, safety, and retention. And, Gallup has found that the importance of employee engagement to business outcomes increases during times of crisis.”
These gains can be significant. Organizations with highly engaged workforces have earnings per share (EPS) numbers 147% higher than their counterparts (Gallup, State of the American Workplace 2017).
Additionally, talent retention is a lot more important than some people realize. The Society for Human Resource Management estimates the average replacement cost of a salaried employee to be 6 to 9 months of that salary. For an employee earning $60,000 per year that totals approximately $30,000-$45,000 in recruiting and training costs.
In my opinion, it’s much more expensive than that.
When someone leaves your organization, especially someone who has been around for a while, they leave with more than just their brainpower. They leave with institutional knowledge of your organization and deep knowledge of your products & systems.
And where do they go?
Most often, it’s to your competitors! How much does that cost your company? It could be incalculable.
Sponsor Satisfaction Metrics
While all trainers ask for feedback on whether or not the attendees enjoyed the class, this doesn’t necessarily correlate to the value received or impact generated.
Chopra-McGowan thinks a more effective approach is to ask managers and leaders what they think, after a period of time, so that they’ve had time enough to judge the difference.
I agree with Chopra-McGowan that assessment needs to be done after a while. However, I don’t believe that asking managers is more effective than asking the folks who were trained themselves.
Both groups of people need to be polled besides metrics tracked that would give a more holistic perspective of the return on investment.
A Final Assessment
One of the best pieces of advice Chopra-McGowan gives which I wholeheartedly agree with is this:
“Too often, CEO's and CFO's ask their learning and development (L&D) teams to demonstrate ROI after a training program is complete, without being clear about the return they want in the first place. L&D teams should require a definition of success - and advise their stakeholders on how to articulate it in a way that everyone can buy in to - before launching any skill-building program.”
This is Scrum.
You always want a Definition of Done established before you start work on any backlog item & upskilling employees should be no different.
And you only want to do it once.
Rework is often cited in Scrum and Lean literature as the biggest source of waste. So if you’re going to send your folks to training, pick the best provider you can afford.
With Scrum training, as with so many other things in life, you get what you pay for.
English 
French 
Spanish