Context:
Corporates spend approx. 1 billion US$ every year on training the six million frontline workers employed in their organizations. But the Return on Investment (RoI) on training has been elusive. Two key factors are responsible for the mismatch between the training outcomes and investment: Attribution and Quantification of benefits.
In this paper, we deep-dive into these two challenges:
1. Kirkpatrick’s 4-level theory for measuring the effectiveness of learning programs:
Donald L. Kirkpatrick was a renowned professor at the University of Wisconsin. He developed the famous concept for evaluating the effectiveness of employee training programs. The framework measures training efficacy on four pillars: Reaction, Learning, Behaviour, and Results.
Level 1-Reaction:
Measure satisfaction levels and immediate reactions of participants, post training.
Level 2-Learning:
Assess the knowledge, skills, and attitudes gained during the training.
Level 3-Behaviour:
Evaluate how well the participants apply their new skills on the job.
Level 4-Results:
Analyse the impact on business outcomes (e.g. productivity, quality, sales)
Most of the current organizations gauge their training programs only on level 1 and level 2. However, RoI calculations are not possible and accurate at these levels. In order to get the expected RoI from training programs, the measurement criteria must meet level 4.
ROI in training is realized when learning translates into measurable improvements—financial gains, productivity boosts, and employee retention. Organizations must track KPI changes, attribute results scientifically, and align training with business outcomes to ensure true effectiveness.
– T Muralidharan
ROI in training is realized when learning translates into measurable improvements—financial gains, productivity boosts, and employee retention. Organizations must track KPI changes, attribute results scientifically, and align training with business outcomes to ensure true effectiveness.
2. The scientific financial model for calculating RoI of training programs:
Calculating the impact of training involves measuring their effectiveness in achieving desired outcomes. Here is a structured approach:
Step 1: Define the training objectives:
Identify specific, measurable, and achievable goals (e.g., increase productivity by up to 10%, reduce errors by up to 15%, improve customer satisfaction by 2 points) and set Key Performance Indicators (KPIs) tied to the organizational objectives (e.g. revenue per employee, task completion rate, residency of employees)
Step 2: Measurement:
Collect KPI data before the training program to establish the benchmark. Track KPI data after the trainings at specified intervals (30 days, 60 days, 90 days, 180 days, 365 days after training).
Step 3: Calculate the % change in the KPI metric:
Step 4: Quantify the change in KPI attributable to training:
This step is tricky. There are three components of attribution, and we need scientific methods to calculate each of these components.
• How much of change in KPI has been “caused” by the training? (therefore, attributable directly to the training).
• How much of the benefit is due to “other factors”?
• How long is the impact of training? In other words, what is duration of benefit that can be attributed to the training program?
The science behind attribution is discussed later in this paper.
The KPI change attributable to training is derived after correcting for the KPI changes due to “other” factors and considering the duration up to which the benefits are realized.
Step 5: Quantify the financial value of the % change in KPI:
Step 6: Calculate the ROI (Return on Investment)
Example:
• Monetary benefits attributed to training (in the form of increased productivity or reduced attrition costs): ₹50,00,000/-
• Training Costs: ₹10,00,000/-
• ROI: (50,00,000 ÷ 10,00,000) x 100 = 500%
This means that for every one rupee spent purely on training, the company earned five rupees in return.
3. Challenges:
the RoI. They are given below:
3.1 Difficulty in attributing correctly is the first challenge to RoI Calculation:
Factors to consider:
• External factors such as market dynamics, economic shifts, competition, or changes in company strategy can skew RoI measurements (attribution to other factors).
• The benefits of training might not be immediate and can take months to fully materialize. Similarly, the duration of the benefit can be either short term (6 months) or long term (1 year or more) (attribution of benefit duration).
3.2. Methodology to combat the above factors:
3.2.1. Attribution of “cause”
The science of people performance improvement is based on identifying specific behaviours linked to training. It connects training
investments with measurable performance improvements which are directly tied to business targets.
The training must be the cause of the change in the KPI. In other words, training must be designed to impact the KPI and there must a scientific basis for directly correlating the training with the KPI. For e.g., soft skills training may impact behaviour, but it is exceedingly difficult to establish that it directly leads to KPI improvement.
3.2.2. Role Performance Modelling Framework (RMF):
Performance happens always in the context of the role. The real “cause and effect” relationship in people performance can be scientifically derived from Role Performance-Modelling Framework (RMF). In this framework, the role is drilled down and broken up into activities, tasks, and sub-tasks. Using the Activity Based Costing (ABC) model, the sub-tasks are ranked with respect to time, cost and quantity, and the cost build-up of the role is established. This helps in arriving at the critical tasks in the role which have a significant, direct impact on performance outcome. These critical tasks are the true levers of performance. The output on these critical tasks is called Lead Indicators of Performance (LIP). Any training which improves the lead indicators will automatically improve the role outcomes. Skills and knowledge must be applied on these critical sub-tasks in the role for performance to happen. Hence, designing training programs to improve these lead indicators and measuring their improvement as the KPI will establish the “cause” attribution. The key axiom is that training intended to improve lead indicators will show an improvement in outcome, and this phenomenon is causal.
3.2.3. Attribution due to “other factors”:
A straightforward method to assess the impact of “other factors” is by computing the change in KPI for two groups: A control group that underwent the training and a non-control group that did not undergo the training. If all other parameters are kept constant between the control group and non-control group, the change in KPIs between the two sets of employees can be attributed only to the trainings.
3.2.4. Attribution of duration of benefit:
How long does the effect of training on the trainee last? The answer to this question depends on the purpose, duration, and scope of training. There are numerous studies and models to arrive at the duration of training impact, a few of which are listed in Annexure 1. It is a good practice to measure the duration of training impact by tracking the KPI until it plateaus. But the author recommends that for frontline, where the role induction training durations are short, the training impact can be assumed to be between 6 to 9 months, after the completion of training.
3.3. Quantification of the Training impact:
The quantification of training impact is calculated by computing the cumulative average of the KPI after the training. Then, the pre-training KPI is subtracted to arrive at the gross change in KPIs. The changes due to “other factors” is then subtracted from the gross value to arrive at the net change in KPI, which can be attributed to training.
4. Difficulty in calculating the monetary value of the KPI is the secondcchallenge Factors to consider:
The cost of training can be easily computed in monetary terms. It broadly includes the cost of the trainer, cost of travel, logistics, stay and food for trainees, and cost of salary paid during the training period. However, the problem lies in the calculation of the monetary value of the KPI change. For e.g., how does one calculate the monetary value of increased customer acquisition or increased productivity or increased retention? Financial methods need to be designed for calculating the monetary value of change in each KPI.
For e.g., the monetary value of increased employee retention by 3 months can be calculated by the below steps:
1. Calculate the average monthly ratio of “revenue to manpower cost” multiplier.
2. Calculate the revenue increase due to 3-month extra retention.
3. Calculate the contribution of the increased revenue.
4. Calculate the manpower cost of the increased retention.
5. Subtract the manpower cost from the contribution.
6. The net figure is the monetary value of the increased retention.
5. Summary
The scientific method to calculate the RoI using the above discussions is given below:
1. Identify a KPI (ideally a Lead indicator of Performance).
2. Design the training program to impact one or more of the lead indicators.
3. Finalise the training impact duration.
4. Calculate the change in the lead indicator before and after training program.
5. Subtract the change due to “other factors” to arrive at the net KPI change.
6. Calculate the monetary value of the net KPI change.
7. Calculate the all-in training investment.
8. Calculate the RoI.
6. Conclusion
Effective ROI measurement in training requires a combination of sound scientific methods, reliable data, training alignment with KPI, financial methods to calculate the monetary value of KPI changes. If the training is designed properly, it is possible to calculate RoI on training.
Leave a comment