By Hollace S. Masino
Your ability to evaluate training has rarely been as vital to organizational success as it is today.
Like many businesses, your organization is likely evolving training programs to keep pace with the impact of the pandemic. These updates may include changes in what skills are prioritized, which resources are leveraged and even where training takes place.
Understandably, such transformations come at a significant cost. Effective evaluation can ensure time and resources are well spent as you strive to meet this moment of disruption and prepare your workforce to succeed.
As you plan your next training, the following steps can help you choose what to measure as well as how to capture solid data and analyze your results to make effective training decisions and communicate the value of your efforts.
Gain Commitment
Training evaluation requires a team as well as a budget. Your organization will need to make a commitment to secure the resources required to effectively evaluate training. This commitment sets the stage, prevents misunderstanding and helps ensure follow-through.
It’s a good idea to include evaluation in the annual training projects planning process so that it becomes part of the workflow and budget just like communication plans for training initiatives.
Identify What and Why
Determining what will be evaluated and why go hand in hand. Establishing your purpose, what you wish to measure and which metrics may be linked will inform your decisions concerning which model, tools and methods to select and vice versa. You will want to begin by looking at learning objectives to help determine evaluation metrics.
Understand, however, that you may need to go a step further to get a more robust picture. For example, your team and organization may want to know:
- What sales representatives think about a new simulation activity, so you can make training improvements.
- How well sales representatives deliver messages during simulation, to determine if enough practice is provided.
- What extent new selling skills are applied on the job, so managers can provide appropriate levels of training reinforcement and ensure accountability.
- How skills transfer impacts sales and what other factors contribute to sales numbers.
- What ways customers and patients are impacted by new selling behaviors.
- How outcomes and benefits from training compare to training input and costs.
- Why training didn’t accomplish intended goals, to identify and remove any obstacles.
Decide upon the metrics that are most important to your goals. As your team prioritizes what will be measured, select a model and methods to inform the process.
There are several available training evaluation models to choose from. Think of each as a guide to help you develop the right questions and determine suitable data collection methods. Though by no means exhaustive, some of the most popular and easy-to-use models are summarized below.
- Kirkpatrick’s 4 Levels is the most popular and widely used evaluation model, because it’s relatively easy to apply. This is a great place to start for those new to the application of training evaluation.
- Phillips ROI follows and builds on Kirkpatrick’s model and includes the additional step of calculating the monetary value of training results. This is a good evaluation model when you need to provide return on investment (ROI) information to stakeholders to prove the monetary value of training. Keep in mind that gathering monetary investment and return data can be a complex endeavor.
- Kaufman’s 5 Levels combines the Kirkpatrick’s and Phillips models while also measuring how training impacts people outside the organization. By doing so, this model provides a more holistic understanding not only of the impact training had on employees, the business and the ROI but also how customers, the community and society at large are affected by the training. This model allows you to measure how training is used as a force for good.
- Brinkerhoff’s Success Case Method (SCM) is unique in that it intentionally evaluates and compares the most and least successful cases to determine how well a program works — or reasons why it doesn’t — from a qualitative perspective. It allows learning and development professionals to tell the story of training. However, it may not give you the quantitative data you need. The SCM is ideal for innovative and individualized professional development, such as coaching programs, because it helps you dig deeper, learn lessons, and spotlight training “wins.”
Finding the Right Model
The accompanying table can help you select the right model for your needs.
For more information on these and other evaluation models, consider referring to Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance, and Change by Darlene Russ-Eft and Hallie Preskill. This academic text provides solid guidance on the models as well as the process of training evaluation.
Make Evaluation Fit Your Needs
If you’re just beginning to apply evaluation methods in your training projects, it may be best to start simple and allow one of these established models to guide you. For example, if you were using Kirkpatrick’s 4 Levels, you might begin by creating participant surveys and short quizzes to gather data on how to improve
training. Once you’re proficient at deploying these, you could then aim to identify the next complex factor, behavior transfer. This deeper dive and the inclusion of higher levels will give a more accurate and robust picture of training and better enable well-informed training improvements.
Keep in mind that you aren’t limited to using just one model. Choose metrics and methods that best serve the aims of your project. Combining them into a hybrid model can be a great way to best suit your organization’s unique needs and constraints.
Prepare for Follow-Through
Plan and design an evaluation package at the same time you plan the learning and communications packages for your training projects. Major milestones might look like this:
- Create an evaluation plan, including schedule, scope and budget.
- Define your model, methods and metrics.
- Design and develop the measurement tools.
- Deploy the tools and gather data.
- Analyze data.
- Draw conclusions, make recommendations and communicate findings to stakeholders.
- Continually improve upon the evaluation process as you would any training program.
Final Considerations
Be mindful that the more complex the model and data collection methods, the more effort, planning, time and money it will take to successfully execute your plan. At the same time, always remember that this is an investment well made.
Training is too important — especially in this moment of ongoing change — not to also focus on the evaluation that enables you to ensure its success.
Hollace Masino is a senior instructional designer with Encompass Communications and Learning. Email Hollace at hmasino@encompasscnl.com.
