Measuring Training Effectiveness: How to Get Started

Why Measure Training Effectiveness?

You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.

Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.

Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.

Level 4 – Results What organizational benefits resulted from the training?

^

Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?

^

Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?

^

Level 1 – Reaction How did participants react to the program?

The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).

In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.

For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.

Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.

Using the Kirkpatrick Model ............................................................................

So, how do you conduct an evaluation? The basic steps are:

1. Design the evaluation.

This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.

2. Collect the data.

Here, you conduct the survey and focus group sessions and collect operational and business performance data.

3. Analyze the data.

Analysis entails converting the raw data into useful information on which you can make evaluative judgments.

4. Report the results.

In this final step, write and distribute the report and debrief client managers and other interested stakeholders.

In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.

Level 1 (Reaction) .....................................................................................

· completed participant feedback questionnaire .......................................................

· informal comments from participants ................................................................

· focus group sessions with participants ..............................................................

Level 2 (Learning) .......................................................................................

· pre- and post-test scores ...........................................................................

· on-the-job assessments .............................................................................

· supervisor reports .................................................................................

Level 3 (Behavior) ......................................................................................

· completed self-assessment questionnaire ............................................................

· on-the-job observation .............................................................................

· reports from customers, peers and participant’s manager .............................................

Level 4 (Results) .....................................................................................

· financial reports ...................................................................................

· quality inspections .................................................................................

· interview with sales manager ...........................................................................

When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?

Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!

References ............................................................................................

Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.

Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.

2006 © Business Performance Pty Ltd. All rights reserved.



Measuring Training Effectiveness: How to Get Started - To learn more about this author, visit Vicki Heath's Website.

Like this article? Share it with your friends

E-Mail | Print | More

Related Articles
An Integrated Training Assessment Program
	An integrated assessment program also provides insight and development goals for the instructor and course design. Assessment is a diagnostic tool that ultimately helps the learners confirm what they have learned an...
ROI: Return on Investment or Return on Illusion?
	Do we really know what makes up a true return on investment? And more importantly, do we know the cost/benefit ratio well enough to really ignore understanding this elusive construct. Read here for the reality about...
Measuring Return On Investment For Soft Skill Training
	This article helps take the fear out of investing in soft skill training and outlines an effective way to measure the success of such training.
Using Analytics with Direct Marketing to Ride Out a Tough Economy -- Part 3
	This is the third in the series of how to be more effective in a tough economy. And, this article will give you some good ideas on all of the elements of each campaign to take a hard look at to hone in on the best ...
Measuring Training Effectiveness: How to Get Started
	So, you’ve been asked by your manager to demonstrate the effectiveness of the training programs you deliver. You may have been running the obligatory “smile sheets”. However, these are no longer sufficient in showin...

---

Related Forum Posts
	7 words or less for Structogram
	Jacqueline Cochran!
	Young Entrepreneur in International trading
	Re: Kevin's Case Study #9 - How do you increase forum traffic?
	Re:Do Entrepreneurs need HR resources?
	Budget.
	Should I change my name.
	Re: MLP Training Introduction