Tweet

Measuring Training Effectiveness: How to Get Started

Written by: Vicki Heath

Article Overview: So, you’ve been asked by your manager to demonstrate the effectiveness of the training programs you deliver. You may have been running the obligatory “smile sheets”. However, these are no longer sufficient in showing your worth to the organization. Where do you begin in sorting out the complexities of training evaluation? Find out with this starters guide.

Measuring Training Effectiveness: How to Get Started

Why Measure Training Effectiveness?

You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.

Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.

Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.


Level 4 – Results What organizational benefits resulted from the training?
^
Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?
^
Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
^
Level 1 – Reaction How did participants react to the program?


The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).

In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.

For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.

Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.


Using the Kirkpatrick Model ............................................................................

So, how do you conduct an evaluation? The basic steps are:

1. Design the evaluation.
This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.

2. Collect the data.
Here, you conduct the survey and focus group sessions and collect operational and business performance data.

3. Analyze the data.
Analysis entails converting the raw data into useful information on which you can make evaluative judgments.

4. Report the results.
In this final step, write and distribute the report and debrief client managers and other interested stakeholders.

In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.

Level 1 (Reaction) .....................................................................................

· completed participant feedback questionnaire .......................................................
· informal comments from participants ................................................................
· focus group sessions with participants ..............................................................

Level 2 (Learning) .......................................................................................

· pre- and post-test scores ...........................................................................
· on-the-job assessments .............................................................................
· supervisor reports .................................................................................

Level 3 (Behavior) ......................................................................................

· completed self-assessment questionnaire ............................................................
· on-the-job observation .............................................................................
· reports from customers, peers and participant’s manager .............................................

Level 4 (Results) .....................................................................................

· financial reports ...................................................................................
· quality inspections .................................................................................
· interview with sales manager ...........................................................................

When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?

Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!


References ............................................................................................

Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.

Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.


2006 © Business Performance Pty Ltd. All rights reserved.

Related Articles
  Measuring Effectiveness of Your Team
  An Integrated Training Assessment Program
  ROI: Return on Investment or Return on Illusion?
  Measuring Return On Investment For Soft Skill Training
  Why do you need Key Performance Indicators?

Article Tags:



Related Forum Posts
7 words or less for Structogram - Some "7 words or less" (more or less) for Structogram for your comments: Training to get your message across(6) Secrets to get your message across (6) Training so people will listen to you (7) Helping you get your message across (6) Training to learn to get your message across (8) Communications training for yourself and your team (7)
Re: What or Who Sparks Your Business Interest - Hi This is really a great thread. For me it was always wanting to be responsible for myself without anyone have the power to make my decisions for me. I have also done many different things in my life. Started as a hairdresser and did that for 20 years, some of them working from home (which was good when my children were young and needed me around). Then worked as a bookkeeper and then a credit manager in a video distribution company for 18 years. Loved the job but hated having to answer to a boss. In between joined an MLM company selling health and weight loss products and then became a bookkeeper again but working for myself with my own clients. Started internet marketing and this I truly love. I can work the hours that suit me and it's totally up to me to do the work if I want to succeed. MichelleJ
Jacqueline Cochran! - Started out as a poor sharecropper, ended up as the founder of a cosmetic line and a pilot, head of the WASP during WWII. From starting from adversity and reaching the top, she's got to be up there as one of the greats.
Young Entrepreneur in International trading - I’m 23. Owns an ice-cream company. Started the company when I was 20. Increased sales compared to previous owners with 50%. Are now looking for new opportunities to expand my business.
Budget. - I believe the biggest barrier is related to budget. Training tends to be a normal practice for a big company. But I have to consider it seriously as an entrepreneur.


Share this article with your friends. Fund someone's dream.

Leave a comment below or share on the left and you'll help support entrepreneurs in Africa through our partnership with Kiva. Over $50,000 raised and counting - Please keep sharing! Learn more.

Tweet

Newsletter

Get advice & tips from famous business
owners, new articles by entrepreneur
experts, my latest website updates, &
special sneak peaks at what's to come!

Name:
Email:

Popular Articles

The Right Job - Part Five 'Compensation'

Making the Most of Your Trade Show Experience

Sales Training – Top Salespeople Are Not Dunces

Suggestions

Email us your ideas on how to make our
website more valuable! Thank you Sharon
from Toronto Salsa Lessons / Classes for
your suggestions to make the newsletter
look like the website and profile younger
entrepreneurs like Jennifer Lopez.