Measuring Training Effectiveness: How to Get Started
Measuring Training Effectiveness: How to Get Started
You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.
Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.
Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.
Level 4 – Results What organizational benefits resulted from the training?
^
Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?
^
Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
^
Level 1 – Reaction How did participants react to the program?
The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).
In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.
For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.
Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.
Using the Kirkpatrick Model ............................................................................
So, how do you conduct an evaluation? The basic steps are:
1. Design the evaluation.
This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.
2. Collect the data.
Here, you conduct the survey and focus group sessions and collect operational and business performance data.
3. Analyze the data.
Analysis entails converting the raw data into useful information on which you can make evaluative judgments.
4. Report the results.
In this final step, write and distribute the report and debrief client managers and other interested stakeholders.
In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.
Level 1 (Reaction) .....................................................................................
· completed participant feedback questionnaire .......................................................
· informal comments from participants ................................................................
· focus group sessions with participants ..............................................................
Level 2 (Learning) .......................................................................................
· pre- and post-test scores ...........................................................................
· on-the-job assessments .............................................................................
· supervisor reports .................................................................................
Level 3 (Behavior) ......................................................................................
· completed self-assessment questionnaire ............................................................
· on-the-job observation .............................................................................
· reports from customers, peers and participant’s manager .............................................
Level 4 (Results) .....................................................................................
· financial reports ...................................................................................
· quality inspections .................................................................................
· interview with sales manager ...........................................................................
When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?
Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!
References ............................................................................................
Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.
Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.
2006 © Business Performance Pty Ltd. All rights reserved.
Measuring Training Effectiveness How to Get Started - To learn more about this author, visit Vicki Heath's Website.
Like this article? Share it with your friends
Why Measure Training Effectiveness?
You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.
Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.
Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.
Level 4 – Results What organizational benefits resulted from the training?
^
Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?
^
Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
^
Level 1 – Reaction How did participants react to the program?
The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).
In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.
For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.
Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.
Using the Kirkpatrick Model ............................................................................
So, how do you conduct an evaluation? The basic steps are:
1. Design the evaluation.
This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.
2. Collect the data.
Here, you conduct the survey and focus group sessions and collect operational and business performance data.
3. Analyze the data.
Analysis entails converting the raw data into useful information on which you can make evaluative judgments.
4. Report the results.
In this final step, write and distribute the report and debrief client managers and other interested stakeholders.
In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.
Level 1 (Reaction) .....................................................................................
· completed participant feedback questionnaire .......................................................
· informal comments from participants ................................................................
· focus group sessions with participants ..............................................................
Level 2 (Learning) .......................................................................................
· pre- and post-test scores ...........................................................................
· on-the-job assessments .............................................................................
· supervisor reports .................................................................................
Level 3 (Behavior) ......................................................................................
· completed self-assessment questionnaire ............................................................
· on-the-job observation .............................................................................
· reports from customers, peers and participant’s manager .............................................
Level 4 (Results) .....................................................................................
· financial reports ...................................................................................
· quality inspections .................................................................................
· interview with sales manager ...........................................................................
When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?
Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!
References ............................................................................................
Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.
Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.
2006 © Business Performance Pty Ltd. All rights reserved.
Measuring Training Effectiveness How to Get Started - To learn more about this author, visit Vicki Heath's Website.
Like this article? Share it with your friends
| |||
 |
No article feedback found. | ||
|
Leave Your Feedback | ||
|
|||
 An Integrated Training Assessment Program
| |||
| An integrated assessment program also provides insight and development goals for the instructor and course design. Assessment is a diagnostic tool that ultimately helps the learners confirm what they have learned an... |
|||
|
ROI: Return on Investment or Return on Illusion?
| |||
| Do we really know what makes up a true return on investment? And more importantly, do we know the cost/benefit ratio well enough to really ignore understanding this elusive construct. Read here for the reality about... |
|||
|
Measuring Return On Investment For Soft Skill Training
| |||
| This article helps take the fear out of investing in soft skill training and outlines an effective way to measure the success of such training. |
|||
|
How effective is your Advertising?
| |||
| Have you ever heard of Guerrilla Marketing? It’s an effective way of marketing execution for mostly small businesses but its techniques can be applied in huge corporations as well.
|
|||
|
FREE Salesforce Grader Tool
| |||
| It's free and it provides you with a score on the relative effectiveness of your salesforce. |
|||
| |||
Linda RichardsonLinda Richardson is the Founder and Executive Chairwoman of Richardson, a global sales training and performance improvement company. As a recognized leader in the industry, she has won the coveted Stevie Award for Lifetime Achievement in Sales Excellence and she was identified by Training Industry, Inc. as one of the “Top 20 Most Influential Training Professionals.” Ms. Richardson is credited with the movement to Consultative Selling and is the author of ten books on selling and sales management, including Sales Coaching — Making the Great Leap from Sales Manager to Sales Coach, and Stop Telling, Start Selling. She teaches sales and management at the Wharton Graduate School of the University of Pennsylvania and the Wharton Executive Development Center. Linda is a frequent speaker at industry and client conferences, has been published extensively in industry and training journals, and has been featured in numerous publications, including The Wall Street Journal, Forbes, Nation’s Business, Selling Power, Success, and The Conference Board Magazine. Learn more about Richardson's sales training and performance improvement solutions at http://www.richardson.com web - Visit Linda Richardson's Website |
|||
Dr. John OdaJohn Oda Ph.D NLP is a business peak performance expert, an author, and speaker frequently called upon to provide corporate training, workshops and seminars for many companies in the United States. He is an expert in coaching sales and business professionals in overcoming the behaviors and obstacles that may impede their sales results and affect their bottom line. Since 1995, John has created a speaking bureau such topics, which include: time management, sales training, human diversity, leadership programs and etc. He provides companies with a strategic plan to increase their bottom line by over 25 percent yearly. - Visit Dr. John Oda's Website |
|||
George LudwigGeorge Ludwig is a recognized authority on sales strategy and peak performance psychology. An international speaker, trainer, and corporate consultant, he helps clients like Johnson & Johnson, Abbott Laboratories, Northwestern Mutual, CIGNA, and numerous others improve sales force effectiveness and performance. Though it's George's strategies and processes that help corporations increase productivity and performance, it's his tremendous energy and dynamism that spark the transformation. Again and again, clients remark on his amazing ability to unleash human capacity and inspire men and women to break out of their comfort zones. The result is a whole new type of salesperson. His customized presentations teach achievers to make stunning advances in their lives. From helping salespeople realize cherished dreams to helping corporations exponentially accelerate revenue streams, George Ludwig leaves audiences and individuals empowered, emboldened, and clamoring for more. George is the best-selling author of Power Selling: Seven Strategies for Cracking the Sales Code and Wise Moves: 60 Quick Tips to Improve Your Position in Life & Business. - Visit George Ludwig's Website |
|||
Kim CastleWith nearly two decades in the advertising and design business, with clients like Domino's Pizza, General Motors, Direct TV, Pedigree, Wolfgang Puck, Higher Octave Music, Hollywood Celebrity Products, Disney, and Paramount, as well as thousands of entrepreneurs around the world define, structure, communicate, and position their business for greater profits, BrandU(R) co-creators Kim Castle and W. Vito Montone discovered that entrepreneurs could experience the same power that big brands command for a fraction of the cost with the world's only process-based results-drive Integral approach to business creation. BrandU(R) is helping entrepreneurs grow with the power of extreme clarity from idea...to brand...to market(TM) and helping one million entrepreneurs become successful and whole so that they can make a difference in the world. Are you one of them? If you want to experience clarity all the way to the bank(TM), get started now at http://www.brandu.com. - Visit Kim Castle's Website |
|||
Staging DivaDebra Gould, aka The Staging Diva®, is President of Six Elements Inc., an internationally recognized home staging company. Inspired by many requests from aspiring home stagers wanting to start similar businesses, Gould created the Staging Diva Home Staging Business Training Program. Gould has trained over 1000 Staging Diva Graduates worldwide to start staging businesses. Buying decorating and selling six of her own homes in four years lead to an interest in real estate staging which she turned into a career with the launch of sixelements.com in 2002. Since then she has staged hundreds of homes in addition to teaching home staging training. Gould is the author of several home staging resources including a series of popular ebooks made up of a Design Guide, Color Guide and Portfolio Guide. For more information about Debra Gould visit stagingdiva.com. - Visit Staging Diva's Website |
|||
Leanne Hoagland-SmithAre your sales where you want them to be? Will you be one of the few who achieves sales or business success or one of the many who have failed to change? Are you tired of being told you are like everyone else? Then you may find my first book on sales of interest. Be the Red Jacket in the Sea of Gray Suits, The Keys to Unlocking Sales available at Amazon or at http://www.processspecialist.com/red-jacket.htm. This book is a reflection of my no-nonsense approach to improving sales to overall business results. If you are truly committed to making sustainable changes, then I can help you secure a positive return on your investment because I focus on executable solutions not telling you the problems you already know you have. From training to corporate (group) coaching to executive one on one coaching, my approach is to assess, create awareness, build a goal driven action plan and then execute. The bottom line question is "Not do you or your employees know it, but do you or they want to do it?" Please call for a free strategy session at 219.759.5601. - Visit Leanne Hoagland-Smith's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
 Vicki Heath (Visit Vicki's Website) Vicki Heath is the Director of Business Performance Pty Ltd, a company providing practical online information and resources in a range of business areas. Her company’s guides, tools and templates assist organizations engage and develop people, manage organizational change and improve project delivery. Visit her at www.busine ssperform.com for free downloadable templates.
|  |
 |
|
|
 |
|
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
 |
 | ||
 |
||
|
||
|
||
|
| ||
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
|
 |
 Establish Self-Beliefs Sample Credit Application Become a Better Closer! Balanced Leadership Balance chart |
|
 | ||
 |
Top 50 Marketing Blogs
Top Blogs To Watch In 2008 | |
 |
Top 50 Diversion Blogs
Top Diversion Blogs of 2008 | |
| ||
 |
|
|
|
|
|||||||||||||||||||||
 |
|
|
||||||||||||
