Measuring Training Effectiveness: How to Get Started
Measuring Training Effectiveness: How to Get Started
You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.
Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.
Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.
Level 4 – Results What organizational benefits resulted from the training?
^
Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?
^
Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
^
Level 1 – Reaction How did participants react to the program?
The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).
In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.
For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.
Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.
Using the Kirkpatrick Model ............................................................................
So, how do you conduct an evaluation? The basic steps are:
1. Design the evaluation.
This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.
2. Collect the data.
Here, you conduct the survey and focus group sessions and collect operational and business performance data.
3. Analyze the data.
Analysis entails converting the raw data into useful information on which you can make evaluative judgments.
4. Report the results.
In this final step, write and distribute the report and debrief client managers and other interested stakeholders.
In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.
Level 1 (Reaction) .....................................................................................
· completed participant feedback questionnaire .......................................................
· informal comments from participants ................................................................
· focus group sessions with participants ..............................................................
Level 2 (Learning) .......................................................................................
· pre- and post-test scores ...........................................................................
· on-the-job assessments .............................................................................
· supervisor reports .................................................................................
Level 3 (Behavior) ......................................................................................
· completed self-assessment questionnaire ............................................................
· on-the-job observation .............................................................................
· reports from customers, peers and participant’s manager .............................................
Level 4 (Results) .....................................................................................
· financial reports ...................................................................................
· quality inspections .................................................................................
· interview with sales manager ...........................................................................
When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?
Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!
References ............................................................................................
Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.
Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.
2006 © Business Performance Pty Ltd. All rights reserved.
Measuring Training Effectiveness How to Get Started - To learn more about this author, visit Vicki Heath's Website.
Like this article? Share it with your friends
Why Measure Training Effectiveness?
You may have been asked by your manager to start to measure the effectiveness of the training programs you provide. Training resources may be shrinking as your client managers complain more often and more loudly that they have not seen any benefits from having their staff away on training. Many training programs today fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers.
Conversely, you may have decided yourself that you need to go beyond your usual “smile sheets”. External pressures may be leading you to think about improving your current programs. The business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances.
Measuring the effectiveness of training programs, however, consumes valuable time and resources – time and resources that are already in short supply. You will need to think carefully about how and to what extent you will evaluate the results of training. Donald Kirkpatrick’s four-level evaluation model remains as the most well-known and used model today. Kirkpatrick developed his model in the late 1950s and the model has since been adapted and modified by a number of writers. However, the model’s basic structure has well stood the test of time and I continue to recommend it. The basic structure of Kirkpatrick’s four-level model is shown below.
Level 4 – Results What organizational benefits resulted from the training?
^
Level 3 – Behavior To what extent did participants change their behavior back in the workplace as a result of the training?
^
Level 2 – Learning To what extent did participants improve knowledge and skills and change attitudes as a result of the training?
^
Level 1 – Reaction How did participants react to the program?
The primary purpose of conducting an evaluation at a particular level is to answer the question posed at that level. Conducting an evaluation at one level is not meant to be better or more useful than conducting an evaluation at another level – it just provides different information. The levels are related, though, as each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2).
In deciding at which levels to pitch your evaluations, you will need to think about an appropriate combination that will suit your organization’s specific needs and available resources. As you go up the levels, generally speaking, the cost and time required for the evaluation rises sharply. So, you will need to choose wisely.
For example, you may decide to conduct Level 1 evaluations for all programs and Level 2 for skill certification programs only. Because of the cost and effort involved, you may leave Level 3 and Level 4 evaluations for programs of high strategic or operational importance, such as project management training.
Above all else, think specifically about why you are performing a particular evaluation - and write it down. This will help you focus on what’s important when resources get constrained or when someone comes up with a “great idea” that will require a lot of work.
Using the Kirkpatrick Model ............................................................................
So, how do you conduct an evaluation? The basic steps are:
1. Design the evaluation.
This first step involves designing survey questionnaires, formulas and spreadsheets for data entry.
2. Collect the data.
Here, you conduct the survey and focus group sessions and collect operational and business performance data.
3. Analyze the data.
Analysis entails converting the raw data into useful information on which you can make evaluative judgments.
4. Report the results.
In this final step, write and distribute the report and debrief client managers and other interested stakeholders.
In designing your evaluation, you will need to think about your data sources. Where should you get your data? Here are some ideas on appropriate data sources for each level.
Level 1 (Reaction) .....................................................................................
· completed participant feedback questionnaire .......................................................
· informal comments from participants ................................................................
· focus group sessions with participants ..............................................................
Level 2 (Learning) .......................................................................................
· pre- and post-test scores ...........................................................................
· on-the-job assessments .............................................................................
· supervisor reports .................................................................................
Level 3 (Behavior) ......................................................................................
· completed self-assessment questionnaire ............................................................
· on-the-job observation .............................................................................
· reports from customers, peers and participant’s manager .............................................
Level 4 (Results) .....................................................................................
· financial reports ...................................................................................
· quality inspections .................................................................................
· interview with sales manager ...........................................................................
When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information?
Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them!
References ............................................................................................
Kirkpatrick, D. L. (1959) Evaluating Training Programs, 2nd ed., Berrett Koehler, San Francisco.
Kirkpatrick, D. L. (comp.) (1998) Another Look at Evaluating Training Programs, ASTD, Alexandria, USA.
2006 © Business Performance Pty Ltd. All rights reserved.
Measuring Training Effectiveness How to Get Started - To learn more about this author, visit Vicki Heath's Website.
Like this article? Share it with your friends
 | |
 |
No article feedback found. |
| |
Leave Your Feedback |
 | |
| |||
John BrennanJohn Brennan Ed.D. Dr. Brennan is President of Interpersonal Development, LLC, a training and development firm. Interpersonal Development has provided sales training and coaching to more than 3,000 sales reps from over 100 companies. A native of Australia, Dr. Brennan received his doctorate from the University of Rochester. His dissertation researched the effectiveness of Behavioral Modeling Technology in training people in interpersonal skills. While he has spent most of his career designing or delivering training, he was also a Vice-President of Sales of a training and development franchise with operations in 25 markets. Dr. Brennan has designed and delivered sales training in North America, Asia, Europe, Australia and the Middle East. He has been a guest speaker at numerous national and regional professional conferences. When Microsoft wanted Best Practices articles on sales for their web site, they called Dr. Brennan. The results are at http://office.microsoft.com/en-us/FX011387391033.aspx His firm’s clients have included Volvo, The Prudential, Merrill Lynch, Eastman Kodak, Gannett, Equifax Europe, the Economist Group and countless small businesses. - Visit John Brennan's Website |
|||
Linda RichardsonLinda Richardson is the Founder and Executive Chairwoman of Richardson, a global sales training and performance improvement company. As a recognized leader in the industry, she has won the coveted Stevie Award for Lifetime Achievement in Sales Excellence and she was identified by Training Industry, Inc. as one of the “Top 20 Most Influential Training Professionals.” Ms. Richardson is credited with the movement to Consultative Selling and is the author of ten books on selling and sales management, including Sales Coaching — Making the Great Leap from Sales Manager to Sales Coach, and Stop Telling, Start Selling. She teaches sales and management at the Wharton Graduate School of the University of Pennsylvania and the Wharton Executive Development Center. Linda is a frequent speaker at industry and client conferences, has been published extensively in industry and training journals, and has been featured in numerous publications, including The Wall Street Journal, Forbes, Nation’s Business, Selling Power, Success, and The Conference Board Magazine. Learn more about Richardson's sales training and performance improvement solutions at http://www.richardson.com web - Visit Linda Richardson's Website |
|||
David AchesonDavid Acheson is the founder of DCJA Consultancy. DCJA Consultancy is a management consultancy business specialising in B2B sales consultancy. They offer bespoke and packaged sales consultancy including Sales Optimisation Review, Interim Sales Management, Sales & Marketing Review, 1:1 Sales & Management Staff Analysis, Management Training, Solution Sales Training, Creation of New Pay Plan, KPI's, run Customer Feedback Campaigns, assist with Recruitment, Coaching, Appraisals and set up Strategic Marketing Campaigns. David spent his early career in accountancy and then moved into sales in 1982, working in Office Equipment, IT, Advertising, Training, Outsourcing and Consultancy. He has held many Senior Positions in SMBs and Global Organisations including Head of Sales Operations & Head of Business Development. His knowledge, skills and great experience of the Sales Industry has led to David making keynote speeches and running educational sessions to key businesses through organisations including The Chamber of Commerce and Business Link. - Visit David Acheson's Website |
|||
George LudwigGeorge Ludwig is a recognized authority on sales strategy and peak performance psychology. An international speaker, trainer, and corporate consultant, he helps clients like Johnson & Johnson, Abbott Laboratories, Northwestern Mutual, CIGNA, and numerous others improve sales force effectiveness and performance. Though it's George's strategies and processes that help corporations increase productivity and performance, it's his tremendous energy and dynamism that spark the transformation. Again and again, clients remark on his amazing ability to unleash human capacity and inspire men and women to break out of their comfort zones. The result is a whole new type of salesperson. His customized presentations teach achievers to make stunning advances in their lives. From helping salespeople realize cherished dreams to helping corporations exponentially accelerate revenue streams, George Ludwig leaves audiences and individuals empowered, emboldened, and clamoring for more. George is the best-selling author of Power Selling: Seven Strategies for Cracking the Sales Code and Wise Moves: 60 Quick Tips to Improve Your Position in Life & Business. - Visit George Ludwig's Website |
|||
John AlexanderJohn has taught keyword research and SEO skills to small groups of business owners and Webmasters from over 80 different countries world wide since 2002. John is also the Director of Search Engine Academy ; Co-director of Training at Search Engine Workshops offering live, SEO Workshops with his partner SEO educator Robin Nobles, author of the very first comprehensive online search engine marketing courses at SEO Training Online and the SEO Workshop Resource Center. I look forward to hearing from you! - Visit John Alexander's Website |
|||
Kim CastleWith nearly two decades in the advertising and design business, with clients like Domino's Pizza, General Motors, Direct TV, Pedigree, Wolfgang Puck, Higher Octave Music, Hollywood Celebrity Products, Disney, and Paramount, as well as thousands of entrepreneurs around the world define, structure, communicate, and position their business for greater profits, BrandU(R) co-creators Kim Castle and W. Vito Montone discovered that entrepreneurs could experience the same power that big brands command for a fraction of the cost with the world's only process-based results-drive Integral approach to business creation. BrandU(R) is helping entrepreneurs grow with the power of extreme clarity from idea...to brand...to market(TM) and helping one million entrepreneurs become successful and whole so that they can make a difference in the world. Are you one of them? If you want to experience clarity all the way to the bank(TM), get started now at http://www.brandu.com. - Visit Kim Castle's Website |
|||
Staging DivaDebra Gould, aka The Staging Diva®, is President of Six Elements Inc., an internationally recognized home staging company. Inspired by many requests from aspiring home stagers wanting to start similar businesses, Gould created the Staging Diva Home Staging Business Training Program. Gould has trained over 1000 Staging Diva Graduates worldwide to start staging businesses. Buying decorating and selling six of her own homes in four years lead to an interest in real estate staging which she turned into a career with the launch of sixelements.com in 2002. Since then she has staged hundreds of homes in addition to teaching home staging training. Gould is the author of several home staging resources including a series of popular ebooks made up of a Design Guide, Color Guide and Portfolio Guide. For more information about Debra Gould visit stagingdiva.com. - Visit Staging Diva's Website |
|||
Stephanie RobeyStephanie Robey is President and CoFounder of Pivot Positive, LLC - an Internet marketing business focused on helping people start work at home ventures. Previously, she was employed at The Search Agency with over 20 years experience in graphic design and 10 years experience in online marketing. She was responsible for launching the Conversion Path Optimization (CPO) unit where she and her team have conducted hundreds of optimization tests for online companies across multiple verticals. She is a successful entrepreneur having started and sold 2 companies and remains on the board of directors of the third, PhotoSpin.com Stephanie began her career in the direct marketing realm creating and producing direct mail for many of the major cable television companies and directly attributes her understanding of Internet marketing to those early offline experiences. Stephanie is a graduate of San Diego State University with a BFA in Graphic Arts and also holds an Executive MBA from the Graziadio School of Business and Management at Pepperdine University. Read Steph's Blog Meet Steph and Dave Sign up for our Free 7-Day BootCamp: Self Employed & Rich - Visit Stephanie Robey's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
|  |
Subscribe to Vicki's articles
 | |
|
|  |
 | |
|
| |
 | ||
|
||
 |
||
 |
||
|
| ||
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
"Learn straight from Evan how you can Make a Full Time Income (And More) from a Website"
Click Here To Learn More |
|
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
 |
 Aromatic Networking Authentic Leadership Optimus Performance Million $ Interview Benefit From Analytics |
|
 | ||
 |
Top 50 Productivity Blogs
Top Blogs To Watch In 2008 | |
 |
Top 50 Diversion Blogs
Top Diversion Blogs of 2009 | |
| ||
 | ||||
| ||||
|
| ||||
| ||||
|
|
|
|
|
||||||||||||
|
Email us your ideas on how to make our website more valuable! Thank you Sharon from Toronto Salsa Lessons / Classes for your suggestions to make the newsletter look like the website and profile younger entrepreneurs like Jennifer Lopez and Sean Combs!
|
|
|
| ||||||||||||
| ||||||||||||
|
|
||||||||||||