Trouble with Personality Tests
Trouble with Personality Tests
The application of Total Quality Management Principles (6 Sigma) to commercial aviation has reduced the defect rate for take offs and landings around 3 parts per million. In contrast the figures quoted for defects in recruitment are in the region of 45%, because we continue to utilise tools that are ineffective. For example, the traditional unstructured interview is only marginally more effective than flipping a coin. The other tool much loved and widely used by managers and HR practitioners are personality tests which are the topic of this article.
Personality tests are a popular component of many or¬ganisations’ hiring processes. As these tests contend to measure traits and characteristics that remain stable over time, it is intuitive to believe information regarding candidates’ individual differences in these areas would be helpful when making selection decisions. Yet evi¬dence supporting the usefulness of personality tests in the hiring process has been called into serious ques¬tion, due to repeated findings that correlations between measures of personality and measures of job performance are not strongly related. After nearly two decades of enthusiastic support for the use of personality assessments, there has been a call for talent management professionals to re-evaluate the merits of these tests.
In a recent article presented by Human Resource Execu¬tive Online, Peter Capelli (2007) briefly reviews the history of using personality tests for hiring and promotion decisions. He remarks that the current popularity of this method is reminiscent of its use as a “best practice” in the 1950s, which he notes is curious given the fact that “by the early-1960s, the consensus among researchers was that personality was not a useful criterion for assessing individuals.” During the 1960s – 1980s “personal¬ity-based assessments ... largely disappeared from the lists of ‘best practices’ in human resources,” however, a resurgence of interest in, and use of, personality testing emerged in the 1990s. Yet the central issue that led to the disfavour of personality tests 40 years ago (i.e., the lack of predictive validity or extent to which the assessment relates to or predicts job performance) still remains an unresolved issue.
A panel of prominent personnel psychologists all former editors of top-tiered journals, recently collaborated on an article discussing the usefulness of personality tests in personnel selection. The one clear theme that emerged from their work was that the validities of personality measures are so low that using them for selecting employees should be questioned. Although research studies have demon¬strated statistically significant relations between some personality factors and certain areas of job performance, the practical significance, or overall usefulness, of these relations remain as weak as those reported 40 years ago. This finding led one author to question, “Why are we now suddenly looking at personality as a valid predic¬tor of job performance when the validities still haven’t changed and are still close to zero?”
While evidence suggesting that personality tests are not robust predictors of job success has been available for decades, the comments made by the panel of experts’ article drew a storm of criticism from other researchers in the field. In a recently published follow-up to the rebuttal articles, the panel underlines that its “fundamental purpose in writing these articles is to provide a sobering reminder about the low validities and other problems in using self-report personality tests for personnel selection.” It is noted that blind enthusiasm for the use of personality testing has stemmed from researchers and practitioners alike, ignoring the basic data demonstrating that personality assessments are poor predictors of job performance. This evidence might be overlooked because of the potential for lowered adverse impact and increased criterion variance explained by the use of personality tests. However, the author’s state that increases in the criterion variance explained has not been realised. In light of these problems, it is noted that Robert Guion’s comments from over 40 years ago still hold true today: “In view of the problems…one must question the wisdom of using personality as instruments of decision in employment procedures.”
Unfortunately, as Capelli asserts, “the least valid of the personality measures are the ones most employers are likely to use: published tests that individual candidates complete themselves.” The most popular personality tests being used for hiring purposes utilise broad-based approaches, such as the Big Five Personality traits and Emotional Intelli¬gence, but these have had limited success. For example, meta-analytic research has found that these tools account for less than 6% of variance in sales effectiveness. One of the reasons for this outcome could be that most personality tests are very broad in scope, whereas the areas of job performance are fairly narrow and spe¬cific. Researchers have posited that the specificity of a predictor (e.g., an assess¬ment measure) should match the speci¬ficity of a construct, or the area of job performance the predictor is designed to predict. It stands to reason that a test designed to predict specific and precise work behaviours and outcomes would predict those specific work behaviours and outcomes better than a test designed to reveal a general and broad sense of an individual’s personality.
Chally concurs with the central tenet of the works described above; however, some context is necessary to avoid the blanket conclusion that any assessment that measures individual differences is bad. Most criticisms apply directly to the broad-based person¬ality tests people in the employment assessment field call “off-the-shelf” measures. These assessments were designed to be general, apply to a wide range of situations (most were not specifically created for workplace application), and are not amenable to customisation. Such measures employ a “one-size-fits-all” approach, which (similar to clothing) does not provide a very good fit in most cases.
The best instruments measures narrow, job-related constructs rather than broad, person¬ality constructs. As opposed to developing a measure descriptive of personal charac¬teristics, the goal should be to develop a measure that best predicted job performance in specific areas. While researchers and practitioners later became interested in the relations among general measures of personality and job performance, the focus must be to predict success on the job. Chally has long championed research designed to measure the competencies, behaviours, traits, and temperaments that predict specific job behaviours. The criterion-related validation approach, which is the statistical demonstration of the relationship between scores on an assessment and the job performance of sample workers, continues to be at the core of Chally’s selection method.
Interestingly, when Morgeson et al., and their critics dis¬cussed ways to improve selection methods, they all agreed that one way to increase validity is to develop tests that keep in mind the outcome, criteria, and/or on-the-job behaviour the end-user wishes to predict. The shared opinion is that keeping these factors in mind will lead to increases in validity and improve one’s ability to defend the use of the test if challenged. This has been a central tenet at Chally since its inception.
The Predictive Power of Chally’s Selection System
The Chally Assessment was designed by taking an actuarial approach (or criterion-related approach) to predict job success, whereas the aim of most published personality measures is to perfectly represent a theory of personality. Researchers agree, regardless of whether they propose using “compound” or “narrow” scales, companies need to measure more than personality traits if they are concerned with predicting job performance. Chally focuses on the competencies, behaviours, and temperaments that predict actual job behaviour. As a result, the Chally Assessment consistently has greater predictive power than existing “off-the-shelf” published personality measures.
Members of Chally’s Centre for Scientific Innovation (C2SI) have published research in several leading ac¬ademic journals and regularly apply their findings in the company’s practice. C2SI’s research supports the conclusion that off-the-shelf approaches have limited success predicting actual job performance because off-the-shelf measures do not consider the possibility that jobs with surface similarities may require different competencies for success. For example, different sales roles require different skills and motivations for success9. Although Extraverts (outgoing people that like to be the center of atten¬tion) tend to make better retail salespeople, they actually perform worse in business-to-business sales10. Business-to-business salespeople focus on listening to the potential customer rather than dominating the conversation.
Chally’s utilization of criterion-related validation studies has led to reductions in turnover of up to 30% and increases in individual productivity of up to 35% in numerous organizations across most industries. Chally has developed a unique assessment based on literally hundreds of actuarial studies (i.e., the rigorous statistical methods used to assess risk in insurance and finance industries). Chally’s research and the research of others con¬sistently demonstrated that personality tests are not robust predictors of job success. Now, top researchers in the field are proclaiming this same conclusion.
How to Recognize a Personality Test
It is not always readily apparent that an assessment is a per¬sonality test designed to describe an individual rather than a work-related measure designed to predict on-the-job behaviours, outcomes, or criteria important to performance. There are three questions that should help one determine the type of assessment being presented.
1. Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour?
2. Was the measure designed for academic or business application?
3. What evidence exists to show how the measure can impact business results?
Most personality tests were designed to describe a theory/model, are academic in nature, and are not likely to impact business results. It is sometimes difficult to avoid personality tests as many have become better known by their acronym. These include the 16pf (16 Personality Factor; IPAT), 6 FPQ (6 Factor Personality Questionnaire; Sigma Assessment Systems), CPI (California Personality Inventory; Westburn Publishers), CPQ (Craft Personality Questionnaire; CraftSys¬tems/Previsor), DPS (Dynamic Personality Shift; Kenexa) HWPI (Harcourt Workplace Personality Inventory; Harcourt), HPI (Hogan Personality Inventory; Hogan Assessment Systems), IPIP (International Personality Item Pool; Oregon Research Institute), JPRF (Jackson Personality Research Form; Sigma As¬sessment Systems), MPQ (Manchester Personality Question¬naire; Hogrefe Ltd.), MMPI (Minnesota Multiphasic Personality Inventory; Pearson Assessments), NEO-PI-R (NEO Personality Inventory – Revised; Psychological Assessment Resources), OPI (Occupational Personality Inventory; Kenexa), OPQ (Oc¬cupational Personality Questionnaire; SHL), PIP (Personality Interview Profiler; PIP), RPQ (Rapid Personality Questionnaire; Kenexa), and WPI Select (Work Personality Index Select; Psy¬chometrics Publishing).
Even more difficult to avoid are those assessments that do not include “personality” in their name, yet are clearly re¬vealed to be personality tests in their accompanying litera¬ture and/or documentation. The following tests would fall into this second category: Caliper Profile11, CDR Character Assessment12, Conditional Reasoning Test of Aggression13, Devine Inventory/Select Best13, DiSC14, Employee Screening Questionnaire14, Gordon Personal Profile Inventory13, MBTI (Myers-Briggs Type Indicator)15, Personal Style In¬ventory (PSI, two different tests with same name from different publishers) 16, 17 and Profiles International Tests18 (ProfileXTTM, ProfileXTSalesTM, Profiles Performance Indi¬catorTM, Profiles Sales IndicatorTM, Customer Service ProfileTM, Pro¬fileEasyTM), Profilers Premium Placements Inc., Job-fit Assess¬ments (The Achiever, The Sales Achiever, The Guardian, The Per¬former, The Scoreboard). 19
Perhaps the most generally effec¬tive way to identify a personality test is to review the output of the measure. If the assessment produces a description of per¬sonality traits, then it can reason¬ably be considered a personality test. Some common personality traits assessed are: Ego, Ego Strength, Ego Resilience, Empathy, Empathetic Outlook, or the Big Five personality traits, namely, Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscien¬tiousness (e.g., DiSC = Dominance, Influence, Steadiness, Conscientiousness). Many assessments disguise the Big Five personality traits by using variations in the trait names such as Emotional Stability, Emotional Control, Sociability, Introversion, Openness, Intellectance, Cau¬tiousness, Dependability, or Responsibility. A reader familiar with a feedback report for the Caliper Profile, the Hogan, or the PreVisor Assessment likely recognized some of these name variations. Likewise, these name variations are common to many assessments that claim to measure predictors of job performance but are actu¬ally measuring personality traits.
Remember these questions: “Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour? Was the measure designed for academic or business application? What evidence exists to show the measure can impact business results?”
Keep in mind the Chally Assessment was specifically designed to predict success in a business environment and that plenty of case studies and testimonials are available to demonstrate how this approach has lead to great success for our clients.
Trouble with Personality Tests - To learn more about this author, visit Peter Gilbert's Website.
Like this article? Share it with your friends
The Trouble with Personality Tests
The application of Total Quality Management Principles (6 Sigma) to commercial aviation has reduced the defect rate for take offs and landings around 3 parts per million. In contrast the figures quoted for defects in recruitment are in the region of 45%, because we continue to utilise tools that are ineffective. For example, the traditional unstructured interview is only marginally more effective than flipping a coin. The other tool much loved and widely used by managers and HR practitioners are personality tests which are the topic of this article.
Personality tests are a popular component of many or¬ganisations’ hiring processes. As these tests contend to measure traits and characteristics that remain stable over time, it is intuitive to believe information regarding candidates’ individual differences in these areas would be helpful when making selection decisions. Yet evi¬dence supporting the usefulness of personality tests in the hiring process has been called into serious ques¬tion, due to repeated findings that correlations between measures of personality and measures of job performance are not strongly related. After nearly two decades of enthusiastic support for the use of personality assessments, there has been a call for talent management professionals to re-evaluate the merits of these tests.
In a recent article presented by Human Resource Execu¬tive Online, Peter Capelli (2007) briefly reviews the history of using personality tests for hiring and promotion decisions. He remarks that the current popularity of this method is reminiscent of its use as a “best practice” in the 1950s, which he notes is curious given the fact that “by the early-1960s, the consensus among researchers was that personality was not a useful criterion for assessing individuals.” During the 1960s – 1980s “personal¬ity-based assessments ... largely disappeared from the lists of ‘best practices’ in human resources,” however, a resurgence of interest in, and use of, personality testing emerged in the 1990s. Yet the central issue that led to the disfavour of personality tests 40 years ago (i.e., the lack of predictive validity or extent to which the assessment relates to or predicts job performance) still remains an unresolved issue.
A panel of prominent personnel psychologists all former editors of top-tiered journals, recently collaborated on an article discussing the usefulness of personality tests in personnel selection. The one clear theme that emerged from their work was that the validities of personality measures are so low that using them for selecting employees should be questioned. Although research studies have demon¬strated statistically significant relations between some personality factors and certain areas of job performance, the practical significance, or overall usefulness, of these relations remain as weak as those reported 40 years ago. This finding led one author to question, “Why are we now suddenly looking at personality as a valid predic¬tor of job performance when the validities still haven’t changed and are still close to zero?”
While evidence suggesting that personality tests are not robust predictors of job success has been available for decades, the comments made by the panel of experts’ article drew a storm of criticism from other researchers in the field. In a recently published follow-up to the rebuttal articles, the panel underlines that its “fundamental purpose in writing these articles is to provide a sobering reminder about the low validities and other problems in using self-report personality tests for personnel selection.” It is noted that blind enthusiasm for the use of personality testing has stemmed from researchers and practitioners alike, ignoring the basic data demonstrating that personality assessments are poor predictors of job performance. This evidence might be overlooked because of the potential for lowered adverse impact and increased criterion variance explained by the use of personality tests. However, the author’s state that increases in the criterion variance explained has not been realised. In light of these problems, it is noted that Robert Guion’s comments from over 40 years ago still hold true today: “In view of the problems…one must question the wisdom of using personality as instruments of decision in employment procedures.”
Unfortunately, as Capelli asserts, “the least valid of the personality measures are the ones most employers are likely to use: published tests that individual candidates complete themselves.” The most popular personality tests being used for hiring purposes utilise broad-based approaches, such as the Big Five Personality traits and Emotional Intelli¬gence, but these have had limited success. For example, meta-analytic research has found that these tools account for less than 6% of variance in sales effectiveness. One of the reasons for this outcome could be that most personality tests are very broad in scope, whereas the areas of job performance are fairly narrow and spe¬cific. Researchers have posited that the specificity of a predictor (e.g., an assess¬ment measure) should match the speci¬ficity of a construct, or the area of job performance the predictor is designed to predict. It stands to reason that a test designed to predict specific and precise work behaviours and outcomes would predict those specific work behaviours and outcomes better than a test designed to reveal a general and broad sense of an individual’s personality.
Chally concurs with the central tenet of the works described above; however, some context is necessary to avoid the blanket conclusion that any assessment that measures individual differences is bad. Most criticisms apply directly to the broad-based person¬ality tests people in the employment assessment field call “off-the-shelf” measures. These assessments were designed to be general, apply to a wide range of situations (most were not specifically created for workplace application), and are not amenable to customisation. Such measures employ a “one-size-fits-all” approach, which (similar to clothing) does not provide a very good fit in most cases.
The best instruments measures narrow, job-related constructs rather than broad, person¬ality constructs. As opposed to developing a measure descriptive of personal charac¬teristics, the goal should be to develop a measure that best predicted job performance in specific areas. While researchers and practitioners later became interested in the relations among general measures of personality and job performance, the focus must be to predict success on the job. Chally has long championed research designed to measure the competencies, behaviours, traits, and temperaments that predict specific job behaviours. The criterion-related validation approach, which is the statistical demonstration of the relationship between scores on an assessment and the job performance of sample workers, continues to be at the core of Chally’s selection method.
Interestingly, when Morgeson et al., and their critics dis¬cussed ways to improve selection methods, they all agreed that one way to increase validity is to develop tests that keep in mind the outcome, criteria, and/or on-the-job behaviour the end-user wishes to predict. The shared opinion is that keeping these factors in mind will lead to increases in validity and improve one’s ability to defend the use of the test if challenged. This has been a central tenet at Chally since its inception.
The Predictive Power of Chally’s Selection System
The Chally Assessment was designed by taking an actuarial approach (or criterion-related approach) to predict job success, whereas the aim of most published personality measures is to perfectly represent a theory of personality. Researchers agree, regardless of whether they propose using “compound” or “narrow” scales, companies need to measure more than personality traits if they are concerned with predicting job performance. Chally focuses on the competencies, behaviours, and temperaments that predict actual job behaviour. As a result, the Chally Assessment consistently has greater predictive power than existing “off-the-shelf” published personality measures.
Members of Chally’s Centre for Scientific Innovation (C2SI) have published research in several leading ac¬ademic journals and regularly apply their findings in the company’s practice. C2SI’s research supports the conclusion that off-the-shelf approaches have limited success predicting actual job performance because off-the-shelf measures do not consider the possibility that jobs with surface similarities may require different competencies for success. For example, different sales roles require different skills and motivations for success9. Although Extraverts (outgoing people that like to be the center of atten¬tion) tend to make better retail salespeople, they actually perform worse in business-to-business sales10. Business-to-business salespeople focus on listening to the potential customer rather than dominating the conversation.
Chally’s utilization of criterion-related validation studies has led to reductions in turnover of up to 30% and increases in individual productivity of up to 35% in numerous organizations across most industries. Chally has developed a unique assessment based on literally hundreds of actuarial studies (i.e., the rigorous statistical methods used to assess risk in insurance and finance industries). Chally’s research and the research of others con¬sistently demonstrated that personality tests are not robust predictors of job success. Now, top researchers in the field are proclaiming this same conclusion.
How to Recognize a Personality Test
It is not always readily apparent that an assessment is a per¬sonality test designed to describe an individual rather than a work-related measure designed to predict on-the-job behaviours, outcomes, or criteria important to performance. There are three questions that should help one determine the type of assessment being presented.
1. Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour?
2. Was the measure designed for academic or business application?
3. What evidence exists to show how the measure can impact business results?
Most personality tests were designed to describe a theory/model, are academic in nature, and are not likely to impact business results. It is sometimes difficult to avoid personality tests as many have become better known by their acronym. These include the 16pf (16 Personality Factor; IPAT), 6 FPQ (6 Factor Personality Questionnaire; Sigma Assessment Systems), CPI (California Personality Inventory; Westburn Publishers), CPQ (Craft Personality Questionnaire; CraftSys¬tems/Previsor), DPS (Dynamic Personality Shift; Kenexa) HWPI (Harcourt Workplace Personality Inventory; Harcourt), HPI (Hogan Personality Inventory; Hogan Assessment Systems), IPIP (International Personality Item Pool; Oregon Research Institute), JPRF (Jackson Personality Research Form; Sigma As¬sessment Systems), MPQ (Manchester Personality Question¬naire; Hogrefe Ltd.), MMPI (Minnesota Multiphasic Personality Inventory; Pearson Assessments), NEO-PI-R (NEO Personality Inventory – Revised; Psychological Assessment Resources), OPI (Occupational Personality Inventory; Kenexa), OPQ (Oc¬cupational Personality Questionnaire; SHL), PIP (Personality Interview Profiler; PIP), RPQ (Rapid Personality Questionnaire; Kenexa), and WPI Select (Work Personality Index Select; Psy¬chometrics Publishing).
Even more difficult to avoid are those assessments that do not include “personality” in their name, yet are clearly re¬vealed to be personality tests in their accompanying litera¬ture and/or documentation. The following tests would fall into this second category: Caliper Profile11, CDR Character Assessment12, Conditional Reasoning Test of Aggression13, Devine Inventory/Select Best13, DiSC14, Employee Screening Questionnaire14, Gordon Personal Profile Inventory13, MBTI (Myers-Briggs Type Indicator)15, Personal Style In¬ventory (PSI, two different tests with same name from different publishers) 16, 17 and Profiles International Tests18 (ProfileXTTM, ProfileXTSalesTM, Profiles Performance Indi¬catorTM, Profiles Sales IndicatorTM, Customer Service ProfileTM, Pro¬fileEasyTM), Profilers Premium Placements Inc., Job-fit Assess¬ments (The Achiever, The Sales Achiever, The Guardian, The Per¬former, The Scoreboard). 19
Perhaps the most generally effec¬tive way to identify a personality test is to review the output of the measure. If the assessment produces a description of per¬sonality traits, then it can reason¬ably be considered a personality test. Some common personality traits assessed are: Ego, Ego Strength, Ego Resilience, Empathy, Empathetic Outlook, or the Big Five personality traits, namely, Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscien¬tiousness (e.g., DiSC = Dominance, Influence, Steadiness, Conscientiousness). Many assessments disguise the Big Five personality traits by using variations in the trait names such as Emotional Stability, Emotional Control, Sociability, Introversion, Openness, Intellectance, Cau¬tiousness, Dependability, or Responsibility. A reader familiar with a feedback report for the Caliper Profile, the Hogan, or the PreVisor Assessment likely recognized some of these name variations. Likewise, these name variations are common to many assessments that claim to measure predictors of job performance but are actu¬ally measuring personality traits.
Remember these questions: “Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour? Was the measure designed for academic or business application? What evidence exists to show the measure can impact business results?”
Keep in mind the Chally Assessment was specifically designed to predict success in a business environment and that plenty of case studies and testimonials are available to demonstrate how this approach has lead to great success for our clients.
Trouble with Personality Tests - To learn more about this author, visit Peter Gilbert's Website.
Like this article? Share it with your friends
 | |
 |
I was only looking for an explanation of what PI.COM personality tests were, however this went further in helping me question not just what they are but if they are even useful as a class of HR filters.
The content is clear and simple to understand.
Commented on Trouble with Personality Tests. |
| |
Leave Your Feedback |
 | |
| |||
Stephanie RobeyStephanie Robey is President and CoFounder of Pivot Positive, LLC - an Internet marketing business focused on helping people start work at home ventures. Previously, she was employed at The Search Agency with over 20 years experience in graphic design and 10 years experience in online marketing. She was responsible for launching the Conversion Path Optimization (CPO) unit where she and her team have conducted hundreds of optimization tests for online companies across multiple verticals. She is a successful entrepreneur having started and sold 2 companies and remains on the board of directors of the third, PhotoSpin.com Stephanie began her career in the direct marketing realm creating and producing direct mail for many of the major cable television companies and directly attributes her understanding of Internet marketing to those early offline experiences. Stephanie is a graduate of San Diego State University with a BFA in Graphic Arts and also holds an Executive MBA from the Graziadio School of Business and Management at Pepperdine University. Read Steph's Blog Meet Steph and Dave Sign up for our Free 7-Day BootCamp: Self Employed & Rich - Visit Stephanie Robey's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
|  |
Subscribe to Peter's articles
 | |
|
|  |
 | |
|
| |
 | |||||||
|
 | ||
 |
||
 |
||
 |
||
|
| ||
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
"Learn straight from Evan how you can Make a Full Time Income (And More) from a Website"
Click Here To Learn More |
|
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
|
 Make It Happen Innovative Thinking Create A Mission Statement Business Coaching Hiring An SEO Expert |
|
 | ||
 |
Top 50 HR Blogs 2009
Top 50 HR Blogs 2009 | |
 |
Top 50 Geek Business Blogs
Top 50 Geek Business Blogs | |
| ||
 | ||||
| ||||
|
| ||||
| ||||
|
|
|
|
|
||||||||||||
|
Email us your ideas on how to make our website more valuable! Thank you Sharon from Toronto Salsa Lessons / Classes for your suggestions to make the newsletter look like the website and profile younger entrepreneurs like Jennifer Lopez and Sean Combs!
|
|
|
