Tweet

Trouble with Personality Tests

Written by: Peter Gilbert

Article Overview: Personality tests are commonly used for hiring in organisations. Personality tests attempt to measure a broad range of basic traits, such as the Big Five Personality traits or Emotional Intelligence. Although these may provide useful information on how to describe a person, research has repeatedly shown that they do little to predict how well a person will perform in a given job. Academic researchers are re-emphasizing what Chally reported over 30 years ago -personality tests are simply too broad to predict on-the-job performance. Chally, on behalf of the US Justice Department, found it necessary to research solutions well beyond the accuracy of this type of assessment; to identify and measure very precise, job-specific competencies, behaviours, and skills that have been scientifically shown to predict on-the-job sucess.

Trouble with Personality Tests

The Trouble with Personality Tests

The application of Total Quality Management Principles (6 Sigma) to commercial aviation has reduced the defect rate for take offs and landings around 3 parts per million. In contrast the figures quoted for defects in recruitment are in the region of 45%, because we continue to utilise tools that are ineffective. For example, the traditional unstructured interview is only marginally more effective than flipping a coin. The other tool much loved and widely used by managers and HR practitioners are personality tests which are the topic of this article.

Personality tests are a popular component of many or¬ganisations’ hiring processes. As these tests contend to measure traits and characteristics that remain stable over time, it is intuitive to believe information regarding candidates’ individual differences in these areas would be helpful when making selection decisions. Yet evi¬dence supporting the usefulness of personality tests in the hiring process has been called into serious ques¬tion, due to repeated findings that correlations between measures of personality and measures of job performance are not strongly related. After nearly two decades of enthusiastic support for the use of personality assessments, there has been a call for talent management professionals to re-evaluate the merits of these tests.

In a recent article presented by Human Resource Execu¬tive Online, Peter Capelli (2007) briefly reviews the history of using personality tests for hiring and promotion decisions. He remarks that the current popularity of this method is reminiscent of its use as a “best practice” in the 1950s, which he notes is curious given the fact that “by the early-1960s, the consensus among researchers was that personality was not a useful criterion for assessing individuals.” During the 1960s – 1980s “personal¬ity-based assessments ... largely disappeared from the lists of ‘best practices’ in human resources,” however, a resurgence of interest in, and use of, personality testing emerged in the 1990s. Yet the central issue that led to the disfavour of personality tests 40 years ago (i.e., the lack of predictive validity or extent to which the assessment relates to or predicts job performance) still remains an unresolved issue.

A panel of prominent personnel psychologists all former editors of top-tiered journals, recently collaborated on an article discussing the usefulness of personality tests in personnel selection. The one clear theme that emerged from their work was that the validities of personality measures are so low that using them for selecting employees should be questioned. Although research studies have demon¬strated statistically significant relations between some personality factors and certain areas of job performance, the practical significance, or overall usefulness, of these relations remain as weak as those reported 40 years ago. This finding led one author to question, “Why are we now suddenly looking at personality as a valid predic¬tor of job performance when the validities still haven’t changed and are still close to zero?”



While evidence suggesting that personality tests are not robust predictors of job success has been available for decades, the comments made by the panel of experts’ article drew a storm of criticism from other researchers in the field. In a recently published follow-up to the rebuttal articles, the panel underlines that its “fundamental purpose in writing these articles is to provide a sobering reminder about the low validities and other problems in using self-report personality tests for personnel selection.” It is noted that blind enthusiasm for the use of personality testing has stemmed from researchers and practitioners alike, ignoring the basic data demonstrating that personality assessments are poor predictors of job performance. This evidence might be overlooked because of the potential for lowered adverse impact and increased criterion variance explained by the use of personality tests. However, the author’s state that increases in the criterion variance explained has not been realised. In light of these problems, it is noted that Robert Guion’s comments from over 40 years ago still hold true today: “In view of the problems…one must question the wisdom of using personality as instruments of decision in employment procedures.”

Unfortunately, as Capelli asserts, “the least valid of the personality measures are the ones most employers are likely to use: published tests that individual candidates complete themselves.” The most popular personality tests being used for hiring purposes utilise broad-based approaches, such as the Big Five Personality traits and Emotional Intelli¬gence, but these have had limited success. For example, meta-analytic research has found that these tools account for less than 6% of variance in sales effectiveness. One of the reasons for this outcome could be that most personality tests are very broad in scope, whereas the areas of job performance are fairly narrow and spe¬cific. Researchers have posited that the specificity of a predictor (e.g., an assess¬ment measure) should match the speci¬ficity of a construct, or the area of job performance the predictor is designed to predict. It stands to reason that a test designed to predict specific and precise work behaviours and outcomes would predict those specific work behaviours and outcomes better than a test designed to reveal a general and broad sense of an individual’s personality.
Chally concurs with the central tenet of the works described above; however, some context is necessary to avoid the blanket conclusion that any assessment that measures individual differences is bad. Most criticisms apply directly to the broad-based person¬ality tests people in the employment assessment field call “off-the-shelf” measures. These assessments were designed to be general, apply to a wide range of situations (most were not specifically created for workplace application), and are not amenable to customisation. Such measures employ a “one-size-fits-all” approach, which (similar to clothing) does not provide a very good fit in most cases.

The best instruments measures narrow, job-related constructs rather than broad, person¬ality constructs. As opposed to developing a measure descriptive of personal charac¬teristics, the goal should be to develop a measure that best predicted job performance in specific areas. While researchers and practitioners later became interested in the relations among general measures of personality and job performance, the focus must be to predict success on the job. Chally has long championed research designed to measure the competencies, behaviours, traits, and temperaments that predict specific job behaviours. The criterion-related validation approach, which is the statistical demonstration of the relationship between scores on an assessment and the job performance of sample workers, continues to be at the core of Chally’s selection method.

Interestingly, when Morgeson et al., and their critics dis¬cussed ways to improve selection methods, they all agreed that one way to increase validity is to develop tests that keep in mind the outcome, criteria, and/or on-the-job behaviour the end-user wishes to predict. The shared opinion is that keeping these factors in mind will lead to increases in validity and improve one’s ability to defend the use of the test if challenged. This has been a central tenet at Chally since its inception.





The Predictive Power of Chally’s Selection System
The Chally Assessment was designed by taking an actuarial approach (or criterion-related approach) to predict job success, whereas the aim of most published personality measures is to perfectly represent a theory of personality. Researchers agree, regardless of whether they propose using “compound” or “narrow” scales, companies need to measure more than personality traits if they are concerned with predicting job performance. Chally focuses on the competencies, behaviours, and temperaments that predict actual job behaviour. As a result, the Chally Assessment consistently has greater predictive power than existing “off-the-shelf” published personality measures.

Members of Chally’s Centre for Scientific Innovation (C2SI) have published research in several leading ac¬ademic journals and regularly apply their findings in the company’s practice. C2SI’s research supports the conclusion that off-the-shelf approaches have limited success predicting actual job performance because off-the-shelf measures do not consider the possibility that jobs with surface similarities may require different competencies for success. For example, different sales roles require different skills and motivations for success9. Although Extraverts (outgoing people that like to be the center of atten¬tion) tend to make better retail salespeople, they actually perform worse in business-to-business sales10. Business-to-business salespeople focus on listening to the potential customer rather than dominating the conversation.

Chally’s utilization of criterion-related validation studies has led to reductions in turnover of up to 30% and increases in individual productivity of up to 35% in numerous organizations across most industries. Chally has developed a unique assessment based on literally hundreds of actuarial studies (i.e., the rigorous statistical methods used to assess risk in insurance and finance industries). Chally’s research and the research of others con¬sistently demonstrated that personality tests are not robust predictors of job success. Now, top researchers in the field are proclaiming this same conclusion.

How to Recognize a Personality Test
It is not always readily apparent that an assessment is a per¬sonality test designed to describe an individual rather than a work-related measure designed to predict on-the-job behaviours, outcomes, or criteria important to performance. There are three questions that should help one determine the type of assessment being presented.

1. Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour?

2. Was the measure designed for academic or business application?

3. What evidence exists to show how the measure can impact business results?


Most personality tests were designed to describe a theory/model, are academic in nature, and are not likely to impact business results. It is sometimes difficult to avoid personality tests as many have become better known by their acronym. These include the 16pf (16 Personality Factor; IPAT), 6 FPQ (6 Factor Personality Questionnaire; Sigma Assessment Systems), CPI (California Personality Inventory; Westburn Publishers), CPQ (Craft Personality Questionnaire; CraftSys¬tems/Previsor), DPS (Dynamic Personality Shift; Kenexa) HWPI (Harcourt Workplace Personality Inventory; Harcourt), HPI (Hogan Personality Inventory; Hogan Assessment Systems), IPIP (International Personality Item Pool; Oregon Research Institute), JPRF (Jackson Personality Research Form; Sigma As¬sessment Systems), MPQ (Manchester Personality Question¬naire; Hogrefe Ltd.), MMPI (Minnesota Multiphasic Personality Inventory; Pearson Assessments), NEO-PI-R (NEO Personality Inventory – Revised; Psychological Assessment Resources), OPI (Occupational Personality Inventory; Kenexa), OPQ (Oc¬cupational Personality Questionnaire; SHL), PIP (Personality Interview Profiler; PIP), RPQ (Rapid Personality Questionnaire; Kenexa), and WPI Select (Work Personality Index Select; Psy¬chometrics Publishing).
Even more difficult to avoid are those assessments that do not include “personality” in their name, yet are clearly re¬vealed to be personality tests in their accompanying litera¬ture and/or documentation. The following tests would fall into this second category: Caliper Profile11, CDR Character Assessment12, Conditional Reasoning Test of Aggression13, Devine Inventory/Select Best13, DiSC14, Employee Screening Questionnaire14, Gordon Personal Profile Inventory13, MBTI (Myers-Briggs Type Indicator)15, Personal Style In¬ventory (PSI, two different tests with same name from different publishers) 16, 17 and Profiles International Tests18 (ProfileXTTM, ProfileXTSalesTM, Profiles Performance Indi¬catorTM, Profiles Sales IndicatorTM, Customer Service ProfileTM, Pro¬fileEasyTM), Profilers Premium Placements Inc., Job-fit Assess¬ments (The Achiever, The Sales Achiever, The Guardian, The Per¬former, The Scoreboard). 19
Perhaps the most generally effec¬tive way to identify a personality test is to review the output of the measure. If the assessment produces a description of per¬sonality traits, then it can reason¬ably be considered a personality test. Some common personality traits assessed are: Ego, Ego Strength, Ego Resilience, Empathy, Empathetic Outlook, or the Big Five personality traits, namely, Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscien¬tiousness (e.g., DiSC = Dominance, Influence, Steadiness, Conscientiousness). Many assessments disguise the Big Five personality traits by using variations in the trait names such as Emotional Stability, Emotional Control, Sociability, Introversion, Openness, Intellectance, Cau¬tiousness, Dependability, or Responsibility. A reader familiar with a feedback report for the Caliper Profile, the Hogan, or the PreVisor Assessment likely recognized some of these name variations. Likewise, these name variations are common to many assessments that claim to measure predictors of job performance but are actu¬ally measuring personality traits.
Remember these questions: “Was the measure designed to describe a theory or model (usually of personality) or predict future behaviour? Was the measure designed for academic or business application? What evidence exists to show the measure can impact business results?”
Keep in mind the Chally Assessment was specifically designed to predict success in a business environment and that plenty of case studies and testimonials are available to demonstrate how this approach has lead to great success for our clients.

Related Articles
  How to pass psychometric tests
  Employee Testing and Assessments
  Using Pre-Employment Tests Can Save You Money
  How To Choose an Assessment Test
  Design your own Employment Test

Article Tags: 6 sigma, commercial aviation, correlations, enthusiastic support, evi, hiring process, hr practitioners, individual differences, ity, landings, management professionals, personality assessments, personality tests, promotion decisions, quality management principles, recent article, resurgence, selection decisions, talent management, unstructured interview



Related Forum Posts
Re: How did we Live before Electricity - You are right Tim, we are spoiled today with the convenience products we have available.Trouble is once you get used to having these things, it becomes very difficult to imagine life without them. Our ancestors never had them in the first place, so they accepted what they had as they never knew anything different. MichelleJ
Re: How do you budget your life? - Good advice David and something that all parents should teach their children by example from a very early age. Trouble is that many people know they need to budget and save, but they have no idea about how to do it. Like Robert Kiyosaki says, financial intelligence should be taught in school. MichelleJ
Picking a corporate lawyer - Hi Everyone, I was just curious to know what's the most important criteria a small business owner should look at when choosing a corporate lawyer? -Cost/hr? -Location? -Reputation/Referral? -Experience? -Personality? How did you pick your lawyer? And what's a fair price to pay per hour? Thanks
Re: First Facebook Ad - Experiment! - Facebook ads are very easy to set up and manage. One difficulty I found, however, is in the difference between Facebook ads and Google ads. With Google you try and find the best keywords for your product, but with Facebook you try to home in on the demographics of the customer. Trouble with that is that I felt that anybody who has keywords such as "Internet business" on their Facebook profile is likely to **already have** what I am offering! (Of course, they might be looking for a new and better version but...). I have just broadened my ads appeal to "small business" and "independent business owner" to see if that makes a difference. It adds about 20,000 people to my potential market...


Share this article with your friends. Fund someone's dream.

Leave a comment below or share on the left and you'll help support entrepreneurs in Africa through our partnership with Kiva. Over $50,000 raised and counting - Please keep sharing! Learn more.

Tweet

Newsletter

Get advice & tips from famous business
owners, new articles by entrepreneur
experts, my latest website updates, &
special sneak peaks at what's to come!

Name:
Email:

Popular Articles

Attracting Passionate Employees

What Aweber Can Do For Your Online Business

Selling with Humor (and a Sorry Butt)

Suggestions

Email us your ideas on how to make our
website more valuable! Thank you Sharon
from Toronto Salsa Lessons / Classes for
your suggestions to make the newsletter
look like the website and profile younger
entrepreneurs like Jennifer Lopez.