Do test scores truly indicate a person’s ability, or accurately reveal their personality? Psychometric assessments are used to make decisions about employees, so it is essential that scores really do indicate a person’s likely behaviour. Accuracy of results is affected by both the test itself and the way in which it is administered and interpreted. Here’s how to ensure scores can be trusted when you use psychometric assessments:
Stage 1: Preparation for Testing and Maximizing Candidate Engagement
Maximizing engagement means motivating the candidate to work earnestly and honestly as they complete their test. People need to see the value of putting effort into testing, and to consider potential outcomes of psychometric assessments worthwhile.
To maximize engagement, make sure that the process and purpose of the test is understood. Build trust on behalf of the candidate that assessment will be accurate, fair, and useful in identifying the best applicants.
If you’re familiar with psychometric qualities, you may recognize this as another way of saying the test should be standardized, reliable and valid. It’s up to you to communicate these qualities to your test takers in non-technical terms they’ll easily understand.
Consider candidate reactions to assessment, too. People are most likely to engage with a test that has both face and faith validity. Face validity refers to whether or not the test appears to be a good measure for its purpose. Does it look the part? Faith validity is the test taker’s belief that the test really does what it claims to. Again, this is about the person’s impression of the measure. Face and faith validity are both subjective reactions that can make a significant difference in how a test is undertaken.
It’s also helpful to explain clearly why candidates should respond accurately and truthfully when completing a test. This can include emphasizing why and how the test results will be used. For example, you can let the test taker know that scores will help assess whether he or she is a good match for a position. You can explain how this benefits the individual as well as your business.
Measures to prevent cheating, faking or lying on psychometric assessments can also be incorporated. There’s a lot that can be done here, ranging from requiring a signed honesty contract to analysing response patterns and statistics for computerized tests. An honesty contract requires signed agreement from the test taker on conditions such as completing the assessment independently, and responding truthfully. If tests are administered on-line, potential analyses of responses include overall time taken for the test, time spent on each item, and incorrect vs.correct answers to questions which vary in difficulty. Sanderson, Viswesvaren and Pace (2011) provide more detail in their discussion of ways to detect cheating on unproctored internet tests. For complex personality assessments, subscales are typically included to detect social desirability or faking. Scores on these subscales are examined to evaluate potential misrepresentation when answering test questions.
Stage 2: Administering the Test
The way a test is administered affects the accuracy of results. Consistency and control over the assessment processes help to keep unwanted influences from affecting test scores.
“Measurement error” is the term used to describe unplanned or uncontrolled influences on test scores. Anything about testing circumstances may fall into this category. Losing power when completing an on-line assessment is a clear example. Other measurement errors could include a noisy or badly lit room, an unprepared administrator, or an exhausted test taker.
Psychometric assessments typically have specific and standardized instructions for administration. These may include time allowed, a script for introducing the test, appropriate methods for administration, and ways to maintain security and maximize honest responding. Strictly adhering to these guidelines helps to ensure recommended procedures for testing are in place, and maximizes the opportunity for accurate measurement.
Stage 3: Scoring and Interpreting Results
Verify Apparent Test Scores
We talk a good deal about the need to treat scores as hypotheses on our Test User courses. It is essential that apparent results are supported with verifying information. For selection, this could include examining competencies through structured interviews. Ratings from others who know or work with the test taker can be used as part of 360 degree feedback for development. Candidates themselves can provide useful perspectives, including the way that they approached assessment, and how they feel they performed. This may take place through an interactive, two-way discussion following assessment. Information shared by the individual can help to indicate whether they were focused, motivated and optimally engaged in test taking. If so, their scores are more likely to indicate their true attribute levels.
Use Confidence Intervals to Interpret Results
Tests are not perfect, even under optimal conditions for assessment. By considering confidence intervals surrounding test scores, you can more fairly and justifiably (or legally) make decisions about results.
Here’s an example: Two candidates are being considered for progression to the interview stage of selection decisions. Steve’s verbal ability test score is 53. Frank’s score on the same test is 57. Are these scores different enough to justify choosing Frank for the job? If you are not sure, image how Steve would feel, or what his reaction would be!
Confidence intervals are constructed based on a statistic called the Standard Error of Measurement. They provide a range of scores that likely include an individual’s actual ability or attribute level, rather than assuming that a single score point (e.g., a 57) is completely accurate. Using confidence intervals increases the chance that you have really “captured” or identified the person’s true score. For example: You may not be completely certain that Mary’s score of 70 precisely indicates her computational skills, but you can be 95% certain that her numerical skills fall within the range of 63 to 77.
SxD trainees learn how to compute and use confidence intervals in our Test User Ability course. Alternately, we offer advice and consultation to anyone working with employment testing on this essential aspect of interpreting scores.
Ensuring accurate psychometric assessments requires planning and preparation across the stages of test set-up, administration and interpretation. By incorporating the suggestions above you can best achieve test results that you can trust. After all, isn’t that a key goal for workplace testing?
Wishing you happy and accurate testing!
Barbara
____________________
Reference
Sanderson, K.R., Chockalingam, V., & Pace, V.L. (2011). UIT Practices: Fair and Effective? The Industrial-Organizational Psychologist, 48(3), 29-37.
