The Importance Of Predictive Validity In Pre-Employment Assessments

In the context of pre-employment assessments, predictive validity refers to how likely it is for test scores to predict the future job performance of the hired employees.

Imagine - would it not be great if there were some way to forecast precisely how an employee would perform before they even started the new job?

While we do not quite have a magical formula for that yet, we do have an excellent way to tell whether the non-magical predictive tests are accomplishing what they ought to.

In 2016, Professor Franck L Schmidt published a meta-analysis titled "The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years."

In it, he compared 31 different assessment methods to determine which are of little value, which could end up being detrimental to the decision, and which assessments truly help to find the best-performing candidates.

Based on these findings, we have been able to identify and design candidate selection methods with higher predictive validation. Let us take a closer look at what that terminology means:

What is predictive validity?

A predictive validity definition refers to the degree to which pre-employment assessment scores accurately predict scores on a criterion measure.

Typically, it is obtained by correlating different forms of measured performance output (such as scores from technical or behavioral tests) with the criteria that the method is evaluating (such as how well a candidate will do on that test). This correlation is represented as r, where r can range between -1 and +1.

It is generally considered impossible to have a correlation of +1, which means 100% accuracy in performance prediction. Neither is it possible to have a correlation of -1, which means that the actual outcome is always the exact opposite of the prediction.

For most pre-employment assessments and other general tests, a score between 0.30 and 0.70 can be expected, with a minimum score of 0.30 for the test to be considered acceptable for use.

How does predictive validity work?

In the hiring context, predictive validity determines how well a specific test can predict candidate performance, be it in job tasks or in terms of cultural fit. A scatter plot is one of the tools commonly used to assess predictive validation.

Essentially, it plots test scores on the X-axis and job performance on the Y-axis, and a horizontal line would represent the 'ideal' test score for performance. Any candidate whose score is on or above that line would be deemed a successful performer.

So essentially, the scatter plot tells us:

• How difficult the test is
• The correlation between test scores and job performance, based on which the test can be deemed acceptable (strong positive correlation) or unacceptable (weak positive correlation)
• Where the ideal score line ought to be set, depending on what the job calls for

Why is predictive validity in candidate selection important?

Companies need to find the right people for the right job if they hope to survive and thrive in today's highly competitive business world. The longer the recruitment funnel is, however, the more expensive it becomes - and meanwhile, there are urgent vacancies to fill up.

In this context, predictive validation helps speed things up by identifying the recruitment tests that are most accurate when it comes to selecting quality long-term candidates.

By measuring the utility and reliability of their selection process, businesses can make the necessary changes and bring the right people on board while being confident that they will stay on and have a long-term impact.

Predictive validity vs. construct validity

These are two terms that are often confused. We break them down as follows:

• Predictive validity - This is an approach used to select the most suitable candidates for a job. It serves as a validity criterion for the best performers who the company can hire, knowing that their long-term performance will be strong.
• Construct validity - This measures how effectively a specific measurement matches the construct that one wishes to measure. It requires the team to prepare a list of relevant questions that will help them know which approach to select.

Examples of predictive validity

Predictive validity is frequently used to determine whether or not a new candidate is suitable for a role. Here is a predictive validity example: a company might conduct a specific test among its existing employees after obtaining their performance reports from their job supervisors. The company can then conduct the same test among new candidates.

The test results are likely to show a big difference in performance between the existing and the new candidates, as the existing employees are more knowledgeable and skilled after having learned with the company for a while.

With appropriate score benchmarks, the test can then predict how well a new candidate will contribute to the organization.

It is essential to mention here that predictive validation is not just a recruitment tool. It is a beneficial way to evaluate the accuracy of assessments in any field and can power essential decisions at the individual and the organizational level.

Some general examples of predictive validity include:

• Personality tests that can predict future career performance
• Depression outcome tests that predict potential future behaviors in mental health patients
• IQ tests that predict how likely someone is to get a university degree somewhere down the line
• Math tests that predict one's ability and success in the science fields

Predictive validity and psychological testing

Certain psychological traits, such as honesty, are critical for any position in any company. A person can be assessed for honesty by comparing their responses to a set of standard responses that have been determined based on commonly accepted scientific standards.

If the test score consistently demonstrates that the people who scored high were honest in their behavior, the test has predictive validation later on. Such tests are in high demand as hiring tools across industries.

The same applies to testing behavioral traits. Over the last few decades, the five-factor personality structure has become the accepted standard for measuring one's personality dimensions, the five factors being extraversion, conscientiousness, agreeableness, neuroticism, and openness to experience.

It has been demonstrated that conscientiousness as a trait has high predictive validity for all jobs, whereas the other four have predictive validity for specific jobs. Moreover, studies show that narrower traits that are subsets of the prominent five attributes could have greater predictive validity in job selection contexts.

Other forms of validity

Various forms of psychometric and statistical validity are used in addition to predictive validity. Some of these are:

• Construct validity - assesses how efficiently a test measures the idea it is designed to measure.
• Criterion validity - assesses the correlation between different test results that measure the same concept.
• Content validity - assesses test content to check how well it represents a sample of the behaviors it measures.
• Statistical conclusion validity - checks the accuracy of conclusions made about the link between test variables.
• Internal validity - checks the conclusions made about cause and effect based on test measurements and methodology.
• External validity - checks how test results can evaluate different candidates and different times outside of the test environment.

Importance of validity in employee assessment

Validity is the most important among the criteria for psychometric quality, and predictive validity is thus vital for measuring future occupational success.

Some of the diagnostic procedures that have been demonstrated to show reasonably good validity include general intelligence tests, structured interviews, work samples, some types of personality tests, and multimodal procedures.

During the stages of instrument construction, content validity is important. Construct validity comes into play when looking at the psychological implications of test scores.

Advantages and disadvantages of predictive validity

An ideal way to directly establish predictive validity is by performing a long-term validity study. You can do so by conducting candidate assessments and checking if those test scores correlate with the hired employees' future job performance. However, like any measure of outcome, predictive validity has its advantages and disadvantages:

Advantages

• Predictive validity has a strong track record of identifying tests that accurately predict future outcomes like job performance or university qualifications.
• It can help companies decide who to hire and promote and what new initiatives to launch.
• It can help improve workforce morale by introducing necessary changes such as more accommodations for the differently-abled.

Disadvantages

• The reliability of chosen criteria and any inherent biases can negatively affect test outcomes.
• Obtaining results can take a considerable amount of time, depending on the number of candidates and the complexity of the test.

Biases in predictive validity

Any assessment test that humans run is likely to have conscious or unconscious biases in one form or another. Often, these are based on preconceptions held by the recruiter that are ingrained in their mind and come out unconsciously.

For instance, they may make assumptions about a candidate based on their surname or their hometown, or they might assume that years of experience necessarily represent more innate ability. However, that is not the right direction to take.

When these biases creep in during or after the interview/test process, they can skew the results and adversely impact the test's predictive validation.

Organizations need to identify and curb biases as promptly as possible to ensure that candidates are chosen on merit alone and that existing employees are safeguarded from discrimination.

The evaluation of test bias seeks to assess whether the link between test and criterion scores is consistent across multiple groups of test candidates.

It is typically done with the help of regression models, which can be applied in two ways. One way involves fitting separate regression lines to the test score and criterion data for each group and then comparing based on intercepts and regression coefficients.

The other way involves using only one regression equation when there is not enough data for separate ones. One variation of this method consists in estimating the coefficients from only the data of the 'majority' candidates and then predicting the criterion data for those in the 'minority' candidates.

Another variant involves data from both majority and minority candidates and then comparing prediction errors across different groups.

Over to you

In conclusion, finding the right person for each job is an ongoing challenge, and there is no perfect way to predict anyone's performance.

Predictive validity, however, can give recruiters more reliable indicators of which factors accurately measure future performance, thus helping to streamline the assessment process and make smarter decisions on the whole.

If you want to introduce pre-employment assessments in your hiring process or change how you test your candidates simply, look no further.

Test candidates for on-the-job skills, cognitive abilities, key personality traits, and more to choose the best people for your organization. Head to our website to know more.