Why Your Candidates Are Failing Pre-Employment Skill Tests

Deepti Chopra -

The simple answer to the question is that this is exactly what pre-employment tests are designed for- to eliminate ~80% of candidates. This enables employers to hire quicker by eliminating unqualified candidates at the first step of the hiring process. The important thing to keep in mind here is that a well planned pre-employment test is designed to eliminate unqualified candidates, not to select the top performers of the test.

Why Your Candidates Are Failing Pre-Employment Skill Tests

Hiring managers who set up these assessments/ work for companies where recruitment teams use these assessments need to be involved in the process and make sure the screening process is fair and candidate friendly.

Too many unqualified candidates apply

While recruitment automation has made the hiring process faster and more efficient, it has also made it very easy for candidates to apply to a job with the click of a button. Since there is no potential downside to applying for roles they might not be even remotely qualified for, there is no reason to hold back. This “Spray and Pray” job application method deployed by candidates makes it essential for recruitment teams to deploy automated screening methods to filter down the candidate pipeline to just the qualified candidates.

Tests focus on irrelevant questions/ puzzles

One of the most common mistakes in the process of setting up skills assessments is to focus on trick questions. While it is good to be a puzzle master (even though this skill can be improved with practice), it is not a strong indicator of how good of a candidate someone is/ how good they're going to be in the role. In addition, this way of measuring skills' for software engineer roles has an inherent bias against more experienced developers.

Time and again, studies have shown that asking puzzles in the hiring process is a waste of time, since there is no correlation with on-the-job performance.

Some employers set up a very difficult test

The other common mistake employers make when setting up a pre-employment test is to make it very difficult because that "have a high bar". While you want to have a high bar for who you hire, you want to use the test only to eliminate unqualified candidates.

While setting up the test you should also keep in mind that it is one thing to solve the test questions at leisure, and a completely different experience when you are solving the same questions in a stressful, time-boxed setup.

You should use shorter assessments (30-60 mins) to make sure candidates can do it ASAP, they are investing as less time as possible, while still enough to showcase their expertise. Even for technical roles, a short assessment which proves that the candidate has the ability to write code is good enough to filter out about 80% of the candidates. Once you have shortlisted the qualified candidates, you should do a more detailed analysis of who the best candidate would be via interviews.

Tests have binary (0/1) scoring

One of the major drawbacks of traditional testing tools is that while they are able to measure the current level of candidate skills, they are unable to capture their ability to learn.

Modern conversational assessments enable you to give hints when the candidate is stuck. If a candidate gets a question wrong, they get a hint and a chance to fix it making it possible to distinguish between candidates who can iteratively fix their errors with a hint versus candidates who can't solve the question even after a hint.

40% of candidates on Adaface use hints. With intelligent hints, our bot Ada is able to understand which candidate was able to pick up concepts on the fly, which is a key indicator for job performance.


What the ideal score distribution looks like

You should expect ~20% of the candidates to do well on a well-designed pre-employment assessment and the score distribution for the assessment should follow the normal curve- standardized tests like SAT and GRE follow the normal distribution (bell curve).

Status quo assessments typically have the exact opposite score distribution as compared to the bell curve. Too many candidates getting a score very close to 0, too many candidates getting a score very close to 100 and too few candidates in the middle. This is typically because they focus on theoretical questions or trick questions, which are very binary in nature- either you know the trick/ theory or you don’t.

If you currently use a pre-employment test for your hiring- go and take a look at the score distribution. What does it look like?

Review performance in detail

While most skills assessments tests are graded, and give you a number to quantify the candidate's qualification with respect to the job - you should also make sure to look at a candidate's performance in detail (esp when you have just set up the test, and are yet to test the validity of the test). The detailed report about which questions the candidate answered, and which ones they got wrong can provide additional insight into the candidate's ability. This might be especially useful for coding tests where the candidate might have actually written most of the logic to solve the question, but because of a lack of time there were unable to fix the errors.

Checklist for a candidate friendly assessment

  1. Make sure your assessment is short (30-60 mins).
  2. Use candidate-friendly messaging in the invite email.
  3. Make sure the candidate knows before hand that they will receive a link to a skills assessment to improve completion rates.
  4. Remove any puzzles/ trick questions from the test.
  5. Verify that the questions on your test are relevant to the role.

Conclusion

If you set your skills assessment test well, it will do a good job of filtering out unqualified candidates so you can focus your time and energy on finding the best from among the qualified candidates.

At Adaface, we set up 45 min candidate-friendly assessments that test for on-the-job skills. 🤩

Adaface assessments have anti-cheating features like non-googleable questions, web and webcam proctoring, IP monitoring and plagiarism detection to enable you to administer tests with confidence. As soon as a candidate completes the assessment, you get a notification and the detailed scorecard is available so you can look at the question-wise performance. The cut-off score is initially set to the industry standard and you can modify it over time as per your requirement. The scorecard also enable to you see the candidate's performance for each skill, and each question.