Why Interviewing to find Talent is Difficult: A Demonstration using Bayes’ Theorem

Suppose you are a hiring manager with a goal to make sure that the person you hire has some special quality called IT. IT can be anything you like. You have developed an interview technique in which the chance of success for a candidate that has IT is 95% and the chance of success for a candidate who does not have IT is only 5%. Suppose that amongst the general population of qualified candidates, only 5% have IT. After all you only hire the top 5% like everyone else, right ? What is the probability that a candidate who does not have IT will be selected ? Surprisingly, 50%, the same chance as a candidate who has IT. For example, suppose it is possible for you to interview 100 candidates. Only 5 have it and you will correctly identify all 5 of them (I’m rounding up). Ninety-five do not have it, yet you will also identify 5 from this group as having IT.

Now suppose IT is really rare and only 1% of the qualified candidates have IT. However, you are better at testing for IT so you can identify those with IT 99% of the time and you can also identify those without IT 99% of the time. The end result is the same as the first example, the probability of selecting a candidate without IT is 50%, you pick 1 from each group. Put another way, the probability that any candidate has IT is 1% prior to the interview but the interpretation of the probability rises to 50% after the interview, regardless of whether they have IT or not.

In both cases, the probability of a false negative was high relative to the prior probability of having IT.  If however, we take the candidates identified as having IT from the first interviewer and a second interviewer with equal skill screens them, then the odds of correctly identifying a candidate with IT are greatly improved because we start with a 50% probability that the candidates have IT.

The next time you are tempted to brag about being a great interviewer capable of finding the very best, keep the thought at Bayes and employ a process of sequential, multiple interviews to improve your odds.

Leave a Reply