Understanding Noise in Human Judgments

I have a reinforcing story to tell relating to the Issues interview with Daniel Kahneman, “Try to Design an Approach to Making a Judgment”(Spring 2022). Among many interesting observations, Kahneman points out how unreliable job interview judgments are, largely due to cognitive shortcuts and biases—what he calls “noise”—that shape, and sometimes misshape, human decisions.

While a member of a large scientific R&D institution (IBM Research) for 25 years, I had the opportunity (and job requirement) of interviewing many dozens of candidates for PhD-level research positions. At one point in my career, I had to move offices and clean out my file cabinets.

Kahneman points out how unreliable job interview judgments are, largely due to cognitive shortcuts and biases—what he calls “noise”—that shape, and sometimes misshape, human decisions.

Coming across 15 years’ worth of my records of recommendations based on these job interviews, I was able to make a subjective evaluation of my own opinions, since a good many of the candidates had started their careers at my own institution or at other places where I was able to follow their progress. Aside from the outliers, the very best and the worst, I was humbled by the randomness of my decisions—which had little correlation between my interview judgments and the candidates’ subsequent success. I even misjudged a future Nobel laureate.

As a result, I did change my interview style to a more structured format, and I claim some subjective improvement, but mostly not at the PhD-level candidate.

Cite this Article

“Understanding Noise in Human Judgments.” Issues in Science and Technology 38, no. 3 (Spring 2022).

Vol. XXXVIII, No. 3, Spring 2022