Understanding Noise in Human Judgments
A DISCUSSION OF
“Try to Design an Approach to Making a Judgment; Don’t Just Go Into It Trusting Your Intuition.”Read Responses From
It was a pleasure to read the interview with Daniel Kahneman, “Try to Design an Approach to Making a Judgment” (Issues, Spring 2022), who is a world leader in research based upon his multiple findings in the areas of attention and decisionmaking, as well as his other contributions to psychology and economics. His interview expresses that expertise and is very helpful in understanding the extent and danger of variability in human judgment.
However, there seems to me to be too strong an implication in the interview that we would be better off if everyone came to roughly the same decisions. In the cited case of insurance actuaries, a 10% variance seems tolerable, not the 50% actually found. The assumption that variability is bad is clarified somewhat in the interview as Kahneman discusses how it may aid creative problem solving by allowing diverse opinions.
To view variability as inherently bad seems to me a judgment error of the type Kahneman has discovered and identified in other situations. Even in the justice system, the effort to impose common minimum sentences for crimes might have reduced variability and increased equality, but it also has had some very bad consequences in filling the prison system.
To view variability as inherently bad seems to me a judgment error of the type Kahneman has discovered and identified in other situations.
If we all thought more or less alike, it seems it would be a more just world—but as a species would we be better off? It seems at first that in some issues where the “correct decision” has expert consensus, such as in vaccination for COVID-19 or climate change, we would be. However, in evolutionary biology variation is celebrated as insurance against some common flaw annihilating the whole species. I have been planting conifers along the river that flows through my forest. They are mixed conifers, as I was warned that planting all fir trees might be bad because a single predator or disease might cause them all to die.
Nearly all scientists say society needs to quit using fossil fuels because of the atmospheric warming it causes. However, can they be really sure that some unknown planetary adaptation might reduce the warming effect? Even though vaccination for COVID-19 has been successful, it is still possible that long-term harm from vaccination will come to some people. In many cases, expert opinion can vary greatly partly because the decision (as in the case of insurance actuaries) depends upon many somewhat separate facets. Kahneman suggests that it might be better to have multiple experts independently rate the importance of the many factors involved so that we could be more certain that all factors will be taken into consideration.
Kahneman has identified an important aspect of human decisionmaking and points also to our lack of awareness of human variability. However, Kahneman and biology tell us that variability can be a good thing for the species at least in some cases, even when we think it unfair.
Michael I. Posner
Professor of Psychology, Emeritus
University of Oregon
I have a reinforcing story to tell relating to the Issues interview with Daniel Kahneman. Among many interesting observations, Kahneman points out how unreliable job interview judgments are, largely due to cognitive shortcuts and biases—what he calls “noise”—that shape, and sometimes misshape, human decisions.
While a member of a large scientific R&D institution (IBM Research) for 25 years, I had the opportunity (and job requirement) of interviewing many dozens of candidates for PhD-level research positions. At one point in my career, I had to move offices and clean out my file cabinets.
Kahneman points out how unreliable job interview judgments are, largely due to cognitive shortcuts and biases—what he calls “noise”—that shape, and sometimes misshape, human decisions.
Coming across 15 years’ worth of my records of recommendations based on these job interviews, I was able to make a subjective evaluation of my own opinions, since a good many of the candidates had started their careers at my own institution or at other places where I was able to follow their progress. Aside from the outliers, the very best and the worst, I was humbled by the randomness of my decisions—which had little correlation between my interview judgments and the candidates’ subsequent success. I even misjudged a future Nobel laureate.
As a result, I did change my interview style to a more structured format, and I claim some subjective improvement, but mostly not at the PhD-level candidate.