This is the top of today’s newsletter. But I’m going to tell you now that the best example I can think of for how to deal with the phenomenon we’re about to explore comes at the bottom.
Literally.
For now, just know that after nearly three decades of working, and about a zillion different careers and pivots, I’d like to think I have a reasonably accurate idea of my own strengths and weaknesses.
I know what I’m good at. I know where I fall short. I could be wrong about both, obviously, but generally I trust my own read.
According to brand-new research, the fact that I feel confident saying that might actually count as evidence in my favor.
The Dunning-Kruger effect
For nearly three decades, one of psychology’s most famous findings has told us something very different: The least competent people are supposedly the most confident, while people who actually know what they’re doing tend to underestimate themselves.
It’s called the Dunning-Kruger effect, after a 1999 paper by David Dunning and Justin Kruger.
You have almost certainly heard of it, even if you’ve never read the paper. The idea escaped academia and became a meme, a management cliché, and a useful way to explain why that one guy in every meeting keeps talking long after everyone else has stopped.
Now, a new study from the University of Bath and the London School of Economics says the whole thing might have been backward.
Highly confident people
The paper, published in July in Psychological Review, was written by Chris Dawson of Bath and David de Meza of LSE.
They went back through the large datasets used to support the original Dunning-Kruger findings and ran the numbers again.
The key difference was that they corrected for something called regression to the mean.
Here is my attempt at the plain-English version, written with less confidence than I had before I started reading about regression to the mean:
When you divide people into groups based on one set of test scores, you are not necessarily dividing them perfectly according to ability.
Somebody might score badly because they got a rough group of questions.
Somebody else might score unusually well because things broke their way.
So the lowest-scoring group will include some people who are not really as bad as their scores suggest, and the highest-scoring group will include some people who are not really quite that good.
‘A universal human trait’
That matters when you compare the scores with how people rated their own abilities.
It can make the people in the bottom group look more delusional than they are. It can also make the people at the top look strangely modest.
Dawson and de Meza say that once they corrected for this, the famous Dunning-Kruger pattern disappeared.
People at all levels were overconfident. But the highest performers were actually the most overconfident of all.
“When we clear away the statistical problems, the apparent pattern disappears,” Dawson said. “Instead, we see that overconfidence is a universal human trait, but it is the most capable among us who exhibit it the most.”
Strategic signalling
Dawson and de Meza have a theory about this. Heck, let’s call it the Dawson-de Meza effect.
People who are genuinely good at something have the most to gain by projecting confidence.
Suppose you are very good at your job, but the people around you cannot easily see or measure how good you are. Maybe they do not even understand your work well enough to judge it.
Talking confidently becomes a way to signal something they cannot observe directly.
And if you really are good, there is less risk in doing it. You are more likely to be able to back up the claim when somebody challenges you.
A person with less ability has more reason to be careful, because the bluff is more likely to be exposed.
The researchers call this strategic signaling. The better you are, the more incentive you have to make sure people know it—or perhaps to make sure they think you are even better than you are.
A practical problem
So, does this mean Dunning and Kruger were careless? Does it mean incompetent people never overestimate themselves?
Or, in perhaps the most ironic possibility of all, did the Dunning-Kruger effect become so famous and get repeated so confidently because it was wrong?
I mean, hardly.
This is one new analysis, and I am sure there will now be arguments over the analysis, the correction, the interpretation, and probably the meaning of regression to the mean itself.
That is how these things usually go.
But it does create a practical problem.
Suppose someone speaks with great confidence about a subject you do not understand.
Under the classic Dunning-Kruger interpretation, confidence might be a warning sign. Maybe the person does not know enough to recognize how little he knows.
Under the Dawson-de Meza interpretation, however, confidence might mean the person is genuinely very good and knows he can support what he is saying.
So which is it?
Getting to the bottom of it
The only solution I can think of is to push back a little and see what happens.
If the confidence is mostly bluff, the cracks should start to show. If it is supported by real knowledge, the person should be able to explain further, answer questions, and give you better reasons to trust him.
Of course, this is much harder when you know almost nothing about the supposed area of expertise.
Which brings us to the example at the bottom that I promised at the top.
Honestly, it might be the best example I have ever come up with, even though I had no idea at the time that I was applying either the Dunning-Kruger effect or the Dawson-de Meza effect.
A few years ago, I passed that magic milestone where you reach the age when your doctor starts telling you that you need a colonoscopy.
I met the doctor who would be performing this intimate procedure only a short time before they wheeled me into the room.
He asked whether I had any questions.
“Just one,” I said. “How many of these have you done?”
I think he found the question amusing.
He started doing the math out loud.
About 25 years with this as his specialty.
Figure he worked perhaps 40 weeks each year.
Maybe two procedures per day.
Multiply 25 by 40 by two, and you get 2,000.
“That’s an interesting question,” he said.
“This is a very unusual day for me,” I explained. “I just wanted to make sure it wasn’t also an unusual day for you.”
It was a good way to gain confidence in his confidence.
Other things worth knowing …
AP: The bodies of a Texas chiropractor and at least one other person have been found after a major weekend flash flood swept through the Grand Canyon, with more than a dozen hikers still missing following waves of rain in the Bright Angel Creek area. Forecasters warned more flooding was possible as heavy rain shrouded the canyon walls of the tourist destination that hosts 4.4 million visitors a year.
ABC News: A jury found Duane “Keffe D” Davis guilty in the murder of Tupac Shakur, 30 years after the rapper was killed in a drive-by shooting in Las Vegas. Jurors deliberated for less than three hours before finding him guilty of murder with the use of a deadly weapon with intent to promote a criminal gang. The 63-year-old faces a possible maximum sentence of life in prison.
Yahoo Finance: Tim Cook stepped down as Apple CEO on Monday, with parting words for the company’s employees who, he wrote, “share a belief that what we build matters.” John Ternus, the company’s senior vice president of hardware engineering, takes over. Cook, CEO since 2011, will remain on as executive chair.
AFP via Yahoo: President Trump attacked opponents of AI data centers on Monday, saying they would end up “backwards and poor” and were helping rival China. A backlash against the energy-guzzling centers has become a surprise issue in November’s midterm elections, with Trump at odds with many Republicans.
WSJ: Prediction market Kalshi banned former Rep. George Santos from its platform for life after a federal regulator alleged he made more than $17,000 betting on his own attendance at Trump’s State of the Union address. Kalshi also announced a three-year ban for a Republican congressional candidate in North Carolina.
NY Post: Happiness has an address — and it’s in California. Fremont has been crowned the happiest city in the country, according to a new WalletHub study ranking 182 of the largest U.S. cities using 29 measures spanning emotional and physical well-being, income and employment, and community and environment.
The Guardian: A 91-year-old hiker has reclaimed his record as the oldest person to complete the Appalachian Trail. By his count, Dale Sanders of Tennessee fell 71 times during the 2,193-mile feat — six of which were hard falls, one requiring a brief hospitalization and another occurring just 200 yards from the finish line.
Thanks for reading. Photo by Bharathi Kannan on Unsplash. I wrote about some of this at Inc.com. See you in the comments!
This line is the line people should remember and tuck away for future reference, as I know I will:
"This is a very unusual day for me,” I explained. “I just wanted to make sure it wasn’t also an unusual day for you.”"
The number of colonoscopies could be too low by as much a factor of 5 unless the doctor only schedules procedures one day a week.