Dunning-Kruger effects in reasoning: Theoretical implications of the failure to recognize incompetence

Abstract

The Dunning-Kruger effect refers to the observation that the incompetent are often ill-suited to recognize their incompetence. Here we investigate potential Dunning-Kruger effects in high-level reasoning and, in particular, focus on the relative effectiveness of metacognitive monitoring among particularly biased reasoners. Participants who made the greatest number of errors on the Cognitive Reflection Test (CRT) overestimated their performance on this test by a factor of more than three. Overestimation decreased as CRT performance increased and those who scored particularly high underestimated their performance. Evidence for this type of systematic miscalibration was also found on a self-report measure of analytic thinking disposition. Namely, genuinely non-analytic participants (based on CRT performance) over-reported their ‘Need for Cognition’ (NC), indicating that they were dispositionally analytic when their objective performance indicated otherwise. Furthermore, estimated CRT performance was just as strong a predictor of NC as was actual CRT performance. Our results provide evidence for Dunning-Kruger effects both in estimated performance on the CRT and in self-reported analytic thinking disposition. These findings indicate that part of the reason why people are biased is because they are either unaware of or indifferent to their own bias.