Dunning–Kruger effect
Transcript:
A webcomic in four panels:
Panel 1: A very young knight, poorly equipped (oversized pants, topless, wooden sword and shield, rusty helmet), screams with overconfidence in front of a cave on a foggy night.
Knight: Master of Dragons! I challenge you to a duel, against ME, the greatest knight the kingdom has ever known!!! My knowledge of fencing is unrivaled!!!
Panel 2: The dragon walks calmly out of the cave, only the large nose is visible while his head is still in shadow. Not a fierce look, but a massive size. He is just tired of this kind of interruption. The knight continues:
Knight: You're terrified, aren't you?! well, you should be! I'm terrifying! Show Yourself!!! Coward!!!
Panel 3: The camera shows an oversized dragon, the knight is the size of a mosquito, attacking his toenails. The dragon just look at the situation with thinking. And an epic full moon illuminates the scene.
Knight: Hyah! Hyah! Partner of the dragon (off panel, from the cave): Honey, who is that?
Panel 4: A close-up of the dragon, turning his head towards the cave to answer his partner:
Master of Dragons: It's no one. As usual: an overconfident newbie, victim of overestimating his own abilities. A classic bias in human nature. Knight (off panel): Hyah! Hyah!
299 - 
542
55 comments
gub
3 ★Je le reconnais ! C'est Jean-Kevin, le stagiaire de la hotline !
Riedler@donotsta.re
the dunning kruger effect has largely been disproven (or at least what people think it is) and people are generally ok at estimating their own abilities.
regardless, good comic 10/10
klmr@mastodon.social
@Riedler That’s not true. There was one recent paper that made a splash claiming this, but multiple studies over the years have corroborated the general existence of such an effect in various scenarios.
scien@social.linux.pizza
@klmr @Riedler There may be a different study that shows this effect, but the original paper was completely incorrect: it was autocorrelation.
As for a "general effect" - we have evidence that novices will misjudge their expertise more than experts, yes, but this goes *both* ways. The original paper simply claims they overestimated (or, at least, overestimated wildly more than they underestimated).
Also, it wasn't simply a *recent* paper that disproved the original - there's papers as early as 2016. That's 8 years ago.
scien@social.linux.pizza
@klmr @Riedler See: this thread I made here about it: https://social.linux.pizza/@scien/112605995976914801
If you know of any studies that indicate a similar conclusion as the original dunning-krueger effect study, I'd be quite interested.
Riedler@donotsta.re
@scien @klmr honestly you just need to think about it a bit to disprove the original study. look at the graph and think about what it really means. yeah, nothing at all, really. With a bit of an asterisk, but it's definitely not what people generally think.
I'd love to do an explainer on this but I'm a bit scared of coming off as too, err… womansplainy? idk, I hope you know what I mean.
marctapages
encore un à croire que c'est malin de dissoudre....
Je pense que je souffre d'un biais aussi :-\
Gynux@the.goofs.space
Ah tiens, je ne connaissais pas cet effet ! Merci !
★Pas sûr que je le retienne par contre...
Sur l'article wikipedia, l'exemple donné (à base de jus de citron) est assez saisissant et j'avoue que je n'aurais pas nécessairement décrit le protagoniste comme quelqu'un souffrant de "surconfiance" 🤣
Ceci dit, je suis sûr que ça m'est déjà arrivé dans une moindre mesure.
NorCalWineLady@sfba.social
★What came to mind? A classic Calvin and Hobbs tale. Still, good and funny at the same time.
overbyte@gamepad.club
funny with beautiful art. #follow
★davidrevoy
@overbyte
Thank you!
★MartyFouts@mastodon.online
DK is my favorite example of bad science that resonates so much that it’s impossible to convince people it’s wrong. The original paper had an error in the statistical analysis. It turns out that there is no correlation between intelligence and the ability to self evaluate it. Over confidence is just as likely among highly intelligent people as it is among everyone else.
scien@social.linux.pizza
@MartyFouts Interestingly, I believe there was a separate study that proved there was a phenomenon present about misjudging your capabilities - but it went both ways.
In short, the study concluded a lack of expertise (in a particular subject, not intelligence mind you) was correlated with both overestimating AND underestimating your capabilities.
Which makes a lot of sense. How do you know where you stand in terms of expertise without knowing of trends in the wider field?
subpanel@mstdn.social
@scien @MartyFouts
On which task did the study measure performance?
scien@social.linux.pizza
@subpanel @MartyFouts
SCLI averages. SCLI stands for Science Literacy Concept Inventory - a 25 item test, which is outlined in this paper: https://journals.asm.org/doi/10.1128/jmbe.v17i1.1036
I found the article I read on this: https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/
In it, they cite a figure from: https://
(Original message has been truncated: read the complete original message here.) 🖼️ 3653f66ecda9ebd0.pngMartyFouts@mastodon.online
@scien @subpanel And there’s an error in the interpretation of the analysis of the data. It’s not exactly the same error as in the original DK paper but it’s from the same class of errors.
They are showing an autocorrelation between degree of expertise and range of expertise that matches both the actual and estimated self evaluation.
scien@social.linux.pizza
@MartyFouts @subpanel
Pardon? I'm not sure I understand. Are you saying this study I linked connects the expertise factors and self-evaluation and thusly results in auto-correlation somehow?
If so, could you elaborate? Afaik the variables are quite separate. Here's another graph demonstrating the effect:
🖼️ 21eadffb2fa87f65.pngscien@social.linux.pizza
@MartyFouts @subpanel
Perhaps the effect is best showcased here. The vertical axis indicates the recorded *error* in estimation of abilities compared with the actual results.
The black dots are the individual estimates (the green zones are the means afaik). Thusly, you can see how the professors group has much more reduced spread, than, for instance, the freshman group.
scien@social.linux.pizza
@MartyFouts @subpanel
Now tbf the study still has its limitations (for instance you can see how the professor count is a pretty low 69), but I'm not sure autocorrelation is one of them... especially considering the authors went at great length to prove this of the original dunning-krueger in the very same paper.
I'd be very interested in learning why, if this was actually the case.
MartyFouts@mastodon.online
@scien @subpanel I think that either interpretation is reasonable and without the raw data we can only rely on our intuition. But I think that it is an experiment design problem.
MartyFouts@mastodon.online
@scien @subpanel The range of skills that they use to describe someone as “expert” is small. The range of skills that they use to describe someone as non expert is large. If accuracy of self assessment was dependent on skill level then the least square differences would be much smaller for experts then they are and the range for non experts larger. That the ranges of actual and self assessment match indicates autocorrelation. /1
MartyFouts@mastodon.online
@scien @subpanel You need to do correlation calculations on the raw data to show that but intuitively it is similar to the error in the original study.
scien@social.linux.pizza
@MartyFouts @subpanel
Is it possible that this has very little effect on the end results? In the paper the authors quite literally input randomized data to show that it doesn't fail to the same fault.
scien@social.linux.pizza
@MartyFouts @subpanel
🖼️ 37ae24c12d3a193a.pngMartyFouts@mastodon.online
@scien @subpanel Possible, yes. I just think that emphasizing the size of the self evaluation set versus the measure set is misleading. I suppose I could write the authors and ask for the raw data but I don’t have the resources now that I’m retired.
MartyFouts@mastodon.online
@scien @subpanel In a related note: one thing I know from teaching is that experts are terrible at predicting how hard problems will be for learners.
scien@social.linux.pizza
@MartyFouts @subpanel :P That reminds me of an XKCD panel: https://xkcd.com/2501/
Btw - thanks for the insight!
It really is quite hard to be sure you're interpreting a paper right, let alone that the paper itself doesn't have some intrinsic issue itself, without the relevant knowledge.
MartyFouts@mastodon.online
@scien @subpanel There’s always an XKCD. Thanks for reminding me of that one.
subpanel@mstdn.social
@MartyFouts @scien
Please tell M. E. the test scores used in the study weren't about estimating "the average person's familiarity" with anyone's field.
m@miruku.cafe
@MartyFouts@mastodon.online @davidrevoy@framapiaf.org
oh? i was under the impression that it corretly claimed that people who are not average (either significantly worse or better than average) tend to guess their abilites lean more towards average.
what you said is interesting and i wouldnt be surprised if its true, do you have a source?
writingshrink@mastodon.social
@m @MartyFouts Marty may have other sources but this one is a good read: https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/
MartyFouts@mastodon.online
@writingshrink @m another take https://www.psychologytoday.com/us/blog/how-do-you-know/202012/dunning-kruger-isnt-real
m@miruku.cafe
@writingshrink@mastodon.social @MartyFouts@mastodon.online @davidrevoy@framapiaf.org
this is very nicely put!
turns out what I thought dunning kruger was was a little bit more sensical than what they claimed, but was still wrong!
cool stuff!
kelson@notes.kvibber.com
@MartyFouts I was always under the impression that the effect was describing a correlation between competence at a specific skill and the ability to evaluate it, not overall intelligence.
MartyFouts@mastodon.online
@kelson It was, but the popular telling has simplified the claim. The actual claim was wrong for the reason I gave: what they thought was a correlation was an error in their analysis.
oldgeek@masto.yttrx.com
@MartyFouts
Yup. I believe SV is full of 'em
sun@shitposter.world
he's a kid, the dragon should encourage him to get better! ★
mhunt@socel.net
you should read the "Dragon Knights"
★davidrevoy
@mhunt Thank you for the recommendation. This one? https://en.wikipedia.org/wiki/Dragon_Knights
mhunt@socel.net
no, this one:
https://www.amazon.co.uk/Chronicles-Dragon-Knights-Vol-Jaina-ebook/dp/B074VGNXC9
It was published by Delcourt but I went to their website and couldn't find it.
derle
@mhunt La geste des chevaliers dragons ?
mhunt@socel.net
@derle yes. I only know the English title. I think it doesn't has new volumes.
derle
@mhunt It has 32 volumes in French ^^ (the editor is Soleil, not Delcourt, even if Delcourt bought Soleil their genre are a bit different)
davidrevoy
@derle @mhunt Wow, that's a large saga. i'll try to keep the title in mind when I visit my book store.
derle
It's very Soleil through, you have been warned ^^ @mhunt
★m@miruku.cafe
@davidrevoy@framapiaf.org
the comic is great but this isnt really dunning kruger
fredds@mamot.fr
si je peux me permettre : le dessin ne ressemble pas trop a Macron, pourtant je suis sur que c'est lui qui est décrit, non? ^^
greenCoder@functional.cafe
"This is but a scratch" ^_^
🖼️ e859b02f39eb09b7.png ★xuxxux@social.tchncs.de
@greenCoder aaaah the classics.
King Ohhh....very nice...I didn't vote for you
- You don't vote for the king.
How'd you get to be king anyway?
xuxxux@social.tchncs.de
@greenCoder
- The lady of the lake, her arm clad in the finest samite, thrust forth Excalibur from the bosom of the water thereby declaring by divine right that, I, Arthur should be your king.
Listen, women laying around in ponds distributing swords is no
basis for a system of government. Supreme executive power is
derived by a mandate from the masses not by some farsical aquatic
ceremony.
breadbin@bitbang.social
The wooden sword gets me:)
★Frischling@wehavecookies.social
@anatoliyl no, those knights only exist, because they smell bad and don't taste.
jmcourtiade
Nice one. Also known as the Dunning–Kruger effect (this is terrible with under-aged bosses or managers...)
chakatfirepaw@universeodon.com
(A week later.)
"Again? Let me guess, you are another great knight?"
★"Me? No no no, I'm just a high school student from another universe who has...."
"HONEY! Pack the hoard and get out of here!"
elCelio@mastodon.uno
an annoying mosquito
★Post a reply
The comments are synchronised every 4h with the replies to this post on Mastodon:How to use this? (click here to unfold)
Open a new Mastodon account on the server of your choice. Then, Copy/Paste the adress above in your Mastodon 'Search' field. The post will appear and you'll be able to fully interact with it. You'll have full control of your posts: edit, remove, etc. After that, your message will appear here.
Just please note that it may take up to 4 hours for your changes to be reflected here.