Amateurs don’t know they don’t know
I’ve been struggling to understand why teaching undergraduate students to write for the workplace is so difficult since around 1988. Quite a while back, I recognized that students have to clear a psychological hurdle to succeed. I just couldn’t figure out what to call that hurdle. Until now.
In a 1999 article in the Journal of Personality and Social Psychology, two Cornell psychologists, Justin Kruger and David Dunning, wrote
. . . those with limited knowledge in a domain suffer a dual burden: Not only do they reach mistaken conclusions and make regrettable errors, but their incompetence robs them of the ability to realize it.
This phenomenon, called the Dunning-Kruger Effect in the authors’ honor, is the explanation I’ve been searching for.
Even the brightest and most motivated college students I’ve taught over the years have needed half a semester to accept that their skill at writing something as seemingly simple as a business letter or informative email is inadequate. For those less skilled or motivated, it has required even more time — if it ever occurs. College students aren’t the only writers with issues. Recently, Brian Garner wrote in the ABA Journal:
For many years in lectures, I’ve likened practicing lawyers, when it comes to writing, to 23-handicap golfers who believe that they’re equal to the touring professionals.
My criticism of students’ early attempts is usually dismissed as faulty (with various explanations for my mistake). Though I never let it influence my evaluation of their work, this situation can be problematic for college instructors whose teaching performance is measured with student evaluations.
In a 2004 article in Psychological Science in the Public Interest, Dunning and his colleagues recommended the following educational interventions to improve the accuracy of students’ self-assessment.
- Introduce desirable difficulties into instruction: spreading training over several sessions, varying the circumstances of the training, reducing feedback, and providing contextual interference (i.e., practicing different subskills under a random rather than blocked schedule).
- Include long-term evaluation of retention of skills.
- Require review of past performance calibrated to review by instructor or expert.
- Incorporate peer reviews of performance only with instructor training and multiple observations.
As an instructor who has fought the tide by introducing “desirable difficulties,” I can attest to students’ resistance. They are far more comfortable with what Dunning called “massed training”:
The method that most effectively promotes the rapid acquisition of knowledge and the highest levels of proficiency at the end of the lesson—and thus the appearance of learning—is the one that ensures that whatever is learned in the classroom will be forgotten rapidly. The method of instruction that produces these effects is massed training, in which instructors train students in one or a few intense sessions. Massed training has advantages. Students undergoing intense training quickly obtain the relevant skill and then display it at a high level.
So massed training means writers remain amateurs in the long term.
The Dunning-Kruger Effect can be counted as one of the obstacles to widespread adoption of plain language writing in the workplace. Individual writers overestimate their writing skill so they’re unwilling to learn how to write differently — especially because crafting a plain language document takes so much time and effort. And those of us who devote our efforts to moving amateurs toward pro writing status by teaching with desirable difficulties, have to accept that amateurs may well be unhappy with us!
Related articles
- Amateurs suffer from too much AND too little knowledge (proswrite.com)
- Why hasn’t plain language become the norm? (proswrite.com)
- The Cost of Good Legal Writing . . . Worth It? (thelegalreformer.com)
No one knows what they don’t know; in design these are called the “unknown unknowns” and this is not an amateur’s problem. Even the greatest designers do not know what they do not know (and this is why techniques have been devised to attempt to unravel these “unknown unknowns” so that people can figure out what they don’t know). There is no reason to believe that writers are immune to this.
So I think the title of the blog post is just a little too sensationalized, even though I have little to disagree with the article itself.
By the way, I have to say reducing feedback can go too far. When this happens it can interfere with the student’s learning process in *other* areas.
It’s not the idea that you don’t know what you don’t know, but that you don’t know. And that is an amateur problem. It goes along with the old saying that “a little bit of knowledge is a dangerous thing.” The idea in the original article shows that people who only have limited understanding of a subject think they know more than they actually do. They aren’t even aware that there is anything else to know about the subject, so they imagine themselves to be skilled or knowledgeable about that subject. As the original article termed it, “unskilled and unaware.” Once they gained more knowledge, they were able to finally see the scope of what they didn’t previously know.
It is not. Everyone have things they don’t know and aware of (the “known unknowns”) and things they don’t know they aren’t aware of (the “unknown unknowns”). I’d even say that the refusal to acknowledge that you don’t know some things are signs of your being an amateur.
I intended the underlying “that” not “what” in “they don’t know” as described by Eleanore. That’s the essense of the Dunning-Kruger Effect.
“I’d even say that the refusal to acknowledge that you don’t know some things are signs of your being an amateur.” Well yes, that’s the point. People/Amateurs don’t admit to not knowing because they aren’t even aware that there is something else to know. And that is a problem.
Yes, “people/amateurs.” It is people and not just amateurs. (And if we take “pros” to mean anyone who have written for 5 years there will a plenty of pros who don’t know.) I have nothing against the content of the article, only its title.
But it’s not all people. Just amateurs. You even defined the word “amateur” that way yourself.
Perhaps I should have written “being amateurish” and not “being an amateur.” I’m not a morning person.
In any case I did say this is not an amateur problem. This affects pros too.
Ambrose, as I’ve written before (see http://proswrite.com/2012/09/13/who-deserves-to-be-called-a-pro/), I’m using the terms “pro” and “amateur” with technical meanings.
•Pro Writer refers to people who have been writing in the workplace for something like five years, know all of the basics, and are comfortable creating messages as a key element in complex projects. People in their work group come to them with questions about writing. There are probably no more than 10,000 pro writers in the world and several levels of expertise higher than this.
•Amateur Writer refers to everyone who has not reached the level of a pro (yet).
A pro must have all of these characteristics — not simply five years of experience.
Then the title is even more wrong than I thought. If “Pro Writer” means all this (i.e., NOT what people normally consider “pros”), then it means A LOT OF “amateur” writers will know they don’t know (as well as things they don’t know they don’t know—to me the two are just different way to saying the same thing).
BTW I’d say this particular definition also artificially decreases the number of people who should have been qualified. There are, you know, work situations where “people come to you about writing issues” will NEVER happen. That means that will be a fair number of people who will never be called a pro according to your definition even though their writing skills are probably on par with or exceed those 10,000 that you quote.
The Dunning-Kruger effect applies to ALL people. It’s a cognitive bias that affects “professionals” as well, in that they tend to OVERestimate the ability of others. Copies of the study are available online. This has fairly far-reaching consequences for academia.
Also, not to belabor the point that Ambrose is making, but technically we’re all amateurs at something. A physician, for instance, might be affected by the Dunning-Kruger effect in that they might have difficulty telling when their area of expertise ends, necessitating another specialist.
Thanks for contributing, Peter. I went back to Kruger and Dunning’s 1999 article because I didn’t remember seeing any results or discussion about perceptions of other people’s skills and knowledge. I didn’t find anything. If you’re referring to their later work, please let me know where to find it.
You correctly note that the Dunning-Kruger Effect applies to all of us. But it’s important to recognize that my focus at Pros Write is on a single area of skill or knowledge: writing in the workplace. So, while an individual may achieve “professional” status in medicine, she may not have achieved the same status in writing. I defined “professional” in this post: proswrite.com/2012/09/13/who-deserves-to-be-called-a-pro/.
I just located a very recent discussion by Dunning in which he clearly states that people are more accurate in judging other people’s behavior than they are with their own. See http://www.thepsychologist.org.uk/archive/archive_home.cfm?volumeID=26&editionID=226&ArticleID=2282