Not very. I read about some interesting research last spring and meant to write more about it then. Here’s the bottom line. The researchers evaluated 9 of the most commonly used formulas. Here’s how Willingham summarizes their findings:
All of the readability formulas were more accurate for higher ability than lower ability students. But only one—the Dale-Chall—was consistently above chance.
Only the Dale-Chall predicted reading ability better than flipping a coin. Ouch!
In general, writing experts dismiss the utility of readability formulas as a means of improving the quality of writing produced in the workplace. I mentioned the range of methods for evaluating quality before. The ACM Journal of Computer Documentation dedicated an entire issue to the topic of readability to usher in the 21st
millennium century (Volume 24 Issue 3, Aug. 2000). I’ve copied their table of contents with live links and available abstracts for those of you with an interest.
Table of Contents
|Introduction to this classic reprint and commentaries|
|The measurement of readability: useful information for communicators|
|George R. Klare|
|Readability and computer documentation|
|Traditional readability concerns are alive and well, but subsumed within several more recent documentation quality efforts. For example, concerns with interestingness and translatability for global markets, with audience analysis and task sufficiency, and with reader appropriateness of technical text all involve readability, but often in ways not easily measured by any formula.|
|Readability formulas have even more limitations than Klare discusses|
|A literature review reveals many technical weaknesses of readability formulas (when compared to direct usability testing with typical readers): they were developed for children s school books, not adult technical documentation;they ignore between-reader differences and the effects of content, layout, and retrieval aids on text usefulness; they emphasize countable features at the expense of more subtle contributors to text comprehension.|
|Readability formulas in the new millennium: what’s the use?|
|Karen A. Schriver|
|While readability formulas were intended as a quick benchmark for indexing readabilty, they are inherently unreliable: they depend on criterion (calibration) passages too short to reflect cohesiveness, too varied to support between-formula comparisons, and too text-oriented to account for the effects of lists, enumerated sequences,and tables on text comprehension. But readability formulas did spark decades of research on what comprehension really involoves.|
|Klare’s “useful information” is useful for Web designers|
|In many ways the writing principles that Klare recommended 37 years ago to promote high readability scores still apply to web-site design. Behind the pursuit of readability lies audience analysis, a concern with the intellectual level, previous experience, motivation, and reading goals of ones intended audience. Suitably adjusted to take account of online interactivity, those same concerns should guide design work on web structure and interfaces today.|
|Readable computer documentation|
|George R. Klare|
|Arguing that current approaches to understanding and constructing computer documentation are based on the flawed assumption that documentation works as a closed system, the authors present an alternative way of thinking about the texts that make computer technologies usable for people. Using two historical case studies, the authors describe how a genre ecologies framework provides new insights into the complex ways that people use texts to make sense of computer technologies. The framework is designed to help researchers and documentors account for contingency, decentralization, and stability in the multiple texts the people use while working with computers. The authors conclude by proposing three heuristic tools to support the work of technical communicators engaged in developing documentation today: exploratory questions, genre ecology diagrams, and organic engineering.|