A comment on What counts as good advice when writing a sales letter led me to today’s post about evidence. (Thanks, Ambrose.) The quality of evidence tells me what I should believe — what we know — about sales letters. I’ll do my best to avoid heady philosophy here, but paying attention to knowledge means we’re dipping into epistemology.

Don’t worry! The word, epistemology, is just philosophy jargon for “studying how we know stuff.” The Stanford Encyclopedia of Philosophy, a marvelous resource for anyone interested in philosophy, contrasts unjustified beliefs and knowledge:

Beliefs arise in people for a wide variety of causes. Among them, we must list psychological factors such as desires, emotional needs, prejudice, and biases of various kinds. Obviously, when beliefs originate in sources like these, they don’t qualify as knowledge even if true. For true beliefs to count as knowledge, it is necessary that they originate in sources we have good reason to consider reliable. These are perception, introspection, memory, reason, and testimony.

So the key to what we know has to do with evidence. I distill this for my undergrads writers by discussing the following properties of evidence. (See Chapter 4 in the third edition of Revising Professional Writing for more.) Let me use these to explain why I chose evidence from genre research over evidence from an advertising professional’s personal experience for advice about sales letters.

• Authoritative Evidence: This can refer to the person who supplied the evidence or to the methods by which the evidence was found.

Genre research wins out over the advertising pro’s personal experience. Both sources of evidence come from people with authority: the researchers over texts in general, the pro over advertising practices. But the methods used in genre research are more authoritative than personal experience. More about this under the following properties.

• Verifiable Evidence: This relates to the amount of information available to assure consistency of evidence across different observers.

Evidence from research wins over personal experience. From Bhatia’s first study, research articles about the sales letter provide a detailed analysis of specific letters in such a way that others can replicate that analysis. In contrast, the details behind the professional’s conclusions are unavailable to us. We have no way to verify that our own conclusions would be consistent with his given the same experiences. Research is authoritative, in part, because it is verifiable.

• Representative Evidence: This means evidence based on the common case is preferred over that based on exceptional cases.

Evidence based on research wins once more. On the one hand, my advice was based on genre research completed by different individuals on multiple sales letters from different organizations and cultures and published in a variety of journals. On the other hand, the pro’s advice was based on one individual’s experience — that experience might have been exceptional rather than common. Research is authoritative, in part, because multiple cases make it more likely conclusions are based on what is common to all sales letters rather than exceptional about specific sales letters.

• Complete Evidence: This refers to avoiding half-truths by presenting all of the evidence.

Again, research provides better quality evidence than personal experience. Genre research is more complete because it is more representative and also because it appears in peer-reviewed journals. This means research conclusions have been vetted by people not involved in reaching those conclusions. Part of the peer review process mandates attention to alternate conclusions based on other published research and other analyses of the data presented. Once more, research provides more authoritative evidence than personal experience because it’s more likely to be complete.

• Timely Evidence: This means more recent evidence is preferred.

Genre research wins again. The oldest study I’ve seen was published in 1991, but others have continued to appear in scholarly journals. On the other hand, the alternate advice (the AIDA heuristic) was developed at the end of the 19th century.

• Relevant Evidence: This means evidence that is as closely related to the audience’s concerns as is possible.

I’m willing to concede the professional’s experience wins in this area. When offering guidance for writing sales letters, my concern is that a writer can use it to create a successful letter. AIDA was created for precisely that purpose. In contrast, genre research conclusions were created to describe common practice without an explicit measure of their success. Thus, the personal experience of the professional is more directly relevant as evidence for guidance.

If the use of research-based evidence for making practical decisions interests you, check out
Evidence Soup. The site provides a nice summary of the different arenas (e.g., business, healthcare, education) in which people are making deliberate attempts to base decisions about practical issues on evidence. And that site sent me to a a very recent whitepaper with the same title as this post written by three management researchers in the UK: Sandra Nutley, Alison Powell and Hue Davies. The paper “reviews the extent to which it is possible to reach a workable consensus on ways of identifying and labelling evidence” that can be applied to social policy.

Like me, the authors

privilege research as a way of knowing. The conduct and publication of research involves documentation of methods, peer review and external scrutiny. These features contribute to its systematic nature and they provide a means to judge trustworthiness of findings. They also offer the potential to assess the validity of one claim compared to another.

We all acknowledge other ways of knowing, like experience-based, tacit knowledge built up over time. We are all, of course, free to believe whatever we want. But, when I ask other people to believe I know something, I want them to believe me based on the quality of evidence I can provide. I simply don’t find the relevance of evidence based on personal experience as compelling as the more authoritative, verifiable, representative, complete, and timely evidence based on genre research in the case of sales letters.