They're all fabulous and wonderful! How to figure out what's real in an inflated letter of recommendation

Over the decades that I’ve worked in universities^[1], I’ve watched academic letters of reference become increasingly inflated. And letter inflation goes beyond the academy; friends who hire in business say they, too, have witnessed reference letter inflation.

Online commentators have noted that writers frequently inflate the positives and don’t divulge the negatives – what they know about a candidate “deep down.” As Joe Schall, author of “Writing recommendation letters online,”^[2] says, “Candor is replaced by gloss, and qualitative distinctions become blurred.” Or as Alison Schneider put it in a much-quoted essay^[3] for the Chronicle of Higher Education, “Puffery is rampant. Evasion abounds.”

What if letter writers supersize applicants, labeling someone who is really average as “good” or possibly even “outstanding”?

Schall remarked that most writers^[4] “feel that if they do not conform to an inflated set of standards within a system where all recommendations are customarily exaggerated,” their letters will unintentionally harm applicants’ prospects.

This could pose a problem when a personnel committee is charged with hiring the most outstanding manager. How can they determine the most outstanding candidate when everyone is described as outstanding? It’s also a problem when academic departments try to identify the most promising scholars, or when professional societies select individuals for awards or promotion.

Lots of letters

For several years I’ve been part of a research team at Cornell University and the Fermi National Accelerator Laboratory, analyzing academic recommendation letters at both institutions.

We’ve examined 2,206 letters written on behalf of applicants for academic positions on various standard linguistic dimensions. Those include words that signal positive or negative tone; notable, standout terms such as “outstanding;” grindstone terms that denote commitment like “hard-working”; and phrases that raise doubt, such as “I will leave to others to comment on X’s research.”

My observations accord with those involved in hiring in other fields. The vast majority of letters portray the applicant very positively. It is rare for a writer to recommend against hiring the applicant – only 1-2% of letters fall below the good-to-excellent range.

It is so unusual to read a negative letter that when one is submitted it raises the question in the mind of the reader that perhaps something personal is going on between the applicant and writer. Is personal animosity behind the writer’s negativity?

Not one letter

This phenomenon is found across many disciplines. Stanford University medical school faculty Anna Messner and Erika Shimahara analyzed 763 letters written on behalf of physicians applying^[6] for hospital residencies. They found that “Not a single (letter) stated that the applicant was not recommended or that he/she should consider another specialty.”

Other published analyses have echoed this finding. One example^[7] comes from Kuheli Dutt and her colleagues at Columbia University’s Lamont-Doherty Earth Observatory. They classified more than a thousand letters written on behalf of postdoctoral applicants into one of three categories: doubtful/unclear, good, or excellent. Over 98% of letters were rated good or excellent – only 31 out of 1,224 were lower than good.

Hiring committees shouldn’t hold their breath waiting for writers to openly profess a lack of enthusiasm.

Searching for implications

In the Dutt study’s data, 76.6% of letters described applicants as above-average or good, reminiscent of Garrison Keillor’s Lake Wobegon^[8], where “all the women are strong, all the men are good-looking, and all the children are above average.”

On the basis of my 42 years reading letters, some current letters that depict applicants as good would have depicted them as fair (or lower) in earlier times.

However, a cultural shift^[9] coupled with fear that if they share unflattering information it will get back to the applicant and result in social shaming or even litigation, has resulted in today’s writers’ realization that if they accurately label an average candidate as average, it is the kiss of death if hiring committees are trying to identify the best individuals.

As Dutt notes, recommendation writers in the past used far greater candor than writers do today. So contemporary readers must probe the text of modern letters for their implied meanings.

In my judgment, amid the blizzard of positive letters, there is a surefire way to identify applicants for whom writers have the greatest enthusiasm.

‘Standout’ vs. ‘grindstone’ terms

It turns out that only 20-25% of letters describe applicants as outstanding.

They do this by invoking the linguistic terms mentioned above: standout terms or phrases, such as “unparalleled,” “one of the best I have worked with,” “incredible,” “fantastic,” “prodigy,” “unrivaled,” “outstanding,” “rising star,” “as good or better than” (a well-known person), “amazing,” and others.

Despite the general inflation of recommendation letters, many studies have shown that only the top quarter (at most) of letters contain standout terms/phrases. The rest are laden with grindstone terms such as hard-working, careful, good, knowledgeable and more.

Granted some employers may want to hire grindstone employees. However, when there is a need to identify the very best applicant, then I offer the following suggestion: Look for standout terms. When the goal is to choose an applicant who is well above average, those depicted only as good are frequently only average.

If writers were provided with adjective checklists, it would be evident whom they viewed as the strongest candidates because they’d check more of the standout than grindstone words. So they could check any of the following CAPITALIZED adjectives that they believe apply to the candidate:

Good, OUTSTANDING, AMAZING, very knowledgeable, UNRIVALED, solid, super careful, FABULOUS, diligent, PRODIGY, hard-working, FANTASTIC.

In my humble opinion, I believe the development of this checklist represents the most outstanding, amazing and unrivaled method to identify the best candidates.

In other words, it is good.

[You’re smart and curious about the world. So are The Conversation’s authors and editors. You can get our highlights each weekend^[10].]

References

1. ^{^} decades that I’ve worked in universities (www.human.cornell.edu)
2. ^{^} Joe Schall, author of “Writing recommendation letters online,” (www.e-education.psu.edu)
3. ^{^} essay (www.chronicle.com)
4. ^{^} remarked that most writers (www.e-education.psu.edu)
5. ^{^} Shutterstock/Sinart Creative (www.shutterstock.com)
6. ^{^} 763 letters written on behalf of physicians applying (doi.org)
7. ^{^} One example (doi.org)
8. ^{^} Garrison Keillor’s Lake Wobegon (www.garrisonkeillor.com)
9. ^{^} a cultural shift (www.e-education.psu.edu)
10. ^{^} You can get our highlights each weekend (theconversation.com)