The Reference Letter Triple Crown

Interviews for the Canadian Residency Matching Service (CaRMS) are over and they were as difficult as ever. One thing nobody appreciates on the medical student side of the CaRMS equation is how difficult it is for the programs to come up with our rank list. The applicants this year were spectacular and ranking them was more difficult than splitting hairs. Fortunately, the depth of the applicants makes us confident that we will be matching two exceptional people. It would be hard not to with so many great ones to choose from!

In any case, with this year’s CaRMS cycle at an end, it is time I started to focus my mentorship posts on the class of 2014. While the CaRMS website will not open for many months, many of you have or will soon begin your electives and the associated hunt for reference letters. My next two posts will discuss these issues. This post will attempt to define “What makes a spectacular reference letter?” (according to me!) while its sequel will focus on how to get them on electives.

So what makes a spectacular reference letter? Before answering, I’d like to say preemptively that my opinions come solely through my own experience reading nearly a thousand reference letters for a Canadian FRCPC-EM program over the past few years. They are neither evidence-based nor likely to be universally agreed with and are potentially somewhat specific to EM. I’d urge you to slot my opinion in with those of others that you respect, to think critically, and then to formulate the truth-according-to-you.

With that said, the best reference letters I read each year meet the three characteristics of the Reference Letter Triple Crown:

-The Mikey Likes It Criteria
-The Award-Winning Author Criteria
-The Important Persons Criteria

It’s extremely difficult to get a reference letter meeting all three criteria, and unfortunately, even if you do you likely won’t know it. More on that later!

The Mikey Likes It Criteria

Named for the classic 1970’s Life Cereal commercial, this criteria is pretty simple:

They like you a lot and are willing to go to bat for you.

If your referee likes you as much as Mikey likes Life Cereal, they’ll likely be able to convince the others to eat it (err… interview you). These best of these reference letters can be described as glowing. While most still try to make professional, accurate assessments, when a physician sounds like they’d name their first-born child after the applicant, you can tell and take it as a strong endorsement.

While not impossible, this is a hard thing to develop over a few random shifts in the ED. Anecdotally, it seems like referees who have known the candidates extremely well are more likely to write references like this, possibly because they have invested a lot in developing you as a physician over the years. Developing this sort of a relationship is, in my opinion, one of the greatest side-benefits of doing research in EM.

Of course, the glowing letter is at one end of the spectrum. It is extremely rare to see a truly negative reference letter (I do not think that I have), but the opposite of the glowing letter is one that doesn’t truly endorse the candidate as being above the norm for a medical student with a similar level of training and EM career-focus.

The Award-Winning Author Criteria

This is a category that, for good reason, is often completely overlooked by medical students. Unfortunately, as a student that hasn’t read any CaRMS reference letters before, you have absolutely no idea who is good at writing reference letters and who isn’t. However, I can tell you from experience that unfortunately, physicians’ ability to write helpful reference letters vary dramatically! This is best partially illustrated by this hilarious spoof published in the BCMJ here (this is highly recommended for anyone that has spent hours of their lives reading these letters!).

What does a bad letter look like?

The worst ones I’ve seen have been incredibly brief and were written in point form with minimal punctuation. Interestingly, these ones often don’t even say anything bad about the candidate. Rather, it seemed that these referees simply didn’t care to put any effort into writing them. Incredibly non-specific letters are unhelpful as are those that do not comment on all of the areas that CaRMS asks for. While a letter like this doesn’t red flag a candidate (I would actually feel bad for the candidate for unknowingly asking a poor writer for a letter), it doesn’t add anything to their application.

What does a good letter look like?

It’s detailed and specific enough to tell that the referee knew the applicant well without being incredibly long. It comments specifically and critically on the areas that CaRMS requests. In some way, it puts the candidate in context for the reader. Is this candidate a superstar? Amazing? Great? Good? Average? This can be indicated in various ways, from percentiles to the tone of a letter relative to others written by the same referee. If I feel that I get a good idea of an applicants strengths and weaknesses as well as the writers gestalt about where this person fits in their CaRMS applicant class from a letter, I know that it was a good one. To speculate, I would guess that physicians involved in reviewing applications for a residency program are more likely (but not guaranteed) to write a solid letter.

The Important Persons Criteria

The important persons criteria for a spectacular reference letter is almost exactly what you’d think and it seems to be the one that everyone guns for. I think the best definition I could give it is the following:

The extent to which your referee is known and respected in the community to which you are sending their letter.

The reason that this is important is because references carry more weight when they come from people that we know and trust. Note that this does not necessarily refer to the most well-known or celebrated physician you can think of, although many people meeting this definition will also be one or both of those things, and that it can be somewhat location specific.

Some illustrative examples:

The Program Director: Generally, all of the program directors know and respect each other. Additionally, they are all educators that work with residents and medical students so they are able to perform comparative evaluations effectively. Their opinions are taken seriously.

The Chief of the ED: The prominence of this person’s position implies that their opinion should be respected and, generally, it is. However, their letters are likely to have more impact in areas where they work because the people there know them personally (ie – the Chief at an ED in Dalhousie would be better known out East than in Vancouver and their letter would therefore carry more weight at that location).

The Random EM Physician that you happened to get scheduled with a lot: These are still important letters and you should definitely use them. However, they get their importance from their performance on the other Criteria more than this one. The exception to can be these physicians’ local program or where they trained. Their letter will have more impact in places where they are well-known.

Dr. Oz: He’s super famous, but not necessarily respected in our community. If the doctors that review your application agree with this article, they probably aren’t going to give this letter much weight even if he promises that you’ll be a better researcher than Ian Stiell.

Conclusion

The Reference Letter Triple Crown is difficult to attain. There are just too many unknowns for an applicant to know if they’ve done it. However, those that do (and also have a strong application otherwise) tend to be the ones that sweep the interview circuit in even extremely competitive specialties like EM. More commonly, strong candidates have a variety of consistently strong letters with various strength/weakness in each of the Triple Crown domains.

This Important Persons Criteria was listed last intentionally because I think that there is already an excessive and unhelpful focus on it. When gunning for letters from high-profile people, remember that while this may be the only thing completely under your control, it is not the only thing that matters. If your important person doesn’t write great letters or think that you’re that exceptional, hounding them for a reference letter probably isn’t worth it (hounding anyone for a letter isn’t a good idea of course – more on that in the sequel!).

So what’s an applicant to do? Stay tuned next week for advice on electives and getting those Triple Crown Reference Letters. Also, if you’re reading this you’ll likely be interested in reading my CaRMS Interview Trilogy: Pre-Game: Preparing for the Interview, Game Time: The Interviews, and Post-Game: The Rank List.

If you found this post helpful, I’d greatly appreciate it if you share it or leave a comment! The appreciative feedback and endorsement is has kept me writing these mentorship posts. I’d appreciate you offering support by e-mailing this to your friends, sharing it on facebook, retweeting it on twitter, following me on twitter, signing up for e-mail notification of new posts (right column), and/or following my RSS feed (top right corner).

Thanks for reading!

Brent Thoma @boringem