Skip to content

Crowdsourced Instantaneous Review: The Peer Review of FOAM?

January 8, 2013

I was talking to my PD about FOAM the other day and mentioned how crazy it was that we have the ability to produce and disseminate content so widely, easily and cheaply. How great it was that this, in some respects, allowed us to unhinge CME from the clutches of drug and device companies and potentially speed up knowledge translation.

He agreed that it was awesome before adding a “but” of wisdom.

“But it’s not peer-reviewed”

That got me thinking. Much of FOAM critically appraises the literature. Other parts offer small tips and tricks. There’s some mentorship and career advice. Still more provides education and expert commentary. But what would happen if someone said something wrong? If they misread or misinterpreted or missed reading some literature and their inaccurate perspective was adopted by unsuspecting med students and residents (staff are omnipotent, aren’t they?)?

Certainly, this could happen in peer-reviewed literature as well, but there’s at least some semblance of protection from it. Here I can say whatever I want and I don’t have to prove how right I am to anyone before I do.

Batman is real.

See?

I think I got my answer to these questions earlier this week. Shortly after I published my recent post on Normal Saline and Ringer’s Lactate I received an e-mail from Dr. Rory Spiegal, an EM resident from Newark, NJ. He respectfully offered his dissenting opinion. Now, I don’t think I posted anything egregiously incorrect in that post, but everyone is entitled their opinion and, after reviewing the same literature and listening to the same podcasts that I had, he came to a different opinion. That’s going to happen.

However, I am somewhat dismayed that the feedback came to me in the form of a private e-mail. From his perspective, he was being polite. He noted that he didn’t want to “overstep” by posting his opinion in the comments. I can understand his perspective and would probably have done the same had I been in his shoes. Additionally, what resident would want to publicly disagree with the FOAM heroes that I referenced like Scott Weingart and Cliff Reid? I sure wouldn’t.

And that’s unfortunate, because I think reluctance to make a comment neutralizes the substitute that FOAM has for pre-publication peer review: 24/7 near-instantaneous post-publication commentary by anyone in the world with an internet connection. This is a tool that would effectively neutralize the spread of inaccurate information.

I realize that the idea is not novel: Wiki sites offer a way to correct their errors and create content through crowd-sourcing – while incorrect information can get there, it generally does not stay for long. However, I had not thought or read about the lack of pre-publication peer-review in the context of FOAM before now (although I expect comments linking to others that have already discussed this).

In conclusion, I think it is extremely important that these comments need to be fostered. If the people reading my blog do not know that their dissenting opinions will be thoughtfully considered and any errors that I make will be publicly corrected, they might not post them. And I think that we’d all be poorer for that lack of interaction.

As I clearly think it is important for dissenting opinions to be heard, I asked Dr. Spiegal if he might be willing to post the thoughts that he shared in his e-mails. He responded positively, but I didn’t think it would do his well researched and thought-out opinions justice to simply add them into the comments section of a post. And so they will get a post of their own – look for it within the next couple of days.

Now… if I’m wrong about all of this, don’t hold back! Right or wrong, I want to get it straight – so tell me about it!

As always, thanks for reading! If you enjoyed this post I’d be extraordinarily grateful if you followed me on WordPress or twitter, forwarded it on, tweeted it or linked to it.

Brent Thoma @boringem

Advertisements

From → EBM & FOAM

14 Comments
  1. Are textbooks peer-reviewed?

    • Danica,

      Sorry for leaving this comment to languish without a reply for so long! I thought I had.

      It’s an excellent question and I’m not completely clear on the answer. My understanding is that a ‘peer-review’ is essentially done by the editors of a textbook. However, I imagine this might be vary based on the textbook (a textbook written by a very small number of people, for example) and that certainly wouldn’t meet the peer-review standards for a Journal.

      This was definitely not an evidence-based answer. Hopefully someone else will come along who knows better than me… Or tweet it! It’s something I haven’t mentioned in the discussion on peer-review for FOAM.

      • Haha yeah I didn’t know either. Your reply is what I also would have speculated.

        I suppose the greatest peer review of all would be having one’s name attached to it. I can’t imagine someone spreading information that they hadn’t done their due diligence to research if their name (and reputation) depended on it.

      • Except for Dr. Oz. He doesn’t seem to care.

      • hahaha 🙂

  2. Excellent post. This topic has been on my mind since I started following FOAMed sources, including podcasts and blogs, 2 years ago. With the explosion of resources online the issue becomes ever more important. I intended to put up a post of my own but haven’t quite gotten around to it yet and you have done a great job here.

    Certainly it is a valid argument to say that blogs, etc fail to meet the traditional peer review process. Of course it is possible, and even likely, that “incorrect” (but really, how much in medicine do we know to be truly “correct” or “incorrect” rather than different shades of uncertain truth) information can be posted on the web.

    But there are several other factors involved in the critique of FOAMed. For one, how truly rigorous or fair is the traditional peer review process in medical journals, fraught with bias/influence from the pharmaceutical giants? Further, the traditional peer reviewed articles are often dominated by one or two experts in a field. The more opinions the better, as a variety of perspectives make the discussions healthy and provides multiple ideas. Secondly, as you mentioned, there is instant peer review in the FOAMed community. Anyone keen enough to use FOAMed sources should also be keen enough to question what they read/hear. Skeptical about what you’ve learned? Confirm it or refute it on your own and do your own research. The responsibility of the reader is to validate the information on their own. This can be facilitated best by having the blog posters provide references for their work.

    FOAMed is fantastic if used responsibly.

  3. Peer review is no gold standard.

    Direct your colleague to the excellent work of Richard Smith ex-editor of the BMJ.

    For example http://blogs.bmj.com/bmj/2011/11/11/richard-smith-a-woeful-tale-of-the-uselessness-of-peer-review/

  4. Thanks Brent
    I too am guilty of not entering comments when I disagree with some fellow bloggists.

    FOAMed in it’s current state is replacing what we’re already doing at the bedside – giving our thoughts to our learners [sometimes backed up with evidence … sometimes not].

    Peer review isn’t always right either [e.g. Cochrane steroids for Spinal cord injury].

    BUT we as a group owe it to each other to comment on blogs so that, through the online conversation – the reader can decide for themselves.

    I should point out that most of us in the “about” section of our blogs – state who we are and what we’re about – so readers should take note of this [and perhaps use FOAMed to stimulate face-to-face dialogue.

    Keep up the good work.

    Nadim

  5. Hi Brent!!!Firstly my compliments for your blog it’s of a very good quality especially for the topics covered!!!! I think that the possibility to dissent, discuss on topics covered by blogs via comment sections or twitter or gmep is what makes FOAM great. You can’t discuss an article or a textbook and not always those means of education are right….so it’s something sterile……FOAM gives way to discussion….if I disagree about a topic I can make my researches to endorse my thoughts and than post my opinion….than the discussion starts…..and me, the author and everyone who wants to join the discussion can weigh pros and cons of a certain statement, therapy, technique etc…I think that this is amazing even if not peer reviewed!!!!!

    • Thanks for the support!
      I checked out your blog as well – great start, I look forward to reading more.
      And I completely agree – I find I learn as much or more from reading, making and responding to comments.

Trackbacks & Pingbacks

  1. FOAM: A Market of Ideas « boringem
  2. Flipside: A Dissenting Opinion on RL and NS « boringem
  3. A Journal of FOAM?? « boringem
  4. A Review of Systematic Reviews « boringem

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: