Peer Review – Pointless, Perfunctionary or Practical?

The twitter heaven gates opened today, although they have been building for some time, with postings around the following blog noted in the tweet below

There has been mixed response to this – some quite clear

Some more contemplative

and some amazingly not related in any way shape or form to the #FOAMed discussion but yet highly relevant!

The term scholarship has been used a lot. How do educators prove to institutions that they have been undertaking ‘scholarly’ activity by producing FOAM materials? What is scholarship? Well there are a few key papers

1. Fincher and Work (2006) Perspectives on the scholarship of teaching

2. Boyer (1990) Scholarship Reconsidered

3. McGaghie (2010) Scholarship, Publications and Career Advancement in Health Professions Education (AMEE Guide 43)

(1 and 2 don’t have a pay wall!) But I am struggling to find a definition I really like. Adrian Stanley at the University of Leicester has talked about

“Scholarship is the body of principles and practices used by scholars to make their claims about the world as valid and trustworthy as possible”

The key issue is the quoted need  (paper 1 above) to have peer review as a fail safe to ensure that standards are up held and maintained. Three issues arise for #FOAMed

i. Time

The beauty of anything #FOAMed is that it exists in the realtime of its creator. When it is ready it goes online. There is no delay. Peer review by the very nature of its objectivity requires a period of reflection which delays the product getting to the people who want to see it.

ii. Standards

Peer review is typically based on ‘peers’ judging your work against some implicit or explicit standards and then having those cross-referenced against a third party editor. These standards may vary between journals, grant reviewers or regulators but there is some criteria none-the-less. #FOAMed is  by definition what the user makes of it. If they like it they go back or spread the word and if they don’t, they don’t (and if they really don’t like it then they may tell people they don’t!). But the burden of ‘peer judgement’  is spread across many peers in what some might describe as crowd sourcing. However the open access nature of FOAMed allows anyone to have there say in a fashion that is easily counted via hits, tweets and likes.

iii. Relevance to a new age

When scholarship began the internet didn’t exist. Who would have thought 100 years ago that a musician may have more followers than an entire country (Lady Ga-Ga), who would have predicted that entire university courses may be taught without you physically being in a lecture (Distance Education at Harvard) and who would have believed that a academic conference in Australia may be accessible to anyone in the world (#SMACC2013)

So if I am an institutional director and I want to promote scholarship in my staff. Do I proceed with a system which takes time, may not be accessible to anyone outside my institution, the published beneficial outcomes only read by a small minority and in which there is no social media presence at all?  If educational resources are of poor quality – how do I know?

Or do I promote my staff producing resources which are instantly available to all, may have hits of 1000s and, if popular, are discussed across a spectrum of discussion sites. If they are of poor quality they will not get used.

Academics will continue to discuss peer-review into the next decade

IF #FOAMed is good enough it simply won’t matter

6 thoughts on “Peer Review – Pointless, Perfunctionary or Practical?”

  1. Hi, thanks for taking the time to write this out. Adds some interesting points to the discourse.

    “Why the clamour for peer review in #FOAMed? Peer review sucks.” via precordialthump.

    Yes, traditional peer-review is inefficient and bulky but it seems the discussion going on is not to have a traditional peer-review for FOAMed. The beauty of this innovative era in medical education is that the community can create a new paradigm…what should peer review look for FOAMed…how can it be structured in a way that is efficient, effective, and breaths life into the open access content that is being shared worldwide. Maybe the consensus will be to scrap the whole notion altogether. Time will tell.

    It would be great if the work of FOAMed contributors could be deemed as scholarly by universities and institutions so that the contend could be openly shared in the academic setting. From what I understand (and I could be in the dark, very possible) is that med students, residents, nurses, are using FOAM content on their own time, to supplement what they are already required to read, watch, etc.

    Why can’t FOAM content be used in the academic setting to train future practitioners. If the answer is “because it’s not scholarly” then I think the effort to make it scholarly is worthwhile. With that said, just curious, are there any universities out there that currently interweave FOAMed with their academic content?

    Done rambling.

    1. Hey D:
      Here are a few more papers that might help:
      – Sherbino J, Arora VM, Van Melle E, Rogers R, Frank JR, Holmboe ES. Criteria for social media-based scholarship in health professions education. Postgraduate medical journal. 2015 Oct 1;91(1080):551-5.

      – Sherbino J. Education Scholarship and its Impact on Emergency Medicine Education. The western journal of emergency medicine. 2015 Nov;16(6):804-9.

      – Flynn L, Jalali A, Moreau KA. Learning theory and its application to the use of social media in medical education. Postgraduate medical journal. 2015 Aug 14:postgradmedj-2015.


  2. Pingback: ICE Blog

Leave a Reply