Peer Review – Pointless, Perfunctionary or Practical?

The twitter heaven gates opened today, although they have been building for some time, with postings around the following blog noted in the tweet below

There has been mixed response to this – some quite clear

Some more contemplative

and some amazingly not related in any way shape or form to the #FOAMed discussion but yet highly relevant!

The term scholarship has been used a lot. How do educators prove to institutions that they have been undertaking ‘scholarly’ activity by producing FOAM materials? What is scholarship? Well there are a few key papers

1. Fincher and Work (2006) Perspectives on the scholarship of teaching

2. Boyer (1990) Scholarship Reconsidered

3. McGaghie (2010) Scholarship, Publications and Career Advancement in Health Professions Education (AMEE Guide 43)

(1 and 2 don’t have a pay wall!) But I am struggling to find a definition I really like. Adrian Stanley at the University of Leicester has talked about

“Scholarship is the body of principles and practices used by scholars to make their claims about the world as valid and trustworthy as possible”

The key issue is the quoted need  (paper 1 above) to have peer review as a fail safe to ensure that standards are up held and maintained. Three issues arise for #FOAMed

i. Time

The beauty of anything #FOAMed is that it exists in the realtime of its creator. When it is ready it goes online. There is no delay. Peer review by the very nature of its objectivity requires a period of reflection which delays the product getting to the people who want to see it.

ii. Standards

Peer review is typically based on ‘peers’ judging your work against some implicit or explicit standards and then having those cross-referenced against a third party editor. These standards may vary between journals, grant reviewers or regulators but there is some criteria none-the-less. #FOAMed is  by definition what the user makes of it. If they like it they go back or spread the word and if they don’t, they don’t (and if they really don’t like it then they may tell people they don’t!). But the burden of ‘peer judgement’  is spread across many peers in what some might describe as crowd sourcing. However the open access nature of FOAMed allows anyone to have there say in a fashion that is easily counted via hits, tweets and likes.

iii. Relevance to a new age

When scholarship began the internet didn’t exist. Who would have thought 100 years ago that a musician may have more followers than an entire country (Lady Ga-Ga), who would have predicted that entire university courses may be taught without you physically being in a lecture (Distance Education at Harvard) and who would have believed that a academic conference in Australia may be accessible to anyone in the world (#SMACC2013)

So if I am an institutional director and I want to promote scholarship in my staff. Do I proceed with a system which takes time, may not be accessible to anyone outside my institution, the published beneficial outcomes only read by a small minority and in which there is no social media presence at all?  If educational resources are of poor quality – how do I know?

Or do I promote my staff producing resources which are instantly available to all, may have hits of 1000s and, if popular, are discussed across a spectrum of discussion sites. If they are of poor quality they will not get used.

Academics will continue to discuss peer-review into the next decade

IF #FOAMed is good enough it simply won’t matter

Why do you do what you do?

This blog actually appeared in its first form on the The-Network Blog site (well worth joining this free initiative if you are interested in quality improvement and health system leadership and management – there are 2000+ other members!). Its posting here was prompted by the following tweet

So writing back in July 2012…..

At the end of last week I attended the International Conference on Emergency Medicine (#icem2012) in Dublin. Like many conferences the benefits of attending (meeting friends and networking) outweighed the costs (exorbitant registration and travel) but resulted in very little practical knowledge gain.

One lecture particularly stuck in my mind and has re-shaped my enthusiasm for medicine. The speaker, from America, was introduced as a giant in the field of Emergency Medicine and an expert in paediatric emergency care. He was speaking on the topic of “Neonatal Emergencies”. About half way he started talking about a 5 day old presenting to the Emergency Department with Jaundice. His slide set finished with the comment – “stopping breast feeding can be used to confirm the diagnosis of Breast Milk Jaundice”. I have rarely been so angry in my entire life. This is not far off saying “to confirm that people get dehydrated don’t let them drink”. Breast milk jaundice is a physiological process which does not need confirming (other causes of jaundice need excluding if you are unable to do this clinically).

In the middle of the lecture I started waving my hand frantically in the air and stopped when I realised people were looking at me strangely (although this is not the first and last time that will happen). I was the first to put my hand up for questions at the end of the talk and politely asked if I had misheard the speaker in their assertion that stopping a normal process to confirm a diagnosis of no practical relevance was a useful medical intervention. The reply included a denial of being in the pay of a formula manufacturer (something I hadn’t been concerned about but now was) and the fact practices varied so discussion with the family should always take place. I remain perplexed that even in America this could be deemed a suitable practice and was relatively reassured by the number of delegates who came up to my afterwards to agree with my concern. I was also equally horrified that a number of non-paediatric emergency physicians were dutifully scribbling down every word.

Recently I have taken on a little too much and my enthusiasm for the clinical side of my work has waned. I have been reflecting on what matters most to me and which direction I should be taking. Clinical credibility has now firmly been planted back into my life plans and I never wish to become so distant from actual clinical practice that I lose sight of fundamental principles.

I am sure I said at some point in my Medical School interview I came into medicine to help people but this also includes helping my colleagues understand bad practice. On reflection this is what I do with my research, representative and leadership roles and is actually what really drives me forward.

Why do you do what you do?

Hijacking Hierarchies: A potential and a peril of social media

Do you remember a time before facebook? There must have been an internet, and there were probably even blogs, but being popular meant a lot people would turn up for drinks at your birthday party. Since social media has taken off there has been an insidious introduction of more formal popularity measures. You have friends on facebook, hits on wordpress, followers on twitter – all potentially irrelevant information but a constant objective ‘measure’ non-the-less. I have mulled over this as on christmas eve a twitter posting (which frustratingly I forgot to favourite) stated the best leaders would concentrate on their families, not new followers, over the holiday period. I am not sure how many people actually look for followers on twitter (apart from the really annoying spam you receive) but the most popular tweeters (in terms of followers) are often clearly not concerned about ensuring wide appeal from their tweeting. However their does seem to be an increasing obsession with measuring popularity on social media. A number of social media ‘personality’ awards now exist. What purpose do these serve? Do those on twitter or facebook actually need reminding who they are all following? They certainly don’t affect those outside social media as they aren’t even on it. A ‘mercury music prize’ equivalent might be more reasonable with up and coming tweeters celebrated . Ultimately though however popular the Mercury might be if you don’t listen to music it probably doesn’t mean much to you! More formal measures of popularity exist (klout and others),  there is some science (seeking influence) and I have always liked [log(number of tweets)*(followers/following)]. So far major healthcare organisations have resisted this ceremony but might we one day see a British Medical Journal #SoMe award…

So is there any reason to continue supporting such narcissism? Can we see a day when gongs may be won on the basis of influence via social media. Well there might be one. If there is one great advantage to the explosion of SoME is the complete breakdown of hierarchy and flattening of communication channels. Who could have thought 5 years ago you could contact directly the chairs of the Royal College of General Practitioners (@clarercgp) , the president of the royal college of paediatrics and child health (@rcpch_president) or the chief executive of NHS Employers (@NHSE_dean) . Just follow their twitter lines to see examples of trainees and colleagues posing questions directly and getting responses. In the short term the promotion of individuals who use twitter (and other tools effectively) may draw attention to this brilliant engagement opportunity. In fact if objective measures of influence, rather than popularity, can be found it may promote greater involvement of organisations who have up to this point resisted dipping their toes in the water.

Ultimately popularity, whether relevant or not, will always be measured. It is now up to those on social media to decided on what the most constructive use of this is.

Am I productive? A triad of system barriers

A recent twitter posting prompted a discussion which took on a number of directions

The resulting discussion can be found here. The concept of productivity caused me reflection during and after the twitter chat; in particular on the theme of individual performance.

Obviously in the big scheme of things the number of patients I see on any given shift has absolutely no bearing on the overall productivity of the NHS. But is my own work rate or output in any way correlated? And more importantly are the metrics themselves even comparable?

What is productivity? Wikipedia describes it as thus:

Productivity is a measure of the efficiency of production. Productivity is a ratio of production output to what is required to produce it (inputs). The measure of productivity is defined as a total output per one unit of a total input.

Is this in anyway meaningful for the health sector? The Kings Fund (@thekingsfund) give a range of possibilities but I am still left asking: Am I productive?

I think back to my last weekend shift. I work in a busy Paediatric Emergency Department which links to a tertiary children’s hospital. I think of myself as having been productive if I see x number of patients (for me x is 20 but I am not sure the number is transferrable as a comparison measure). Although the actual case-mix is very variable virtually every weekend contains a number of specific events/presentations which has a large bearing on my self-productive rating (and sense of achievement by the end of the shift).

  • The state of the department prior to my arrival. A weekend shift runs from 12pm-10pm and there is a back log of reviews to clear (I am a relatively senior registrar) even before I can see new patients. Any productivity I bring in respect of patient decisions or interventions is not additional benefit it is based on prior inputs (or patient presentations in a health care sense) to my arrival. Is it efficient to potentially reduce my de novo productivity at the outset of my shift? 
  • The number of emergency cases (defined as patients requiring immediate, potentially life saving, intervention). A prolonged resuscitation requires multiple resources and, regardless of the efficiency of the team, drains time from seeing other patients.
  • The number of complex non-emergency dischargeable (CoNED) cases (bear with me on this…). Appreciating the subjective nature of ‘complex’ these are cases which do not immediately  fit a pattern which an experienced health care professional would recognise.  I suppose it is self evident that lots of complex cases will require more time and therefore less patient turnover per clinician. However in an emergency department once it is clear a patient needs to be admitted you become less productive if you spend effort utilising resources that could be done by the inpatient team. Conversely from a patient perspective there are some investigations or managements if commenced early save time later in the patient journey. So there is a balance between ‘fast tracking’ and the overall length of stay. Additionally you must also have insight that this is a complex case; often reviews on patients on behalf of juniors reveal patients who were thought not be complex but in fact are (and vice-versa!). Ultimately the ability to manage a complex case requiring admission is a skill which improves with experience and I am not sure affects my overall productivity . However if a complex patient does not need to be in hospital this may be particular time leeching. From a 4 hour target perspective (see @drmarknewbold‘s brilliant blog on this) it may well be easier to admit but this is not always in the patients best interest. The number of CoNEDs is in part a function of the success of modern medicine and the ability of health services to provide effective and prompt follow-up.

I am very happy to concede that the number of patients seen is not a brilliant metric for productivity and that emergency medicine is only a small part of the NHS workload. I believe, the triad of, the current capacity of system, the number of serious cases and the number of complex cases not requiring referral to another provider is an important factor in determining productivity. The solution therefore may not depend on the individual. That will certainly not stop me working very hard to get x as high as it safely can be.

#APEM 2012 Highlights

Thanks to the efforts of Dr. Mark Lyttle (@mdlyttle) APEM 2012 proved to be a resounding success. Here is a selection of some of the hot topics, mainly via the superb tweets of Dr. Natalie May (@_nmay) to fill those in who weren’t there and prompt further discussion and debate. A more detailed twitter feed can be found on my Storify site for Day One and Two and all the presentations will shortly be available via apem.me.uk. The links within the tweets should all work (let me know if not!)

1. Dr. Nick Sargent “Anaphylaxis – an evidence based update

Not something I had really considered and wonder if I have ever missed this. It does appear studies on adrenaline versus salbutamol for acute asthma have taken place fairly recently http://www.ncbi.nlm.nih.gov/pubmed/16490653

Useful to  consider how your local allergy/anaphlaxis pathways ensure suitable follow up?

2. Dr. Anne Frampton “PEM Training Update

Although not directly related to the theme of the talk this is causing a lot of concerned conversations

Has your unit fully implemented toxbase guidance (sorry can’t link as password protected) that children should have bloods at 75mg/kg cut off? This technically means the calpol bottle glugger may need investigations when previously they could have gone home. It’s not entirely clear what consultation occurred before this change but consensus was this will result in unnecessary tests.

3. Dr. Mike Clancy “The future of Emergency Medicine

Mike Clancy emphasised the need for departments to take the bull by the horns in engaging with the new world of commissioning, especially with LETBs . The workforce crisis has been taken on board by the DOH but solutions will not happen overnight.

4. Prof. Ronan O’Sullivan “Paediatric Procedural Sedation – an evidence based approach

Ronan O’Sullivan has sent up an extensive curriculum around paediatric procedural sedation, in which consent must be obtained even for Nitrous Oxide. The reason being that the mindset created around consenting ensures the proper respect is shown to the procedure. It was great to see some anecdote being supported by other APEM delegates

5. Prof. Adam Finn “The impact of new vaccines in Paediatric Emergency Medicine

So a number of vaccines will shortly be available in the UK – rotavirus from next year and a flu vaccine. The effects on Paediatric Emergency Departments potentially may be profound. Add in the addition of Men B (potentially) and you are left wondering what we all might be doing in a decade! Some food for thought…

6.  Dr. Natalie May and Dr. Damian Roland “This house believes paediatric Emergency Medicine in the UK would benefit from more doctors in the UK being active on mainstream Social Media

The against argument is available here

7. Dr. Lisa Munro Davies “Is there a role for ultrasound in Paediatric Emergency Medicine?”

Utilising Ultrasound in Paediatric Emergency Medicine is an inevitable progression as technology advances but the true overall utility has yet to be defined. There was much discussion about the best methods of gaining, developing and maintaining skills. What was clear is the Paediatric Emergency Medicine community would like to be masters of their own destiny in this regard.

Day Two

8. Dr. Anne Kerr ” Should we use Tranexamic Acid in Paediatric Trauma

TXA has a good safety profile in paediatric surgery but despite the large amount of patients in CRASH-2 we don’t have the paediatric data to know when and in which category of patient to most effectively give it. The RCPCH guidance should promote its use.

9. Dr. Catherine Bevan “Paediatric Cervical Spine Injuries – a pain in the neck?

An interesting conundrum – true C-Spine injury astonishingly rare but consequences of missing potentially catastrophic. A sharp mind ad flexible thinking required.

10. Dr. Ffion Davies “Paediatric Trauma Networks: the national picture

It was noticed that whatever national system is put in place there remains not an insignificant number of patients who present with high trauma scores who are brought directly to Emergency Departments by their parents….

11. Dr. Simon Chapman “Simulation in Practice

Simulation continues to expand but the traditional APLS model is increasingly becoming replaced by more immersive scenarios. Key message was importance of debrief and need for role credibility to be maintained i.e. play the role you actually are!

Thanks for reading!

Remember the consciously incompetent: defining what Social Media is and isn’t

At #APEM2012 Dr. Natalie May (@_nmay) and I gave a talk entitled:

“This house believes Paediatric Emergency Medicine in the UK would benefit from more doctors being active on mainstream social media”

Natalie was “for” and I was (for the sake of the talk) against. Natalie did an excellent presentation (without bullet points and only pictures – @ffoliet would have been proud) and I responded with no audiovisual media (apart from the nativity social media you-tube video) to strengthen my argument (the transcript of the talk and the video can be found here).

A number of unexpected questions and responses came up which I think are useful to share as it is important that social media, #FOAMed and other potentially extremely helpful learning media are not tarred with the wrong brush!

1. Social Media is a concept not one thing (twitter is utterly different from facebook which in turn is not blogging).

2. Facebook ‘scares’ have done the reputation of Social Media significant harm. Health care professionals sticking naked pictures of themselves on their own personal sites does not mean that everyone must follow suit.

2. #FOAMed is a concept that may span many types social media, is certainly not contained by it, and can simply be information via a website…

3. There is great concern that not enough material is quality assured, “…but what if the information is wrong?” we were asked. I found this interesting as it is up to any clinician to decide themselves the quality of information they receive. Does reading one journal article mean you go out and change your practice completely? Do you not weigh up the information, discuss with colleagues and seek other sources of evidence? #FOAMed represents a methodology of the best content percolating upwards to the attention of  interested clinicians. Does everything need to come with a guaranteed evidence based seal of approval? Certainly not in my opinion – what better way to slow things down and stifle debate.

4. Consent and Patient confidentiality cause anxiety, “…but how do you talk about the most interesting, and therefore usually identifiable cases.” This is a very valid concern and has been debated previously (see the comments section). This is not an easy question to answer but in some ways is similar to [3]. In itself not a reason not to engage in the multitude of learning resources out there. Time, experience and legal testing will enable the public and regulators to determine what is appropriate or not but currently there is clear guidance on consent from the GMC (and other healthcare organisations around the world).

I am left with the feeling that the most engaged in developing education 2.0 need to remember they are unconsciously competent and that some work needs to be done to reach the consciously incompetent.

An alternative view of social media

The following is a transcript of the ‘against’ argument at #APEM2012 (www.apem.me.uk)

“Paediatric Emergency Medicine in the UK would benefit from more doctors on mainstream social media”

So let us be clear what we are talking about here. We are talking about social media – the facebook, the linkedIN , the twitter. All the things your children, nephews and nieces get involved with.

Who here has never used any of those things?

Excellent – well done. Clearly hasn’t done you any harm.

Those who support such things would have you believe that by being part of, what is it called  “ a global phenomenon”  that you are somehow missing out.  Missing out on what exactly  – arrogant, self-obessed individuals keen on knowing how many followers they have. We are paediatric emergency medicine specialists; we treat dying children. We don’t follow – we lead! Our knowledge if very valuable – why share it with a community of people you have never met. Much better if it stays within the four walls of the hospital you work; you have the best approach. You must do – no-one has challenged it for it years!

And lets say for the sake of argument I did want to dabble in the nonsense. Who has time for it? With all the google searching, trying to find the one paper you read 5 years ago in a dozen similarly labeled folders, putting together presentations from scratch how can you possibly have any PAs (that sessions for those of you in the US)  left.

And then clearly you will get sued! Remember that chap from Wales.  Posted something along the lines of “don’t like working in birth sheds, prefer the cabbage patch. The public were a little aggrieved when they found out he was talking about obstetric units and ICUs.

This shouldn’t really be a path we go down. A study from the US of 600 staff involved in the admission process for medical schools and residency programmes found that 2/3 were familiar with researching individuals on social networking sites. Furthermore, over half (53%) agreed that online professionalism should be a factor in the selection process and that “unprofessional behaviour” evinced from wall posts/comments, photos, and group memberships should compromise an applicant. Clearly the racous behaviour that the PEM community get up to on the evening of the APEM meal may well count against them in the future? I suppose it was re-assuring that only a small proportion (3-4%) said they used the information they found to reject a candidate.

Although this maybe only a US phenomenon. Jared Rhoads, senior research analyst with CSC’s Global Institute for Emerging Healthcare Practices, said feedback on therapies is one of the most valuable uses for social media — and possibly one of the easiest to facilitate (via amednews).

“If 10,000 people start talking about a side effect of a drug, it won’t be that hard to find that out,” he said. Really???

So basically if you are too busy, already know your stuff and what stay in the ‘real world’ social media is not for you

DISCLAIMER: Dr Damian Roland does not believe a word of the above 😉

%d bloggers like this: