Research

New publication: Grading evidence from qualitative research

1 minute read

I’m pleased to announce the publication of a new paper in the journal RMD Open with colleagues from a EULAR working group on the integration of qualitative evidence into EULAR’s guideline and recommendation processes.

This paper reports the work of the group in identifying existing systems for rating and evaluating qualitative evidence, based on a systematic review. The EULAR team kindly invited me to participate in the research project, advising on methodological issues relating to the use of tools such as evidence hierarchies, as well as feeding in on the results of my own previous review of hierarchies of evidence, which identified a small number of evidence hierarchies which include qualitative evidence.

It was an interesting experience to work with this EULAR group, and gratifying to see the dedication within that team to broadening the range of evidence admissible in their guideline development processes. International collaborations like EULAR are, of course, quite slow-moving beasts, but the incorporation of qualitative evidence into their processes is a worthwhile task to undertake.

I hope in the near future to follow up this publication here with some more detailed reflections on the role which philosophers of medicine can play within this sort of project, and some of the challenges inherent to the idea of ranking and rating qualitative evidence. The context of evidence grading and evaluation for qualitative research throws some of the underlying assumptions of both hierarchical and non-hierarchical evidence rating systems into sharp relief, and I think it has much to tell us about how those tools function (or malfunction).

For now, I present:

Sekhon, M. et al. (2024) ‘Synthesis of guidance available for assessing methodological quality and grading of evidence from qualitative research to inform clinical recommendations: a systematic literature review’, RMD Open, 10 (2). doi: 10.1136/rmdopen-2023-004032

More in Research

  1. The Pyramid Schema: The Origins and Impact of Evidence Pyramids

    [This paper has been updated in 2026 from the 2022 preprint] Evidence pyramids are amongst the most recognisable artefacts of the Evidence-Based Medicine movement. Yet no study has established the origins of evidence pyramids, or analysed whether they offer any information beyond simple lists or tables. In this paper, I establish the origins of the first evidence pyramid and argue that the pyramidal turn is a retrograde step in evidence appraisal.

  2. Cultural Bias in Language Models: The Top 10 Test

    How can AI-generated "Top 10" lists of cultural influencers expose the cultural biases and defaults that are built into our most popular language models? I propose and report an initial test using lists of the Top 10 most influential musicians.

  3. Golem.AI: The Experimenter's Regress and AI Decision Systems

    How can the concept of the experimenter's regress help us understand the problems in AI decision systems? I explore two forms of circularity that can underpin AI decision tools through the lens of Collins and Pinch's 'The Golem'.