Interpreting clinical trial results through a frame of reference: Why take the time?

While all trials have their limitations, not all randomised controlled trials (RCTs) are designed, conducted, and reported with the same methodological rigour, and particular flaws can compromise the validity of findings^1,2.

In the first article of this series, some key flaws in the design, conduct and reporting of clinical trials were explored, along with their potential impact on clinical decision-making. Although some steps towards improved standards of reporting and transparency have been implemented in recent years^3,4, many issues still remain unresolved^5-7.

While it is well-known that evidence-based practice involves using clinical trial findings to inform treatment decisions for individual patients^1,8-10, knowing that some of the evidence base may be flawed adds further complexity for time-poor healthcare professionals when trying to decide if trial results could inform patient management¹¹.

In this second article of a 3-part series, we explore this predicament faced by healthcare professionals, and outline some potential solutions.

Is it feasible to expect healthcare professionals to be proficient in critical appraisal?

It is often noted that clinicians should practice critical appraisal skills, including risk of bias assessments, as part of evidence-based practice^12-14. Critical appraisal skills are clearly beneficial as part of training in evidence-based medicine – particularly for healthcare professionals with a keen interest and sufficient time to practice these skills¹². However, it may not be feasible to expect practising healthcare professionals to be proficient in these in-depth skills and practice them on a regular basis¹².

Time constraints can be a serious barrier that can prevent healthcare professionals from keeping up to date with new research¹² — not to mention undertaking detailed critical evaluations of original research studies

A frame of reference for a more informed interpretation of study findings

Healthcare professionals can benefit from an understanding – or ‘refresher’ — of key issues that influence the quality, certainty and trustworthiness of evidence, including study design, risk of bias, treatment effect, and precision¹². The ability to interpret an RCT through this frame of reference, and consider its applicability to patients in the real world, may enable healthcare professionals to gain a more informed interpretation of the study’s findings².

Reading the abstract is not enough

Although it may be tempting to rely solely on publication abstracts for clinical decisions — due to lack of time, limited access to full-text articles and limited expertise in critical appraisal^15,16 — this practice can result in an incorrect interpretation of a study and misleading conclusions^17,18

An analysis of over 49 systematic reviews and meta-analyses on RCTs in dentistry reported over 65% of abstracts failed to mention adverse events that occurred with the intervention, 2% omitted reporting on primary outcomes, and over 4% selectively reported positive results and omitted negative results for primary outcomes¹⁷. However, when the full-text paper was read, these proportions reduced to 14%, 0% and 0%, respectively¹⁷.

In addition, multiple groups have reported the presence of spin in abstracts of RCTs, non-randomised studies, systematic reviews and meta-analyses, whereby the benefits of an intervention are over-emphasised compared to the full text^18-21. A two-armed parallel-group RCT tested the impact of spin in the abstracts of published RCTs on clinician interpretation, comparing to their interpretation of abstracts rewritten without spin¹⁸. The results indicated the presence of spin had a significant impact on the clinician’s interpretation of results, particularly in rating the treatment as more beneficial, compared to the version of the abstracts written without spin¹⁸.

It is crucial to remember that the study abstract is only a concise presentation of the key points in a study’s design and results, and is highly restricted by journal requirements and space limitations¹

Secondary sources of trustworthy information

Secondary sources of trustworthy information can be a useful source of information and commentary on clinical trial publications¹². This information may be available through:

• Evidence-based clinical practice guidelines¹²
• Systematic reviews¹², particularly those from credible and trustworthy sources such as the Cochrane Collaboration

Such sources may contain useful information on quality and risk of bias assessments for key clinical trials in particular fields¹². However, secondary sources of trusted information are limited.

It is also important to consider potential and declared sources of bias and conflicts of interest in secondary sources of information. The author(s) and/or experts should include appropriate declarations on conflicts of interest, and disclose any funding from the pharmaceutical industry. Based on this information, and the quality of evidence and commentary provided, the reader can then make an informed judgement about whether bias may have potentially influenced the materials.

What about editorials and expert commentaries?

Editorials and other commentaries can provide insights into the study’s results in the context of other research, and may explore clinical implications and highlight limitations to consider when interpreting study results¹. However, it is up to healthcare professionals to read a publication in detail, considering its complexity, strengths and limitations¹, particularly when using its results to inform clinical decision-making.

Key considerations when reading clinical trial publications: Suggestions for healthcare professionals

A more complete picture of a trial and its findings can be gained by reading the full publication. In particular, results of RCTs should be interpreted within the context of^{1,2,8,9,22,23}:

• The importance of the research question
• The design and conduct of the trial
• The study results, treatment effect, and their clinical importance
• Applicability of the study to real-world patients
• Other pre-existing evidence

These considerations require careful reading of the entire article, including study methods, results and discussion¹. The discussion section will usually describe key study limitations and caveats that are important when interpreting the study’s findings¹.

Up next: Looking under the surface of clinical trials - are the findings valid and relevant to your patients?

To assist healthcare professionals to determine if trial results are of sufficient quality to inform their decision-making in practice, the final article in this series will explore the above key aspects of clinical trials in more detail.

Read article 3 in this series - Looking under the surface of clinical trials: Are the findings valid and relevant to your patients?

Did you miss article 1 in this series? Click the link below to learn more about key issues reported with the quality and accuracy in clinical trial publications, and their potential impact on clinical decision-making.

Read article 1 in this series - How can flaws in clinical trial design, conduct and reporting impact clinical decision-making?

Explore medical education and publication summaries on Medthority

Explore a range of independent and sponsored medical education across therapeutic areas including cardiology, immunology, oncology and more. Upskill on key developments via continuing medical education in various formats including Learning Zones, with expert insights from key opinion leaders in videos and podcasts, as well as webinars, quizzes and more. Medthority Learning Zones also feature publication digests, which provide concise summaries of key publications across different fields.

Explore medical education content by specialty on Medthority