Skip to main content

Table 3 The nine conceptsa included in the IHC podcast

From: Can an educational podcast improve the ability of parents of primary school children to assess the reliability of claims made about the benefits and harms of treatments: study protocol for a randomised controlled trial

Concepts

Explanations

Implications

1.1 Treatments may be harmful

People often exaggerate the benefits of treatments and ignore or downplay potential harms. However, few effective treatments are 100% safe

Always consider the possibility that a treatment may have harmful effects

1.2 Personal experiences or anecdotes (stories) are an unreliable basis for assessing the effects of most treatments

People often believe that improvements in a health problem (e.g. recovery from a disease) were due to having received a treatment. Similarly, they might believe that an undesirable health outcome was due to having received a treatment. However, the fact that an individual got better after receiving a treatment does not mean that the treatment caused the improvement, or that others receiving the same treatment will also improve. The improvement (or undesirable health outcome) might have occurred even without treatment

Claims made about the effects of a treatment may be misleading if they are based on stories about how a treatment helped individual people, or if those stories attribute improvements to treatments that have not been assessed in systematic reviews of fair comparisons

1.3 A treatment outcome may be associated with a treatment, but not caused by the treatment

The fact that a treatment outcome (i.e. a potential benefit or harm) is associated with a treatment does not mean that the treatment caused the outcome. For example, people who seek and receive a treatment may be healthier and have better living conditions than those who do not seek and receive the treatment. Therefore, people receiving the treatment might appear to benefit from the treatment, but the difference in outcomes could be because of their being healthier and having better living conditions, rather than because of the treatment

Unless other reasons for an association between an outcome and a treatment have been ruled out by a fair comparison, do not assume that the outcome was caused by the treatment

1.4 Widely used treatments or treatments that have been used for a long time are not necessarily beneficial or safe

Treatments that have not been properly evaluated but are widely used or have been used for a long time are often assumed to work. Sometimes, however, they may be unsafe or of doubtful benefit

Do not assume that treatments are beneficial or safe simply because they are widely used or have been used for a long time, unless this has been shown in systematic reviews of fair comparisons of treatments

1.6 Opinions of experts or authorities do not alone provide a reliable basis for deciding on the benefits and harms of treatments

Physicians, researchers, patient organisations, and other authorities often disagree about the effects of treatments. This may be because their opinions are not always based on systematic reviews of fair comparisons of treatments

Do not rely on the opinions of experts or other authorities about the effects of treatments, unless they clearly base their opinions on the findings of systematic reviews of fair comparisons of treatments

2.1 Evaluating the effects of treatments requires appropriate comparisons

If a treatment is not compared to something else, it is not possible to know what would happen without the treatment, so it is difficult to attribute outcomes to the treatment

Always ask what the comparisons are when considering claims made about the effects of treatments. Claims that are not based on appropriate comparisons are not reliable

2.2 Apart from the treatments being compared, the comparison groups need to be similar (i.e. ‘like needs to be compared with like’)

If people in the treatment comparison groups differ in ways other than the treatments being compared, the apparent effects of the treatments might reflect those differences rather than actual treatment effects. Differences in the characteristics of the people in the comparison groups might result in estimates of treatment effects that appear either larger or smaller than they actually are. A method, such as allocating people to different treatments by assigning them random numbers (the equivalent of flipping a coin), is the best way to ensure that the groups being compared are similar in terms of both measured and unmeasured characteristics

Be cautious about relying on the results of nonrandomised treatment comparisons (for example, if the people being compared chose which treatment they received). Be particularly cautious when you cannot be confident that the characteristics of the comparison groups were similar. If people were not randomly allocated to treatment comparison groups, ask if there were important differences between the groups that might have resulted in the estimates of treatment effects appearing either larger or smaller than they actually are

4.1 The results of single comparisons of treatments can be misleading

A single comparison of treatments rarely provides conclusive evidence and results are often available from other comparisons of the same treatments. These other comparisons may have different results or may help to provide more reliable and precise estimates of the effects of treatments

The results of single comparisons of treatments can be misleading

5.1 Treatments usually have beneficial and harmful effects

Because treatments can have harmful effects as well as beneficial effects, decisions should be informed by the balance between the benefits and harms of treatments. Costs also need to be considered

Always consider the trade-offs between the potential benefits of treatments and the potential harms and costs of treatments

  1. aFrom Austvoll-Dahlgren et al. [31]