Dr. Epling's Biostatistical "Pearls" for Reading Biomedical Research Papers

Set #1: Posted July 16, 2008

Study Design

• Identify a randomized controlled trial
• Identify a sub-analysis of a trial or study (warning, if these do not show a difference, worry about POWER)

Outcomes/Endpoints

• Distinguish between primary and other endpoints of a trial - the primary endpoint is what the power analysis is based on

Statistics

• Interpret a hazard ratio and describe when it is used - it's like a risk ratio, but accounts for variable length of follow-up of the patients (usually in a long trial where patients are enrolled throughout the study and they won't all have the same amount of follow-up time).
• Correctly interpret directionality when assessing risk and hazard ratios
• Correctly interpret confidence intervals for relative statistics - the interval should not cross 1 to be significant, but look also at the width of the interval for precision

Analysis

• Describe sensitivity analysis - a method of analyzing the effect of bias in a study by adjusting the data for different assumptions and re-running the statistics to see if the results are robust (i.e., do not change) with the assumptions. For example, if you have a group of subjects in your study that vary from the rest of the subjects, and you got a significant result in the analysis, you would exclude the "variant" subjects and re-run the analyses to see if the difference persisted (see also: http://www.jr2.ox.ac.uk/bandolier/booth/glossary/sensanal.html)
  
  
  

• Identify the authors' approach to missing data – do they fill in (impute) data which would cause the treatment to look worse (a “conservative” approach), do they use “last observation carried forward,” or do they just cite loss to followup?

Statistics

• Describe and interpret a pairwise comparison of differences and the Sign Test – a method for calculating the statistical significance of data in paired samples that does not assume normal distribution.  It uses a count of the number of differences above a given value (a median value or zero) compared with the number below (and usually excludes ties), and then assigns a p-value to that difference.

  Calculate and Interpret a Number Needed to Harm – analagous to Number Needed to Treat, but used for adverse events. Calculate the rates of adverse events in the intervention and control group, take the difference and inverts it (NNH= 1/|CER-EER|). * Note, this doesn't tell you ANYTHING about the relative severity of the adverse events in each group.

Analysis

• Describe and justify an intention to treat analysis – subjects are analyzed in the groups to which they are randomized whether or not they actually got the assigned intervention/control.  This preserves the randomization and gives a better understanding of the “effectiveness” of an intervention (as opposed to its “efficacy”).

• Surrogate Outcomes - outcomes that are short of the “harder” clinical endpoints but have been shown in other studies to correlate well with those endpoints.  WARNING: finding an association between a surrogate outcome and a clinical/patient-oriented outcome does not always mean that achieving the surrogate outcome will lead to achievement of the clinical outcome.
  
  
• Composite Outcomes – a grouping of outcomes that (ideally) should be closely related.  For instance, a composite cardiovascular outcome could include: admission for angina and non-Q wave MI, MI, cardiovascular mortality and overall mortality.  WARNING:  these are often studied in order to increase the power of the study – achievement of at least one of the outcomes counts, thereby reducing the time and sample size needed to show a difference in outcomes.  But, are admission for angina and overall mortality really equivalent?
  
  

Reports

• Case Report – a report of a single clinical case – often used to write up an interesting medical case – often accompanied by a literature review about that case
  
  
• Case Series – a report of a number of similar clinical cases.  Sometimes the diagnosis is not clear, and the authors are reporting a new syndrome (AIDS was first described this way in 1981 in NEJM).

• Number needed to treat = 1/ARR = 1/|CER-EER|
  In English, “You'd need to treat X patients with this drug for Y years to prevent one additional outcome.”
  
  
• Error   
  Type I error = Alpha.  By convention, the acceptable type 1 error rate is 0.05 (5%, corresponds to the p-value).
  Type II error = Beta.  By convention, especially in power analysis, the acceptable type II error rate
  is 10-20%
  
  
• Power = 1-beta (the chance that you won't make a type II error) = usually 80-90%