This week Harry quizzes Ulo Palm, the senior vice president of digital sciences at Allergan, about the long and problematic reign of the p-value in statistical analysis, and why it may be time for the biopharma industry to look to more nuanced measures of whether a drug trial succeeded.
Though the p-value “determines everything we do in drug development or medical research,” says Dr. Ulo Palm , it may be one of the most misunderstood and misused quantities in experimental science—drug discovery included. At its core, the p-value shows the probability that an observed effect was due to random chance. In other words, if a drug seems to outperforms a placebo with an associated p-value of 0.05, there’s only a 5 percent chance that the study was wrong and that the drug is, in fact, no better than the placebo. A p-value of 0.05 is the accepted threshold for validity in most scientific research, even though it’s an arbitrary standard set nearly a century ago by statistician Sir Ronald Fisher. “People don’t often realize that this p-value of 5 percent was pulled out of thin air,” Dr. Palm says. “If Sir Ronald Fisher had had six fingers, we would all be using a p-value of 6 percent.”
The issue, Palm says, is that an arbitrary dividing line of 0.05 leads journal publishers (and paper authors themselves) to reject or ignore real effects that don’t happen to meet the threshold. If a drug trial yields a p-value of more than 0.05, “You should never ever say it is not working,” he tells Harry. “You can only say we were not able to make a determination. That’s it.” By examining the spread of a data set, confidence intervals, data from individuals, and other measures, Palm says, today’s researchers can get a more realistic picture of the promise of a new compounds as medicines.