The Savvy Scientist

Particle of doubt: the Higgs boson and scientific uncertainty

Posting in Cancer

Inconclusive news about a possible physics breakthrough may be frustrating, but the uncertainty at the heart of all science deserves respect.

This morning at 8 a.m. EST -- only a couple of hours after this column is published -- at one of the most awaited press conferences of 2011, two groups of physicists will announce results from experiments run on the Large Hadron Collider (LHC), the world's most powerful accelerator. They will break news on the search for a long-and-feverishly sought fundamental particle called the Higgs boson, whose discovery would help to validate the theoretical framework for much of current physics.

At the risk of making a prediction that may therefore be laughably wrong by the time you read it, I will go out onto a short, sturdy limb and guess that the scientists' observations will be intriguingly suggestive of the Higgs… but fall well short of proof that it exists. The announcement will be rife with uncertainty, caveats, and disclaimers. I'll further argue, however, that the physicists' uncertainty is a hallmark of good science, not a failure of it -- a point too often lost on unscientific critics.

To be sure, a certain amount of scientific grandstanding might be going on. Sometimes scientists do report results so weak and preliminary that outsiders wonder why they bothered -- and the answer is often that the scientists felt a need to keep themselves or their work visible, perhaps to protect their funding. Maybe that will be the case with these Higgs announcements; maybe not. In a sense it doesn't matter to the scientific process, though, because uncertain, preliminary results are a natural step along the road to better, firmer ones.

The weight of evidence

Over the past half century, physicists have developed a comprehensive theory known as the Standard Model to describe all the known and suspected subatomic particles and the forces that control them. Perhaps the most conspicuously unverified part of that model is the hypothetical Higgs boson, which gives rise to an energy field that lends all other particles their mass.

Interactions of the Higgs boson with other classes of subatomic particles.

In short, the Higgs is why anything weighs anything -- or at least that's what most physicists have believed. More than a decade ago, using the highest-energy colliders available, they searched fruitlessly for the Higgs boson and determined that its mass had to be greater than the equivalent of 114 giga-electron volts (GeV), roughly the mass of an entire atom of the element indium. But they had no way of searching in the next higher band of possible masses -- up to 140 GeV -- until three years ago, when the European Organization for Nuclear Research (CERN) completed construction of the LHC.

Informed speculation circulating in the week leading up to the announcement suggests that the research groups using two of the LHC's instruments, the CMS and the ATLAS detectors, have seen indications that might indicate a Higgs particle with a mass around 125 GeV. One such reading could be a fluke; two starts to look like independent corroboration… but major doubts remain.

And that is why my prediction that this Higgs news will be cautiously tentative is a safe bet. For starters, to rein in expectations, CERN spokespeople have already started telling some journalists that the announcement will not be definitive. (I do love a sure thing.) So the official position of CERN will be that nothing was decisively found.

During installation of the ATLAS detector. (Credit: ATLAS experiments © 2011 CERN)

It also helps that neither the CMS nor the ATLAS detector can observe the Higgs directly. Nothing can. Instead, physicists have to look for evidence of the Higgs's spontaneous decay into a shower of other particles. From those decay byproducts, they can infer the presence of the Higgs. The catch is that the physicists can't absolutely rule out the possibility that collisions between other particles didn't create those byproducts and confound the results.

But the most important reason physicists will be reluctant to celebrate too enthusiastically is that their statistical confidence in the results is probably too weak. Statisticians characterize the amount of random variation in sets of data with the measure called a standard deviation (or sigma). The normal threshold for a scientific discovery is a result that shows at least 5 sigma difference from the control or baseline, which means they can be 95 percent confident it is real. (The sigma value doesn't reflect the odds that a result is correct, only that the result is not simply a statistical fluke, like a flipped coin randomly coming up heads five times in row.)

[Correction: 95 percent is the confidence level used for most types of scientific papers. In physics, however, the standard for credibility is typically more like a one-in-a-million chance that the result is a fluke. Thanks, hawleyj@ in comments, for pointing this out.]

Those informed rumors suggest that the confidence levels of the two LHC experiments may only be 2.5 or 3.5 sigma. If so, the reports are interesting but untrustworthy observations that might turn out to be completely misleading in the long run. Lots more data will be needed to raise confidence in these observations and to rule out competing explanations, and the LHC scientists will no doubt be gathering it over the next year to dispel the uncertainty.

Knowing what you know

Uncertainty is a crucial strength of science, not a weakness. It's not merely an admission of ignorance but a quantified assertion about what is known confidently and reasonably. Yet science's embrace of uncertainty drives certain critics crazy -- usually ones who find themselves on the losing side of an argument against science.

The tobacco industry harped for decades on the uncertainties in the medical literature linking smoking to cancer. In the late 1990s, then-GOP political consultant Frank Luntz wrote an influential memo (later leaked) that observed the "scientific debate is closing [against us]" and recommended emphasizing uncertainty in the climate science as the strategy most likely to sway the public away from taking global warming seriously. Representatives of the Discovery Institute, an intelligent-design creationism think tank, point to holes in the fossil record and unanswered questions about the evolution of life as proof that biologists don't know what they're talking about. Activists have also clutched at straws to make cases based on uncertainty against the safety of childhood vaccinations and genetically modified foods.

Such dogmatic thinking is at odds with what makes real scientific progress possible. The physicists at CERN can speak about tentative results on the Higgs boson today because months from now, without losing credibility, they can say that additional experiments have either upheld or disproved them. In words often (though unreliably) attributed to the economist John Maynard Keynes: "When my information changes, I change my mind. What do you do, sir?"

Update (added Dec. 13, 10:00 a.m. EST): Indeed, the CERN announcements were both as excited and as tentative as expected. Here's a link to CERN's official press release on the news. A relevant quote:

Taken individually, none of these excesses is any more statistically significant than rolling a die and coming up with two sixes in a row. What is interesting is that there are multiple independent measurements pointing to the region of 124 to 126 GeV. It's far too early to say whether ATLAS and CMS have discovered the Higgs boson, but these updated results are generating a lot of interest in the particle physics community.

Image: CMS data from the LHC. (Credit: © 2011 CERN)

Share this

John Rennie

Contributing Editor

Columnist, Science John Rennie is the former editor-in-chief of Scientific American. He has written for IEEE Spectrum, New York Times and The Economist and has appeared on the History Channel, Discovery Channel, NPR and Minnesota Public Radio. He has spoken at the World Business Forum, Massachusetts Institute of Technology, Wharton School of Business, University of Tennessee and Middlebury College and is an adjunct instructor at New York University, editor at large for Txchnologist.com and contributing editor to ecomagination.com. He is based in New York. Follow him on Twitter. Disclosure