How a Finding Survives

The public story of a scientific discovery has a shape: someone works hard, finds something, publishes, and the finding enters the world. Every part of that sequence is true and the last step is wrong in a way that matters.

Publication does not make a finding real. It makes it available for the process that determines whether it is real, and that process is slow, thankless, and largely invisible to anyone outside the specific subfield doing it.

Four things that have to happen

From having watched this from inside a laboratory and then from outside one, I would describe the survival process as four separate filters, each of which kills claims that passed the previous one.

1. 01Somebody has to try it again, which requires a person willing to spend a year on a project whose best outcome is confirming what is already believed.
2. 02The method has to be reconstructible from the paper, which is far less often true than the existence of a methods section suggests.
3. 03The result has to survive being used for something, which is a harsher test than replication because a claim that is nearly right fails differently when it is load-bearing.
4. 04The claim has to be revised when it is partly wrong, rather than defended or abandoned, and this is the filter that depends most on the temperament of individual people.

The second filter deserves more attention than it gets. Methods sections are written under length constraints, by people who know the technique so well that the difficult parts are invisible to them. The knowledge required to actually run an experiment often lives in a postdoc’s notebook, or in a conversation at a conference, and not in the paper.

The correction problem

The fourth filter is the one I find most interesting, because it is not epistemological. It is social.

A claim that turns out to be 80 percent right is the hardest case in science. There is no mechanism designed for it. Retraction is too strong and reserved for misconduct or gross error. Silence is what usually happens: the original paper stands, subsequent work quietly stops relying on it, and the citation count keeps climbing for another eight years driven by people citing it as background.

The fields that handle this well have almost always built a specific cultural artifact for it — a review series, a consensus process, a curated database with a maintainer who is empowered to downgrade an entry. What they have in common is that somebody is paid to keep the record current, and that revising an entry is a routine act rather than an accusation.

None of this is an argument for distrusting published research, and I want to be precise about that, because the argument is frequently misappropriated. The process I have described is why scientific claims are more reliable than other kinds, not less. It is simply slower and more human than the announcement-shaped version most people encounter.

The practical consequence for a reader is modest and useful. A single new result is interesting and provisional. A result that has been used, contested, refined, and is still standing ten years later is a different category of thing entirely — and it almost never gets a headline, because by then it is not news.