Preregistration (science)

Based on Wikipedia: Preregistration (science)

The Scientists Who Promised to Tell the Truth Before They Knew What It Was

Imagine a detective who writes down who the murderer is before examining any evidence, seals that prediction in an envelope, and only opens it after the investigation concludes. Absurd, right? Yet for decades, science has operated in precisely the opposite way—and that's been a problem.

Scientists would gather data, analyze it six different ways, find something interesting on the seventh try, and then write papers claiming they'd suspected this all along. It was, in a sense, the detective opening the envelope, reading "the butler did it," quickly scribbling over their original guess, and saying "See? I knew it was the butler."

Preregistration flips this script. It's the practice of writing down your hypothesis, your methods, and your analysis plan before you collect a single data point. Then you seal that promise in a public, timestamped registry. Whatever you find, the world can compare your results against what you said you'd look for.

It sounds almost embarrassingly simple. And yet this idea—really just a formal commitment to intellectual honesty—is reshaping how science works.

Why Would Scientists Need to Make Promises?

Scientists are humans, and humans are spectacularly good at fooling themselves. This isn't about fraud or deliberate deception. It's about the thousand small decisions that happen between collecting data and publishing results.

Consider a researcher studying whether a new teaching method improves test scores. She collects data from 200 students. The initial analysis shows no effect. But wait—what if the method works differently for boys versus girls? She splits the data. Still nothing. What about students who actually attended all the sessions? Now there's a small effect. What if we remove three outliers with unusually low scores? Now it's statistically significant.

Each of these decisions might be perfectly reasonable in isolation. But together, they constitute a kind of treasure hunt through the data, stopping only when you find gold. Researchers call this "p-hacking," named after the p-value—that magical number below 0.05 that often determines whether a finding gets published or forgotten.

There's also HARKing: Hypothesizing After the Results are Known. This is the practice of discovering an unexpected pattern in your data and then rewriting your paper to claim you predicted it all along. Your surprising finding transforms from "we stumbled upon this" to "as we hypothesized." The detective scribbles over the envelope.

None of this requires malice. It requires only the very human desire to find meaningful patterns, combined with career incentives that reward positive findings over null results. The cumulative effect, however, has been devastating.

The Crisis That Broke Science's Confidence

Starting around 2011, a series of high-profile failures to replicate famous findings shook the scientific community. The effect was most dramatic in psychology, where a massive effort to repeat 100 published studies found that only about a third produced the same results the second time around. But the problem extended far beyond psychology—to cancer biology, economics, and medicine.

Publication bias was a major culprit. Scientific journals, like newspapers, favor exciting stories. A study showing that a drug works is more interesting than one showing it doesn't. A paper revealing a surprising effect gets more citations than one confirming the obvious. So negative results—the "we tried this and nothing happened" studies—accumulated in file drawers, unpublished and unknown.

The published literature became a highlight reel of successes, creating a distorted view of reality. Meta-analyses—studies that combine results from many individual papers—were drawing conclusions from a skewed sample. Imagine trying to understand how well students perform on exams, but only having access to the grades of those who chose to show you their report cards.

Preregistration offered a way out. If you commit to your methods before seeing the data, you can't massage them afterward. If journals agree to publish based on the research question rather than the results, null findings enter the literature alongside positive ones. The file drawer empties.

How It Actually Works

The standard preregistration process is refreshingly straightforward. Before collecting any data, a researcher creates a document specifying the essential details of their study: What hypothesis are they testing? How many participants will they recruit? How will they measure the outcomes? What statistical tests will they run? Under what conditions will they exclude data points?

This document gets uploaded to a public registry—platforms like the Open Science Framework or AsPredicted, which timestamp everything and make it impossible to alter retroactively. The researcher then conducts their study exactly as planned. When they publish, they include a link to the preregistration so anyone can compare the planned analysis to the actual one.

Deviations are allowed. Science is messy, and sometimes you discover that your planned approach won't work, or you notice something unexpected worth exploring. The requirement isn't rigid adherence to the plan—it's transparency about any changes. You can still analyze your data in new ways, but you must clearly label these as exploratory rather than confirmatory.

This distinction matters enormously. A confirmatory analysis—one specified in advance—provides strong evidence because the data had only one chance to support or refute the hypothesis. An exploratory analysis—one discovered in the data—is more tentative, a promising lead that requires independent confirmation. Both have value; the problem arises when exploratory findings masquerade as confirmatory ones.

Registered Reports: Taking It Further

Preregistration keeps you honest, but it doesn't solve publication bias. Journals might still reject your paper if the results are boring. Enter the registered report—preregistration with teeth.

In this format, you submit your introduction, methods, and analysis plan to a journal before collecting data. Peer reviewers evaluate your research question and methodology. If they approve, the journal offers "in principle acceptance"—a promise to publish regardless of what you find. Only then do you run the study.

This changes everything. The journal commits to the question, not the answer. Null results become publishable because the decision was made before anyone knew what the results would be. Reviewers focus on whether the methods are sound rather than whether the findings are exciting.

The results have been striking. Studies comparing registered reports to traditional publications find dramatically higher rates of null results in the registered format—closer to what we'd statistically expect, rather than the parade of positive findings that characterizes conventional literature. The file drawer is opening.

Over 200 journals now offer registered reports, with the number roughly doubling each year. Nature Human Behaviour, one of the field's most prestigious outlets, adopted the format because it "shifts the emphasis from the results of research to the questions that guide the research and the methods used to answer them."

Clinical Trials: Where It Started

The scientific reform movement often looks to medicine for both cautionary tales and solutions. Clinical trial registration—preregistration's older sibling—has been mandatory in many contexts for nearly two decades.

The push began with a scandal. In 2004, New York State Attorney General Eliot Spitzer sued GlaxoSmithKline, a major pharmaceutical company, for concealing trial results suggesting that certain antidepressants might harm children. The company had run multiple trials; some showed risks, others didn't. Guess which ones they published?

Shortly after, the International Committee of Medical Journal Editors—the gatekeepers of prestigious medical publishing—announced they would no longer consider papers from unregistered trials. If you wanted to publish in the New England Journal of Medicine or The Lancet, you needed to register first. Researchers, suddenly facing a stark choice between transparency and obscurity, chose transparency.

ClinicalTrials.gov, run by the United States National Library of Medicine, became the world's largest registry. Today it contains over 400,000 studies from more than 200 countries. The World Health Organization maintains an international portal linking registries worldwide, working toward the goal "that a complete view of research is accessible to all those involved in health care decision making."

The Declaration of Helsinki—the foundational document of research ethics—was revised in 2008 to require that "every clinical trial must be registered in a publicly accessible database before recruitment of the first subject." Registration transformed from a reform proposal into a professional norm.

The Skeptics' Concerns

Not everyone is convinced that preregistration is the answer. Critics raise several thoughtful objections worth considering.

Some argue that analytical flexibility isn't inherently bad. Different research questions naturally allow different amounts of wiggle room, and experienced readers can evaluate findings accordingly. A theory that makes precise, specific predictions is more severely tested than one offering vague directional claims. Reviewers don't need a preregistration to recognize this—they can simply ask whether the test was demanding.

Others point out that practical constraints already limit researchers' ability to game their analyses. Scientific communities have norms and conventions. Results need to survive robustness checks—analyses using different methods to verify the finding holds up. Conclusions must fit together coherently. It's harder to cherry-pick than it might seem.

There's also concern that preregistration could discourage the kind of open-ended exploration that leads to breakthrough discoveries. Science often advances through serendipity—the unexpected observation that reshapes understanding. If researchers feel locked into predetermined questions, might they miss the interesting patterns lurking in their data?

Proponents respond that preregistration doesn't forbid exploration; it just requires labeling it honestly. You can still follow interesting leads. You simply can't pretend you predicted them in advance.

The Unexpected Benefits

Something interesting happened as preregistration spread: researchers started reporting benefits beyond the prevention of bias.

The act of writing a detailed preregistration forces you to think through your study carefully before collecting data. What exactly is your hypothesis? How will you know if it's supported? What could go wrong? Questions that might otherwise surface mid-analysis—when they're most disruptive—get addressed upfront.

Students report that preregistration improves their understanding of their own research. Supervisors use it as a teaching tool, shaping good research habits from the start. The discipline of committing to a plan paradoxically creates freedom—once the plan is set, you don't waste mental energy second-guessing every decision.

A 2024 study in the Journal of Political Economy: Microeconomics examined preregistration in economics and found something important: preregistration reduced p-hacking and publication bias, but only when accompanied by a detailed pre-analysis plan specifying exactly what analyses would be conducted. Simply registering the existence of a study without committing to methods didn't have the same effect. The details matter.

Beyond Psychology: A Growing Movement

While psychology led the reform movement, preregistration is spreading across disciplines. Researchers have developed approaches for qualitative studies, where the methods differ fundamentally from statistical hypothesis testing. Templates exist for single-case designs, electroencephalogram research, experience sampling, and even exploratory research where the goal is discovery rather than confirmation.

Preclinical research—the animal and cell studies that precede human trials—is increasingly embracing preregistration. This matters beyond scientific accuracy. Animal experiments involve ethical trade-offs; animals experience stress and harm in service of potential human benefit. If those experiments never get published—if the results disappear into file drawers because they're not exciting enough—the ethical bargain is broken. The animals suffered, but humanity learned nothing.

The World Health Organization, building on decades of experience with clinical trial registration, views transparency as essential to "strengthen the validity and value of the scientific evidence base." The push for registration continues to accelerate.

The Deeper Lesson

Preregistration works not because scientists are dishonest, but because they're human. The practices it prevents—HARKing, p-hacking, selective reporting—aren't moral failures. They're predictable consequences of working in systems that reward positive findings, under cognitive biases that favor pattern-seeking over skepticism.

The solution isn't to demand superhuman objectivity. It's to design systems where honest behavior is also easy behavior. Make commitments in advance, when you have no stake in the outcome. Make those commitments public, so deviation carries social costs. Evaluate research based on questions asked rather than answers found.

This is, at heart, a lesson about institutions. Individual virtue is unreliable; structural incentives are powerful. If you want scientists to report null results, create journals that publish them. If you want researchers to resist the temptation to tweak their analyses, make it easy to commit in advance and hard to change later.

The detective with the sealed envelope isn't more honest than other detectives. But the envelope keeps honest detectives honest—and makes dishonest ones easier to spot.

What It Means for Reading Science

For those of us who read scientific findings—in news articles, policy documents, or our own explorations—preregistration offers something valuable: a way to calibrate confidence.

A preregistered study with a detailed analysis plan, conducted exactly as specified, provides stronger evidence than a traditional publication where the analysis was chosen after seeing the data. A registered report, accepted before results were known, is less likely to be a cherry-picked success story than a conventional paper selected from a larger pool of unpublished failures.

This doesn't mean unregistered research is worthless. Most of scientific history occurred without preregistration, and many robust findings emerged. But when evaluating new claims—especially surprising or consequential ones—knowing whether the researchers committed to their methods in advance provides useful information about how much the data actually constrained their conclusions.

Science is, fundamentally, an elaborate system for constraining human belief with evidence. Preregistration is one mechanism in that system: a way of ensuring that what we think we know is actually driven by what we found, not by what we hoped to find. It's a small tool with large implications—a promise made before the answer is known, kept by making the promise impossible to break.