The genius of the amateur

Works in Progress is becoming a print magazine. Our first print issue, Issue 21, will land in November. If you live in the United States or the United Kingdom, you can subscribe here. If you live outside the US or UK and want to be notified as soon as subscriptions are live in your country, leave your details here.

Many of the most important scientific advances came from unexpected people. A draper was the first to observe bacteria, a clockmaker solved the problem of longitude, a musician discovered Uranus, and a Hollywood actress helped invent secure wireless communication. Let’s call these people outsiders.

Most scientists are ‘insiders’ – experts and specialists who spend their careers inside one academic discipline, mastering its ideas and methods over time. Outsiders, on the other hand, may come from another discipline, work outside established institutions, or be early enough in their careers that they are not yet part of the senior establishment of a discipline.

Outsiders often succeed because scientific progress is in part about generating models about how the world works, and in part about testing, applying and refining these models. The former is how we got quantum theory and the latter is how we got lasers and the MRI machine.

Insiders are often better at fleshing out theories in detail. But they get attached to their theories and can be bad at seeing when those theories need to change. Outsiders have accumulated less expertise, but being less attached to specific theories, they are more willing to update them through ‘paradigm shifts’: creating new theories to predict facts and define research questions. A productive system needs both kinds of work.

Academia has a comparative advantage in ‘outsider’ work – unlike industry research, which has a tendency to be applied, narrow, and focused towards a practical goal, academics naturally have the freedom and job security to take the outside view. But academia can be hostile to outsiders and is becoming more hostile as it comes to represent a larger and larger fraction of science.

This narrowing of opportunities for outsiders has weakened science’s ability to generate paradigm shifts. Many of the biggest scientific leaps began when an outsider spotted a puzzle, imported a method, or sketched a new theory. To encourage more of these leaps today, we need to create space for outsiders by giving people more freedom to switch fields, work independently ...