Open-source intelligence
Based on Wikipedia: Open-source intelligence
The Spies Who Read Newspapers
During World War II, American intelligence analysts discovered something remarkable: they could track the success of Allied bombing raids on French railway bridges by watching the price of oranges in Paris.
Think about that for a moment. Bombs fell in the countryside. Bridges collapsed. Trains stopped running. Oranges from southern France couldn't reach the capital. Prices spiked. All of this became visible in publicly available market data—no secret agents, no intercepted communications, no codebreaking required.
This is the essence of open-source intelligence, known by its acronym OSINT (pronounced "oh-sint"). It's the art and science of learning secrets from information that isn't secret at all.
What Makes Intelligence Different from Research
You might wonder: isn't this just research? Anyone can read newspapers, browse social media, or look up public records. What makes open-source intelligence different from a particularly thorough Google search?
The distinction lies in the purpose. Research gathers information. Intelligence transforms that information into specific, actionable knowledge designed to help a particular person or group make a particular decision. A researcher might compile everything publicly known about a country's military budget. An intelligence analyst takes that same information and answers: "Will this nation be able to sustain a conflict for more than six months?"
The analyst applies tradecraft—systematic methods developed over decades—to verify, cross-reference, and interpret what would otherwise be a flood of disconnected facts. They're looking for patterns, anomalies, and the telling details that reveal what someone doesn't want you to know.
Six Windows Into the World
Open-source intelligence draws from six main categories of information, each with its own strengths and peculiarities.
Traditional media—newspapers, magazines, radio, and television—remains surprisingly valuable. A careful reader can learn enormous amounts from what state-controlled media chooses to emphasize, downplay, or omit entirely. During the Cold War, Western analysts became experts at reading between the lines of Soviet publications, noting which officials appeared in photographs and which had mysteriously vanished.
The internet has transformed the field entirely. Social media posts, blogs, discussion forums, and user-generated content create an unprecedented window into societies worldwide. A soldier posts a selfie without realizing the background reveals his unit's location. A government official's spouse shares vacation photos that contradict his stated whereabouts. This source moves faster than any other and grows by the petabyte daily.
Public government data includes everything from budget documents and press conferences to telephone directories and official speeches. Governments themselves publish vast amounts of information, and even mundane bureaucratic records can reveal patterns invisible to casual observers.
Academic and professional publications—journals, conference proceedings, dissertations, and theses—contain detailed technical knowledge that once would have been closely guarded. A doctoral thesis on a country's nuclear program might reveal more about its capabilities than any spy could steal.
Commercial data encompasses satellite imagery you can purchase, financial assessments, industrial databases, and business intelligence. You no longer need to be a superpower to see the world from space; companies like Planet Labs and Maxar sell imagery detailed enough to count tanks in a military depot.
Grey literature refers to the hard-to-find documents that fall outside traditional publishing: technical reports, working papers, patents, internal business documents, and draft publications. These often contain the most current and detailed information, precisely because they haven't been sanitized for public consumption.
The American Story
The United States formally organized open-source intelligence collection in 1941, just before Pearl Harbor, when it created the Foreign Broadcast Monitoring Service. The name sounds bureaucratic, but the mission was urgent: monitor enemy radio broadcasts and extract every scrap of useful information.
That wartime agency evolved into the Foreign Broadcast Information Service, which for decades produced translations and analyses of foreign media. Intelligence professionals knew its value, but open-source work always played second fiddle to the more glamorous—and expensive—world of secret collection.
Then came the attacks of September 11, 2001.
The postmortems were damning. The 9/11 Commission and the Iraq Intelligence Commission both concluded that American intelligence had severely neglected open sources. Information that might have provided warning was publicly available but never systematically collected or analyzed. In 1996, the Aspin-Brown Commission had already flagged this as a critical weakness, calling improved open-source collection a "top priority."
The wake-up call finally registered. In November 2005, the Director of National Intelligence created the Open Source Center, tasked with collecting and analyzing everything from internet postings to commercial satellite imagery. The center absorbed the old Foreign Broadcast Information Service and expanded its mission dramatically. For the first time, open-source intelligence had a dedicated home at the highest levels of the American intelligence community.
How the World Defines It
Different organizations frame open-source intelligence in ways that reflect their particular concerns.
The United States Department of Homeland Security emphasizes speed, defining it as intelligence derived from publicly available information that's collected and disseminated promptly to address specific needs. In the security world, yesterday's information is often worthless.
The North Atlantic Treaty Organization, or NATO, adds an interesting wrinkle. It includes not just fully public information but also "unclassified data with limited public distribution or access"—things like academic databases or specialized commercial services that technically anyone can access but few actually do.
The European Union stresses the actionable nature of the product, focusing on how open-source collection supports national security, law enforcement, and business intelligence.
The United Nations has recognized open-source intelligence as a tool for monitoring whether member states comply with international obligations in areas like public health and human rights. When a government claims it's meeting its treaty obligations, satellites and social media can reveal the truth.
Private companies have their own definitions, naturally oriented toward business applications. IBM describes it as gathering publicly available information to assess threats and inform decisions. Cybersecurity firms like CrowdStrike focus on threat intelligence—understanding who might attack you and how.
The Information Explosion Problem
Here's the paradox: we've never had more open information, and that abundance is itself a problem.
The volume of publicly available data grows at a rate that overwhelms any attempt to systematically analyze it. A single day produces more content than a team of analysts could review in a lifetime. Social media platforms generate billions of posts. Satellite constellations photograph the entire Earth's surface repeatedly. Every government, company, and organization publishes more than ever before.
This creates what intelligence professionals call the "needle in a haystack" problem, except the haystack is growing exponentially and the needles are moving.
Some organizations have experimented with crowd-sourcing, enlisting amateur volunteers to help process information. During the search for Malaysia Airlines Flight 370 in 2014, thousands of volunteers scanned satellite imagery looking for debris. Similar approaches have been used to track troop movements and document human rights abuses. But crowd-sourcing brings its own challenges: quality control, security concerns, and the difficulty of coordinating untrained observers.
The real solution will likely involve artificial intelligence—systems that can filter, prioritize, and flag potentially significant information at machine speed. But we're not there yet, and until we are, the flood of available data will continue to exceed our capacity to drink from it.
The Dark Side: Contamination and Deception
Open sources have an Achilles heel: precisely because they're public, anyone can pollute them.
Disinformation—deliberately false information spread to deceive—poses a fundamental challenge to open-source intelligence. If you're analyzing publicly available information, you're analyzing information that adversaries know you can see. That creates an irresistible temptation to plant false data.
Governments and state actors have been caught intentionally seeding false information through news outlets, social media, official statements, and even manipulated geospatial data. A military might move decoy equipment to fool satellite watchers. A government might plant false stories in friendly media outlets, knowing they'll be picked up by analysts. An intelligence service might create fake social media personas that build credibility over months before starting to spread lies.
Extremist groups use open-source channels to spread ideology and misinformation, sometimes deliberately trying to get their narratives picked up by mainstream analysts who will inadvertently amplify them.
This means open-source intelligence requires rigorous verification—cross-checking information against multiple independent sources, looking for telltale signs of manipulation, and maintaining healthy skepticism about anything that seems too convenient. A photograph might be authentic but taken at a different time or place than claimed. A document might be genuine but deliberately leaked to create a false impression. A social media account might belong to a real person who's been deceived into spreading lies they believe are true.
The contamination problem has no perfect solution. It's an arms race between those trying to learn the truth and those trying to obscure it.
The Legal Boundaries
An important distinction separates open-source intelligence from espionage: the legality of the collection.
True open-source intelligence involves only information that's legally accessible. Anyone can read a newspaper, browse a public website, or purchase commercial satellite imagery. No laws are broken; no secrets are stolen.
But the line can blur. A private individual who systematically collects information for a foreign military or intelligence agency may be committing espionage, even if every individual piece of information was publicly available. The intent and the recipient matter. In most countries, working as an unregistered agent of a foreign power is illegal, regardless of whether your tradecraft involves hacking classified databases or reading newspapers in a coffee shop.
This legal complexity means that professional open-source practitioners pay careful attention to operational security—using virtual private networks, cached copies of web pages, and browser sandboxes to avoid exposing themselves or their investigations. The tools and techniques resemble those used for legitimate privacy protection, but they're employed to ensure that intelligence collection remains clearly legal and defensible.
Becoming a Professional
The field has professionalized significantly. Organizations like the McAfee Institute offer certification programs recognized by the U.S. Department of Defense and Department of Homeland Security. The Certified in Open Source Intelligence credential validates competency in legal and ethical online investigation, digital profiling, and open-source data analysis.
The OSINT Foundation serves as a professional association for practitioners in the American intelligence community, working to raise the profile of the discipline and establish professional standards.
For those entering the field, dozens of aggregated tool lists and frameworks are available online. The OSINT Framework, maintained as an open-source project on GitHub, catalogs over thirty categories of tools. These range from social media analysis platforms to geolocation services to archived webpage retrieval systems.
The Democratization of Intelligence
Perhaps the most profound change in open-source intelligence is who can now practice it.
The techniques once available only to government agencies with massive budgets have become accessible to journalists, human rights investigators, academic researchers, and private citizens. The same satellite imagery that helped the CIA track Soviet missile programs is now available to anyone with a credit card. The same social media analysis that intelligence agencies use to understand foreign populations can be performed by a determined individual with a laptop.
This democratization cuts both ways. On one hand, it enables accountability. When governments lie, independent investigators can sometimes prove it. When atrocities are denied, open-source evidence can document them. The Berkeley Protocol on Digital Open Source Investigations, published jointly by the United Nations and UC Berkeley's Human Rights Center, establishes standards for using open-source techniques to investigate violations of international law.
On the other hand, the same tools enable stalkers, harassers, and authoritarian governments to track individuals. The line between legitimate investigation and invasive surveillance isn't always clear, and the capabilities continue to outpace the ethical frameworks meant to govern them.
The Future Written in Orange Prices
Those World War II analysts tracking orange prices in Paris were pioneers of something that would take decades to mature. They understood an essential truth: secrets hide in plain sight, revealed by their effects on the observable world.
Today, the orange prices are tweets and satellite photos and shipping data and patent filings and a million other streams of publicly available information. The challenge isn't access—it's making sense of the flood. The future of open-source intelligence lies in the systems and methods we develop to drink from the fire hose, to separate signal from noise, to verify truth in an age of disinformation.
And perhaps most importantly, the future lies in deciding who gets to wield these powerful techniques, and for what purposes. Open-source intelligence can expose war crimes and hold the powerful accountable. It can also enable surveillance and harassment. The tools are neutral; the uses are not.
The spies who read newspapers have become all of us, scrolling through our feeds, watching our world reveal itself one public post at a time.