Efficient-market hypothesis

Based on Wikipedia: Efficient-market hypothesis

The Impossible Dream of Beating the Market

Here is a thought experiment that has haunted investors for over a century: Imagine you discover that a major company is about to announce a merger. This is valuable information. You could buy the stock now and profit when the news becomes public.

But wait. If you know this, who else knows? The company's executives, certainly. Their lawyers. The bankers handling the deal. Perhaps a few journalists sniffing around. And if they know, some of them might already be buying.

The price starts creeping up. Other traders notice the unusual activity. They don't know why the stock is moving, but they start buying too. By the time the merger is announced, the stock price has already absorbed the news. Your "inside information" is worth nothing.

This is the core insight of the efficient-market hypothesis, often abbreviated as EMH. It suggests that trying to outsmart the stock market is fundamentally futile—not because you're not smart enough, but because the market itself is a vast information-processing machine that has already done the thinking for you.

What Efficient Actually Means

The word "efficient" here doesn't mean what you might think. It's not about whether markets allocate resources wisely or produce good outcomes for society. An efficient market could be efficiently pricing shares in a company that makes nothing but cigarettes and slot machines.

No, efficiency in this context has a very specific meaning: prices reflect all available information.

Think about what that implies. If a stock is priced at fifty dollars, that price already incorporates everything known about the company—its earnings, its debts, its competitive position, the quality of its management, the outlook for its industry, the state of the economy, and countless other factors. Millions of investors, analysts, and traders have sifted through this information and placed their bets. The price is the result.

This doesn't mean the price is "correct" in some cosmic sense. It doesn't mean the company will succeed or that fifty dollars represents true value. It means that given what is currently known, you cannot systematically identify stocks that are too cheap or too expensive.

Three Flavors of Efficiency

In 1970, the economist Eugene Fama published an influential paper that divided market efficiency into three categories. These distinctions matter because they make very different claims about what kind of information gets reflected in prices.

The weak form of efficiency says that prices already reflect all historical price information. If this is true, then technical analysis—the practice of studying charts and patterns to predict future prices—is pointless. It doesn't matter if the stock has been rising for six months or if it just formed a "head and shoulders" pattern. Past prices tell you nothing about future prices.

The semi-strong form goes further. It says prices reflect all publicly available information—not just past prices, but earnings reports, news articles, economic data, analyst recommendations, everything. If this is true, then fundamental analysis—carefully studying a company's financial statements to find undervalued stocks—is also pointless. By the time you read the annual report, thousands of other people have read it too, and the price has adjusted.

The strong form makes the most extreme claim. It says prices reflect all information, including private information known only to company insiders. If this is true, then even illegal insider trading wouldn't generate reliable profits, because somehow the market would already know.

Almost nobody believes the strong form. Insider trading prosecutions regularly turn up cases where people made millions from non-public information. But the weak and semi-strong forms have been taken very seriously by academics and practitioners alike.

A French Mathematician in 1900

The efficient-market hypothesis is usually associated with Eugene Fama and the University of Chicago in the 1960s and 1970s. But the core idea is much older.

In 1900, a French mathematician named Louis Bachelier submitted his doctoral thesis at the Sorbonne in Paris. The thesis was titled "The Theory of Speculation," and it analyzed the behavior of prices on the Paris stock exchange.

Bachelier's opening paragraph contains a remarkable insight: "Past, present and even discounted future events are reflected in market price, but often show no apparent relation to price changes."

Read that again. In 1900—before automobiles were common, before radio broadcasting existed, before the First World War—Bachelier articulated the essential logic of market efficiency. Prices already contain everything known, so price changes must come from something unknown: new information that arrives unpredictably.

Bachelier went further. He developed a mathematical model showing that if prices reflect all available information, they should follow what mathematicians call a "random walk." Each price change should be independent of the previous one, like a drunk stumbling home from a bar. The drunk might go left or right at each step, but knowing his last ten steps tells you nothing about where he'll go next.

For decades, Bachelier's work was largely forgotten in economics, though mathematicians continued to cite it. The story goes that the statistician Leonard Savage rediscovered the thesis in the 1950s and circulated it to economists, including Paul Samuelson at MIT. By the 1960s, Bachelier's ideas were ready for their moment.

The Market as Information Processor

Why would markets be efficient? The economist Friedrich Hayek provided a compelling answer in 1945, though he wasn't specifically discussing stock markets.

Hayek argued that markets are remarkably good at aggregating information that is scattered across millions of minds. No single person knows everything relevant to pricing a barrel of oil or a share of stock. But when people trade based on their individual knowledge, their actions get combined into a price that reflects far more than any individual knows.

A geologist in Texas knows about new drilling techniques. A shipping executive in Singapore knows about tanker availability. A banker in London knows about credit conditions. A politician in Riyadh knows about production quotas. None of them has the complete picture, but when they all trade, the price of oil somehow incorporates all their knowledge.

This is what economists call the "aggregation of dispersed information." Markets do it automatically, without anyone designing the system or even understanding how it works.

There's also a powerful incentive mechanism at play. If you genuinely know something that the market doesn't, you can profit by trading on it. A hedge fund that discovers a company is about to go bankrupt can short the stock and make millions. But in doing so, they push the price down, and the information gets incorporated.

The very act of exploiting market inefficiency eliminates it.

Random Walks and Drunkards

The mathematical implications of market efficiency lead to some surprising conclusions. If prices reflect all available information, then price changes should be unpredictable—not because markets are chaotic, but precisely because they are rational.

Think about it this way. If you could predict that a stock would rise tomorrow, you would buy it today. So would everyone else who made the same prediction. The buying pressure would push the price up today, not tomorrow. The predictable rise would happen immediately and then be gone.

For price changes to be predictable, they would have to persist even after people noticed them—and that requires explaining why nobody trades on the pattern and eliminates it.

This leads to the random walk model. Stock prices wander unpredictably, like that drunk stumbling home. Tomorrow's price is today's price plus some random shock that nobody can foresee. The technical term is that prices follow a "martingale," a mathematical concept borrowed from gambling theory.

Empirical studies in the 1930s through 1960s found that stock prices did indeed behave something like random walks in the short term. This was surprising and uncomfortable for many investors. It suggested that all their analysis—poring over balance sheets, reading industry reports, building financial models—might be wasted effort.

The Rebellion Against Efficiency

Not everyone accepted the efficient-market hypothesis. Some of the world's most successful investors built their fortunes by doing exactly what the theory says is impossible: consistently beating the market.

Warren Buffett, perhaps the most famous investor alive, has outperformed the market for decades. He's pointed out, with characteristic wit, that if markets were truly efficient, he would be "a bum on the street with a tin cup."

George Soros made a billion dollars in a single day in 1992 by betting against the British pound, correctly predicting that the Bank of England couldn't maintain its exchange rate peg. The market was very much not pricing this information correctly.

These examples are troubling for efficient-market theory. The standard response is that such investors are statistical outliers—in any large population of investors, some will beat the market by pure luck, and we only hear about the winners. But when the same people beat the market year after year, luck becomes a less convincing explanation.

The Behavioral Challenge

A more systematic challenge came from behavioral economics, a field that studies how real human decision-making deviates from the rational ideal assumed by economic models.

The psychologists Daniel Kahneman and Amos Tversky documented dozens of cognitive biases—systematic errors in human thinking. People are overconfident in their predictions. They overreact to recent news and underreact to base rates. They're more sensitive to losses than to equivalent gains. They see patterns in random data.

If investors are subject to these biases, how can markets be efficient? Wouldn't the biases show up in prices?

The economist Richard Thaler—who won the Nobel Prize in Economics in 2017—spent his career documenting market anomalies that seemed to reflect behavioral biases. Stocks with low price-to-earnings ratios tended to outperform, even after adjusting for risk. Small companies earned higher returns than their risk would justify. Stocks that had risen recently tended to keep rising for a while, a phenomenon called momentum.

These findings were hard to square with market efficiency. If low price-to-earnings stocks are systematically underpriced, why doesn't buying pressure push their prices up until the advantage disappears?

Defenders of market efficiency had an answer. Perhaps these anomalies weren't really anomalies at all. Perhaps low price-to-earnings stocks were riskier in some way that the standard models didn't capture. The higher returns were compensation for bearing that risk, not evidence of market inefficiency.

This response highlights a fundamental problem with testing market efficiency, known as the "joint hypothesis problem." To test whether the market is efficient, you need to define what returns a stock should earn given its risk. But any model of risk might be wrong. When you find an anomaly, you can't tell whether the market is inefficient or your risk model is incomplete.

It's like trying to determine whether your bathroom scale is accurate without access to any other way of measuring weight.

Sunshine and Stock Prices

Some of the evidence against market efficiency borders on the absurd.

Researchers have found that stock market performance is correlated with the amount of sunshine in the city where the main exchange is located. Sunny days in New York are associated with slightly higher returns on the New York Stock Exchange. This has been replicated across multiple countries and time periods.

There is no rational reason why sunshine in Manhattan should affect the value of corporations around the world. The most plausible explanation is that good weather improves traders' moods, making them slightly more optimistic and willing to buy. If true, this is a direct violation of market efficiency—prices are being influenced by information (the weather) that has no relevance to fundamental value.

The effect is small, and after accounting for transaction costs, it's not profitable to trade on. But its mere existence suggests that prices aren't purely driven by rational information processing.

The Curious Case of Closed-End Funds

Some of the most puzzling evidence comes from closed-end funds, a particular type of investment vehicle.

Most mutual funds are "open-end," meaning they can issue new shares when investors want to buy and redeem shares when investors want to sell. The price is always very close to the value of the underlying assets.

Closed-end funds are different. They issue a fixed number of shares at inception and then trade on stock exchanges like any other stock. The market price is whatever buyers and sellers agree on, and it can diverge significantly from the value of the assets the fund holds.

Here's the puzzle: closed-end funds frequently trade at substantial discounts to their net asset value. You might be able to buy a share in a fund holding a hundred dollars worth of stocks for only ninety dollars. Sometimes the discount reaches twenty or thirty percent.

This seems like free money. Why would anyone pay ninety dollars for a hundred dollars worth of assets? And if the discount persists, why doesn't someone buy up all the shares and liquidate the fund to capture the difference?

Various explanations have been proposed—taxes, management fees, illiquidity—but none fully account for the phenomenon, especially in extreme cases. The economists Owen Lamont and Richard Thaler called these anomalies violations of the "Law of One Price," the principle that identical goods should sell for identical prices.

The View From 2008

The global financial crisis of 2008 dealt a severe blow to faith in market efficiency.

The housing bubble that preceded the crisis was, in retrospect, obvious. Home prices had risen far beyond what incomes or rents could justify. Lending standards had collapsed. Mortgage-backed securities were being sold with ratings that bore no relation to their actual risk.

If markets were efficient, how could this happen? How could the collective wisdom of millions of investors, analysts, and rating agencies produce prices so wildly divorced from reality?

Richard Thaler, in a report on the crisis, identified "complexity and herd behavior" as central culprits. Financial instruments had become so complicated that even sophisticated investors couldn't evaluate them properly. And once prices started rising, people bought simply because others were buying, not because of any analysis of fundamental value.

The crisis didn't definitively disprove market efficiency—defenders could argue that the information needed to predict the crash was genuinely unavailable, or that the crisis represented a rational response to new information. But it made the strong claims of efficiency much harder to sustain.

What Remains

Where does this leave us? The efficient-market hypothesis, in its pure form, is almost certainly wrong. Markets are not perfect information processors. Prices can and do deviate from fundamental value, sometimes dramatically.

But the hypothesis captures something important. Markets are remarkably good at aggregating information, even if they're not perfect. Beating the market consistently is genuinely difficult, as the track records of most active fund managers attest. The average actively managed fund underperforms its benchmark index after fees.

Perhaps the fairest assessment is that markets are "efficiently inefficient"—efficient enough that most attempts to exploit inefficiencies fail, but inefficient enough that exceptional investors with superior information or analysis can succeed.

Daniel Kahneman, despite decades of research on investor biases, remains skeptical that ordinary investors can beat the market. "They're just not going to do it," he has said. "It's just not going to happen."

The efficient-market hypothesis has also had enormous practical influence. It provided the intellectual foundation for index funds—investment vehicles that simply buy all the stocks in a market index rather than trying to pick winners. If you can't beat the market, why not just buy it?

Index funds now hold trillions of dollars and have dramatically reduced the costs of investing for ordinary people. Even if the hypothesis that inspired them is imperfect, the practical benefits are real.

The Philosophical Stakes

At a deeper level, the debate over market efficiency is about the limits of human knowledge and the nature of collective intelligence.

If markets are efficient, then prices are a kind of oracle—a distillation of everything humanity knows about the future, updated in real time as new information arrives. No individual, no matter how brilliant, can reliably do better than this collective wisdom.

If markets are inefficient, then prices are partly illusion—shaped by fear, greed, herd behavior, and cognitive limitations. The collective can be wrong, sometimes spectacularly so. Individual insight and independent thinking can triumph over the crowd.

The truth probably lies somewhere in between. Markets are smarter than any individual, but they're not as smart as they think they are. Prices usually get things approximately right, but sometimes they get things catastrophically wrong.

For investors, this suggests humility. You're probably not going to beat the market, but the market isn't going to give you perfect guidance either. For society, it suggests caution about treating market prices as the final word on value. Markets are useful tools, but they're not infallible judges.

And for anyone tempted to think they've found a sure thing—a pattern in the data, a sector about to boom, a stock the market has overlooked—the efficient-market hypothesis offers a sobering question: If this is such a good opportunity, why hasn't someone else already taken it?