Are Prediction Markets Accurate? What the Research Shows

The Iowa market beat 964 polls 74% of the time and missed by about 1.5 points. What older research says about prediction prices, and where they fail.

OrderBlock.net Research··4 min read·42 views
Key takeaways
  • Short answer: in the best-studied case, US presidential elections, small-stakes prediction markets were more accurate than polls: the Iowa Electronic Markets missed the two-party vote by about 1.5 points in the final week against 2.1 for the final Gallup poll (four elections, 1988–2000).
  • A later study compared the Iowa market with 964 national polls over five elections, 1988–2004: the market was closer to the result 74% of the time, and significantly better in every election when the forecast was more than 100 days ahead.
  • A price is not automatically a probability. It equals the market’s expectation only under conditions (risk neutrality, enough liquidity), and economists documented a favorite–longshot bias: unlikely outcomes tend to be overpriced, especially on small exchanges.
  • The evidence is old and mostly political, with small stakes (the Iowa market capped positions at USD 500). It does not measure today’s large prediction markets, so it is evidence about the mechanism, not a guarantee for a particular contract.

Short answer. For elections, the research is favorable: small prediction markets forecast better than polls on average. The evidence is also old, narrow and conditional. A price is a useful estimate of what traders collectively believe, not a certified probability, and it is least reliable for rare events and for thin markets.

How accurate were prediction markets against polls?

The classic evidence is the Iowa Electronic Markets (IEM), a University of Iowa market where traders held positions of up to USD 500. Wolfers and Zitzewitz’s review reports that in the week before four US presidential elections the market’s forecast of the two-party vote had an average absolute error of around 1.5 percentage points; over the same elections the final Gallup poll erred by 2.1.1

Chart: Iowa Electronic Markets prediction error about 1.5 percentage points against 2.1 for the final Gallup poll, US presidential elections 1988–2000
In the week before four US presidential elections the Iowa market missed the two-party vote by about 1.5 points on average; the final Gallup poll by 2.1.

A longer study compared the market with 964 national polls over the five presidential elections from 1988 to 2004. The question: whether the poll or the contemporaneous market price was closer to the final two-party vote. The market was closer 74% of the time and “significantly outperforms the polls in every election” when forecasting more than 100 days ahead.2

Chart: in 964 comparisons over five US presidential elections the Iowa market was closer to the actual result than the poll 74% of the time
Across 964 national polls the market won the comparison three times in four. The market was best more than 100 days before the election.

Beyond elections, the 2004 review cites an internal market at Hewlett-Packard that forecast printer sales more accurately than the firm’s own process, and says a Hollywood film market predicted Oscar winners about as well as an expert panel.1

Is the price a probability?

For a “winner-take-all” contract that pays $1 if an event happens, the price is the market’s expectation of the probability, assuming risk neutrality.1 Two later papers show how fragile that sentence is. Manski argued that, with risk-neutral traders who hold different beliefs, the price only partially identifies the average belief: the mean belief lies in an interval whose midpoint is the price.3 Wolfers and Zitzewitz replied with sufficient conditions under which prices equal mean beliefs, and argued that across a broad class of models they are usually close, so prices offer useful, if sometimes biased, estimates of average belief.4

Diagram: how to read a prediction market price as a probability and three caveats: favorite-longshot bias, thin markets and price versus calibrated probability
A price of $0.62 is read as roughly 62% only under assumptions; the research lists three reasons to be careful.

Where do prediction markets go wrong?

  • Longshots are overpriced. There is a well-known favorite–longshot bias in horse racing, and the 2004 review found extreme outcomes for the S&P 500 relatively over-priced on TradeSports compared with prices implied by CME options, a gap that persisted for much of summer 2003. It adds that the bias seems more pronounced on smaller-scale exchanges.1
  • Thin markets can be pushed. Known manipulation attempts had no discernible effect except for short periods, because the profit motive draws in counter-trades; the review adds that how manipulable a market is depends at least partly on how thin it is.1 Compare the order-book version of this problem in what is spoofing.
  • Old, small, political. The evidence covers elections and small corporate markets up to the mid-2000s. It says nothing about the accuracy of the large markets of today, which have to be judged on their own track records.

What a trader can take from this

  • Read a price as an estimate with an error band. The Iowa market missed the vote share by about 1.5 points; for a rare event, a price of 3% may be worth less than it looks because of the longshot bias.
  • Check the market’s size before trusting it. Depth and the number of independent traders are what make the evidence above apply. A contract that one account can move says little.
  • A probability is an input, not a position size. If you hold your own estimate against a market price, sizing it is a different problem: see the Kelly criterion, where overestimating your edge is the main danger.

This is educational material, not investment advice.

Footnotes

  1. Wolfers, J., Zitzewitz, E., “Prediction Markets”, NBER Working Paper 10504 (2004; the Journal of Economic Perspectives version is 18(2), 107–126; working-paper text read). Winner-take-all contract and price as probability under risk neutrality; Iowa Electronic Markets average absolute error of about 1½ points in the final week against 2.1 for the final Gallup poll, four elections 1988–2000 (Figure 1); Hewlett-Packard and Hollywood markets; favorite–longshot bias and the TradeSports versus CME S&P 500 comparison (Table 4); manipulation attempts and thin markets. ↩ ↩2 ↩3 ↩4 ↩5

  2. Berg, J. E., Nelson, F. D., Rietz, T. A., “Prediction market accuracy in the long run”, International Journal of Forecasting 24(2), 283–298 (2008), as summarized on the University of Iowa Iowa Electronic Markets research page: 964 national polls, five US presidential elections 1988–2004, market closer 74% of the time, significantly better in every election more than 100 days ahead. ↩

  3. Manski, C. F., “Interpreting the predictions of prediction markets”, Economics Letters 91(3), 425–429 (2006); NBER Working Paper 10359 (abstract read): with risk-neutral price-taking traders with heterogeneous beliefs, the price partially identifies the mean belief, which lies in an interval whose midpoint is the price. ↩

  4. Wolfers, J., Zitzewitz, E., “Interpreting Prediction Market Prices as Probabilities”, NBER Working Paper 12200 (May 2006; abstract read): sufficient conditions for prices to equal mean beliefs; prices generally useful, if sometimes biased, estimates of average beliefs (paraphrase of the abstract). ↩

Frequently asked questions

Are prediction markets more accurate than polls?

In the studied US presidential elections, yes on average. The Iowa Electronic Markets’ final-week forecast erred by about 1.5 percentage points of the two-party vote against 2.1 for the final Gallup poll (1988–2000), and in 964 national polls over five elections (1988–2004) the market was closer to the result 74% of the time. These are small-stakes political markets, not a guarantee for any single contract.

Is the price of a prediction market contract a probability?

Roughly. A contract paying $1 if an event occurs trades at the market’s expectation of its probability under assumptions such as risk neutrality. Wolfers and Zitzewitz show prices are usually close to the mean belief of traders; Manski shows price alone only bounds the mean belief. Treat a price as a useful estimate, not an exact probability.

What is the favorite–longshot bias in prediction markets?

People tend to overvalue small probabilities and undervalue near certainties, so unlikely outcomes are often priced too high. Wolfers and Zitzewitz found extreme S&P 500 outcomes overpriced on TradeSports relative to CME option-implied prices, and said the bias seems stronger on smaller exchanges.

Can prediction markets be manipulated?

Known attempts in the studied markets had no discernible effect except for short periods, according to the 2004 review, because the profit motive brings in traders who bet against the distortion. The authors add that how manipulable a market is depends in part on how thin it is.

Do these studies cover Polymarket or Kalshi?

No. The cited work covers the Iowa Electronic Markets, TradeSports and internal company markets up to the mid-2000s. Modern markets differ in size, fees, contract types and participants, so their accuracy has to be measured on their own record.

Sources

  1. Prediction Markets. Justin Wolfers, Eric Zitzewitz. NBER Working Paper 10504 (2004); Journal of Economic Perspectives 18(2), 107–126, 2004, 2004
  2. Prediction market accuracy in the long run. Joyce E. Berg, Forrest D. Nelson, Thomas A. Rietz. International Journal of Forecasting 24(2), 283–298, 2008, 2008
  3. Interpreting Prediction Market Prices as Probabilities. Justin Wolfers, Eric Zitzewitz. NBER Working Paper 12200, May 2006, 2006
  4. Interpreting the predictions of prediction markets. Charles F. Manski. Economics Letters 91(3), 425–429, 2006 (NBER Working Paper 10359), 2006
#prediction-markets#forecasting#probability#polls#iowa-electronic-markets#longshot-bias
OrderBlock.net Research

The team behind the OrderBlock.net scanner. We read the primary research and exchange documentation so you do not have to, and cite every source.

This article is research, not investment advice. Results on history do not guarantee future results.

Related articles