Monday, March 14, 2011

From Order Books to Belief Distributions

In my last post I argued that the price at last trade in a prediction market doesn't generally correspond to the belief of an average or representative trader in any meaningful sense, and ought not to be interpreted loosely as the "perceived likelihood according to the market" that the underlying event will occur.

In contrast, the order book, which is the collection of all unexpired bids and offers that cannot currently be matched against each other, contains a wealth of information about the distribution of trader beliefs. Under certain assumptions about the risk preferences of market participants, one can deduce a distribution of trader beliefs from this collection of standing orders. The imputed distribution may then be used to infer what the average trader (in a well-defined sense) perceives the likelihood of the underlying event to be. Furthermore, it can be used to gauge the extent of disagreement about this likelihood within the trading population.

To illustrate, consider the order book for the contract PRESIDENT.DEM.2012, which pays $10 if the official nominee of the Democratic Party wins the next presidential election. At the time of writing my last post, the collection of unexpired and unfilled orders looked like this:

BID
QtyPrice
1862.5
44162.1
50062.0
461.1
7961.0
8660.0
159.9
17359.0
158.4
157.7
6757.4
156.8
20056.6
556.3
10056.1
ASK
PriceQty
62.91
63.130
63.41
63.52
63.61
63.710
63.86
63.91
64.0111
64.12
64.21
64.5100
64.6101
64.810
65.0200


Prices are expressed as percentages of face value, so the highest bidder was willing to pay $6.25 per contract, while the lowest offer was at $6.29 per contract. The frequency distributions of orders on the two sides of the market were as follows:



These distributions necessarily have disjoint supports, otherwise some orders would be matched and exit the book. The median bid was 62.0 per contract, while the median offer was 64.6. 

How might one deduce a belief distribution from this data?

The first point to note is that anyone placing an order that cannot immediately be filled (and does not immediately expire) faces two kinds of risk. There is the obvious risk that the event may or may not occur; even those whose orders trade immediately are exposed to this. But for an order that remains in the book for some time there is a second source of risk: new information might arrive that substantially alters the likelihood that the event will occur, and results in the order being matched before it can be removed. Given these two sources of risk, traders will post bids at prices that are below their subjective estimates of the likelihood that the event in question will occur. Similarly, those who post offers to sell will do so at prices that lie above their subjective estimates of this likelihood.

A simple way to take these effects into account is to assume that the risk preferences of traders are given by a linear mean-variance objective function of the kind that may be found in any standard text on Investments, with risk aversion parameter A. As an example to illustrate the procedure, suppose that all traders have the same degree of risk-aversion given by A = 0.15, and that buyers post the highest price that they are willing to pay for the asset, while sellers post the lowest price that they are willing to accept. Then the order distribution implies the following distributions of beliefs on the two sides of the market:



Note first that buyers on the whole assign greater likelihood to the occurrence of the underlying event than sellers do, even though all bid prices lie strictly below the lowest of the offer prices. This is a direct consequence of risk-aversion, which induces buyers to post prices below their subjective beliefs and sellers to offer at prices above theirs. The median buyer belief is 0.65 while the median seller belief is 0.59.

Second, buyer beliefs are spread across a wider range than are the beliefs of sellers. This simply replicates a pattern in the order book, which is characterized by many large bids at varying prices but a concentration of offers in a narrower price range.

Third, the belief supports are not disjoint: there is a range of beliefs that is represented on both sides of the market. These beliefs probably correspond to orders placed by market makers who simultaneously place bids and offers with the aim of profiting from the spread. For instance, the bid for 200 contracts at 56.6 and the offer of 200 at 65.0 both imply an imputed belief of about 0.60 under the assumed value for the risk-aversion parameter. It is quite conceivable, indeed very likely, that these orders were placed by the same individual.

Aggregating the buyer and seller belief distributions yields the belief distribution for the market as a whole:



Since bids are more numerous than offers, this aggregate belief distribution lies closer to that for buyers. The median belief in this case is 0.63, which happens to be slightly above the price at last trade.

This is one way of making precise the idea of "the perceived likelihood according to the market." Under the specifications adopted here, this perception is close to the equilibrium price. But it need not be in general. Higher values of the risk-aversion parameter would generate belief distributions for buyers and sellers that are further apart. While the theoretical effect of this on the median belief is ambiguous, for the particular example considered here, a risk-aversion parameter of A = 0.25 would generate a median belief of 0.65.

Furthermore, changes in beliefs within the population of traders could make their presence known through changes in bids and offers without any change in the equilibrium price. That was essentially the point of my last post: any given equilibrium price is consistent with a broad range of belief distributions. By focusing on the complete distribution (rather than just a point estimate) one can get a better sense of where market perceptions really lie.

One interesting question that follows from the arguments advanced here is this: could one use an imputed belief distribution to predict short-term movements in the equilibrium price?

Not necessarily. Even if one felt that the belief corresponding to the median order was in some sense the best forecast regarding the likelihood of the underlying event, one would not be induced to place an order that moves the equilibrium price. This is simply due to the fact that bids lie below subjective beliefs while offer prices lie above them. If large numbers of individuals simply adopted the imputed median belief as their own forecast, they would be induced to enter bids and offers around this belief, affecting the variance of the belief distribution but not necessarily its median. Nevertheless, it is worth noting that high-frequency trading outfits in US equity markets do manage to use proprietary data feeds to make effective short-run price forecasts.

As Andrew Gelman put it in his (very kind) response to my earlier post:
Markets are impressive mechanisms for information aggregation but they're not magic. The information has to come from somewhere, and markets are inherently always living in the phase transition between stability and instability... This is not to say that prediction markets are useless, just that they are worth studying seriously in their own right, not to be treated as oracles.
Prediction markets are indeed worth studying seriously not only because they are complex and interesting mechanisms for information aggregation, but also because the simplicity of the contracts traded can allow strong inferences to be made about the behavior of market participants. And some of these insights could be generalized to apply to speculative asset markets with much greater volume, liquidity, and economic importance.

Sunday, March 06, 2011

On the Interpretation of Prediction Market Data

As the election season draws closer, considerable attention will be paid to prices in prediction markets such as Intrade. Contracts for potential presidential nominees are already being scrutinized for early signs of candidate strength. In a recent post on the 2012 Republican field, Nate Silver used prediction market data (among other sources of information) to generate the following very interesting chart:


Source: FiveThirtyEight: Nate Silver's Political Calculus

While Nate's post was concerned primarily with the positioning of candidates along two-dimensions of the political spectrum, he used market prices as a proxy for the probabilities of eventual nomination:
[The] area of each candidate’s circle is proportional to their perceived likelihood of winning the nomination, according to the Intrade betting market. Mitt Romney’s circle is drawn many times the size of the one for the relatively obscure talk-radio host Herman Cain because Intrade rates Mr. Romney many times as likely to be nominated.
This interpretation of prices as probabilities is common and will be repeated frequently over the coming months. But what could the "perceived likelihood according to the market" possibly mean?

Markets don't have perceptions. Traders do, but there is considerable heterogeneity in trader beliefs at any point in time. Prediction market prices contain valuable information about this distribution of beliefs, but there is no basis for the common presumption that the price at last trade represents the beliefs of a hypothetical average trader in any meaningful sense. In fact, to make full use of market data to make inferences about the distribution of beliefs, one needs to look beyond the price at last trade and examine the entire order book.

As an example, consider Intrade's market for the presidential election winner by party. This market consists of three contracts comprising a mutually exclusive and exhaustive set of outcomes. One  contract pays out if the winner is a Democrat, a second if the winner is a Republican, and the third if the winner is not the official nominee of either party. The current prices of these contracts are as follows:

Contract Bid AskLast
PRESIDENT.DEM.2012 62.562.9 62.5
PRESIDENT.REP.2012 35.1 35.5 35.0
PRESIDENT.OTHER.2012 2.2 2.3 2.2


These prices are expressed as percentages of contract face value, which in each case is $10. That is, the price at last trade of the DEM contract was $6.25. The buyer risks this amount (per contract purchased) and stands to receive $10 if (and only if) the specified event occurs. The seller risks $3.75 to take the opposite side of the bet.

It's tempting to interpret the price at last trade as a probability because the sum of these prices adds up to approximately 100% of the contract face value. The reason for this is that the sum of the ask prices must be no less than 100, otherwise an arbitrage opportunity would exist: one could buy all contracts and be sure that one will expire at face value, thus generating in a risk-free profit. Similarly, the sum of bid prices must be no greater than 100. If the market is liquid, so that bid-ask spreads are small, then all prices (bid, ask, and last) will sum to approximately 100. This is the basis for the claim that, at current prices, the "market" is predicting that the Democratic nominee will win the White House with probability 62.5%.

But is this interpretation reasonable? All that the price at last trade can tell us about is the beliefs of the two parties to this transaction. If both are risk-averse or risk-neutral, they each must believe that entering their respective positions will yield a positive expected return. Hence the buyer must assign probability at least 62.5% to the event that the Democrat is elected, while the seller assigns a likelihood of at most 62.5% to this event.

This tells us nothing about the beliefs of traders who are not party to this transaction. However, additional information about the distribution of beliefs in the trader population can be obtained by looking at the order book, which at present looks like this:

BID
QtyPrice
1862.5
44162.1
50062.0
461.1
7961.0
8660.0
159.9
17359.0
158.4
157.7
6757.4
156.8
20056.6
556.3
10056.1
ASK
PriceQty
62.91
63.130
63.41
63.52
63.61
63.710
63.86
63.91
64.0111
64.12
64.21
64.5100
64.6101
64.810
65.0200


Note that there are several large orders (in excess of 100 contracts) but these are unevenly distributed on the two sides of the market. Consider, for instance, the bid for 500 contracts at 62. Whenever such an order is placed, Intrade freezes funds in the trader's account equal to the worst case loss, which in this case is $3,100. Upon expiration, these contracts will be worth either $5,000 (if the event occurs) or they will be worthless. Again, assuming risk-aversion or risk-neutrality, one can impute to the potential buyer a belief that the event will occur with probability at least 62%.

But this imputation will be an underestimate for at least two reasons. First, the greater the degree of risk-aversion, the more compensation a trader will demand to enter a risky position. Since these positions are indeed very risky, it is likely that many of those placing large standing bids have significantly positive expected returns, and hence believe that the probability of the event occurring exceeds by some measure the imputed value.

Second, traders placing large bids are aware that conditional on their order being met, it is likely that some news will have emerged that makes the event less likely to occur. That is, they understand that a trade against their order is more likely to occur in the event of bad news (from their perspective) than good news. Taken together, these factors imply that traders placing large bids must be considerably more optimistic about the occurrence of the event than the naive imputation of 62% would suggest.

The same reasoning applies to those taking positions on the sell side: traders placing large limit orders must believe that the event is considerably less likely to occur than a naive reading of their posted price would suggest.

What, then, can one say about the distribution of beliefs in the market? To begin with, there is considerable disagreement about the outcome. Second, this disagreement itself is public information: it persists despite the fact that it is commonly known to exist. That is, traders don't attribute differences in beliefs simply to differences in information applied rationally to a common prior. (This follows from Aumann's famous theorem which states that individuals who have common priors and are commonly known to be rational cannot agree to disagree no matter how different their private information may be.) As a result, the fact of disagreement is not itself considered to be informative, and does not lead to further belief revision. The most likely explanation for this is that traders harbor doubts about the rationality or objectivity of other market participants.

Third, there is a cluster of large buy orders at around 62, and a cluster of large sell orders in the 64-65 range. Hence there are some traders who believe quite confidently that Democrats will hold the White House with probability considerably greater than 62%. And there is another group who believe, also confidently, that the chances of this occurring are quite a bit below 64%. As things stand, the former group appear to be either more numerous or more confident in their judgments.

More generally, it is entirely possible that beliefs are distributed in a manner that is highly skewed around the price at last trade. That is, it could be the case that most traders (or the most confident traders) all fall on one side of the order book. In this case the arrival of seemingly minor pieces of information can cause a large swing in the market price. Of course, such swings may draw into the market other participants whose beliefs are not currently represented in the order book. But the bottom line is this: there is no meaningful sense in which one can interpret the price at last trade as an average or representative belief among the trading population.