Learning Performance of Prediction Markets with Kelly Bettors Alina Beygelzimer John Langford David Pennock IBMResearch Yahoo! Research Yahoo! Research [email protected] [email protected] pennockd@yahoo- inc.com ABSTRACT Keywords 2 Inevaluatingpredictionmarkets(andothercrowd-prediction Auctionandmechanismdesign,electronicmarkets,econom- 1 mechanisms), investigators have repeatedly observed a so- ically motivated agents, multiagent learning 0 called wisdom of crowds effect, which can be roughly sum- 2 marized as follows: the average of participants performs 1. INTRODUCTION muchbetterthantheaverageparticipant. Themarketprice— n an average or at least aggregate of traders’ beliefs—offers a Consider a gamble on a binary event, say, that Obama a better estimate than most any individual trader’s opinion. will win the 2012 US Presidential election, where every x J In this paper, we ask a stronger question: how does the dollars risked earns xb dollars in net profit if the gamble 1 market price compare to the best trader’s belief, not just paysoff. Howmanydollarsxofyourwealthshouldyourisk 3 theaveragetrader. Wemeasurethemarket’sworst-caselog if you believe the probability is p? The gamble is favorable regret,anotioncommoninmachinelearningtheory. Toar- ifbp−(1−p)>0,inwhichcasebettingyourentirewealthw ] I rive at a meaningful answer, we need to assume something willmaximizeyourexpectedprofit. However,that’sextraor- A about how traders behave. We suppose that every trader dinarily risky: a single stroke of bad luck loses everything. . optimizes according to the Kelly criteria, a strategy that Over the course of many such gambles, the probability of s provably maximizes the compound growth of wealth over bankruptcy approaches 1. On the other hand, betting a c [ an (infinite) sequence of market interactions. We show sev- small fixed amount avoids bankruptcy but cannot take ad- eral consequences. First, the market prediction is a wealth- vantage of compounding growth. 1 weightedaverageoftheindividualparticipants’beliefs. Sec- The Kelly criteria prescribes choosing x to maximize the v ond,themarketlearns attheoptimalrate,themarketprice expected compounding growth rate of wealth, or equiva- 5 reacts exactly as if updating according to Bayes’ Law, and lently to maximize the expected logarithm of wealth. Kelly 5 the market prediction has low worst-case log regret to the bettingisasymptoticallyoptimal,meaningthatinthelimit 6 bestindividualparticipant. Wesimulateasequenceofmar- overmanygambles,aKellybettorwillgrowwealthierthan 6 kets where an underlying true probability exists, showing an otherwise identical non-Kelly bettor with probability 1 1. that the market converges to the true objective frequency [1, 3, 7, 16, 17]. 0 asifupdatingaBetadistribution,asthetheorypredicts. If Assume all agents in a market optimize according to the 2 agentsadoptafractionalKellycriteria,acommonpractical Kelly principle, where b is selected to clear the market. We 1 variant, we show that agents behave like full-Kelly agents consider the implications for the market as a whole and : with beliefs weighted between their own and the market’s, properties of the market odds b or, equivalently, the mar- v i and that the market price converges to a time-discounted ket probability pm = 1/(1+b). We show that the market X frequency. Ouranalysisprovidesanewjustificationforfrac- prediction pm is a wealth-weighted average of the agents’ r tional Kelly betting, a strategy widely used in practice for predictions pi. Over time, the market itself—by reallocat- a ad-hoc reasons. Finally, we propose a method for an agent ing wealth among participants—adapts at the optimal rate to learn her own optimal Kelly fraction. withboundedlogregrettothebestindividualagent. When a true objective probability exists, the market converges to it as if properly updating a Beta distribution according to CategoriesandSubjectDescriptors Bayes’ rule. These results illustrate that there is no“price I.2.11 [Artificial Intelligence]: Distributed Artificial In- of anarchy”associated with well-run prediction markets. telligence—Intelligent agents, Multiagent systems WealsoconsiderfractionalKellybetting,alower-riskvari- ant of Kelly betting that is popular in practice but has less GeneralTerms theoretical grounding. We provide a new justification for fractional Kelly based on agent’s confidence. In this case, Economics the market prediction is a confidence-and-wealth-weighted averagethatempiricallyconvergestoatime-discountedver- sion of objective frequency. Finally, we propose a method for agents to learn their optimal fraction over time. ShortVersionAppearsin:Proceedingsofthe11thInter- nationalConferenceonAutonomousAgentsandMultiagent Systems (AAMAS 2012),Conitzer,Winikoff,Padgham,andvander 2. KELLYBETTING Hoek(eds.),June,4–8,2012,Valencia,Spain. When offered b-to-1 odds on an event with probability p, odds reached when all agents are optimizing, and supply the Kelly-optimal amount to bet is f∗w, where anddemandarepreciselybalanced. Recallthatthemarket’s probability implied by the odds of b is p =1/(1+b). We f∗ = bp−(1−p) will show that p is (cid:80) w p . m b m i i i is the optimal fixed fraction of total wealth w to commit to 4.1 Payoutbalance the gamble. Thefirstapproachwe’lluseispayoutbalance: theamount If f∗ is negative, Kelly says to avoid betting: expected ofmoneyatriskmustbethesameastheamountpaidout. profitisnegative. Iff∗ ispositive,youhaveaninformation edge; Kelly says to invest a fraction of your wealth propor- Theorem 1. (Market Pricing) For all normalized agent tional to how advantageous the bet is. In addition to max- wealths w and agent beliefs p , i i imizing the growth rate of wealth, Kelly betting maximizes (cid:88) thegeometricmeanofwealthandasymptoticallyminimizes pm = piwi the mean time to reach a given aspiration level of wealth i [17]. Proof. To see this, recall that f∗ = (p −p )/(1−p ) for Supposefairoddsof1/baresimultaneouslyofferedonthe i i m m p > p . For p < p , Kelly betting prescribes taking the oppositeoutcome(e.g.,Obamawillnot wintheelection). If i m i m other side of the bet, with fraction bp−(1−p) < 0, then betting on this opposite outcome is favorable;substituting1/bforband1−pforp,theoptimal (1−pi)−(1−pm) = pm−pi. fraction of wealth to bet becomes 1−p−bp. 1−(1−p ) p m m An equivalent way to think of a gamble with odds b is as Sothemarketequilibriumoccursatthepointp wherethe apredictionmarketwithpricep =1/(1+b). Thevolume m m payout is equal to the payin. If the event occurs, the payin of bet is specified by choosing a quantity q of shares, where is each share is worth $1 if the outcome occurs and nothing otherwise. The price represents the cost of one share: the (1+b) (cid:88) pi−pmw = 1 (cid:88) pi−pmw . amount needed to pay for a chance to win back $1. In this 1−pm i pm 1−pm i i:pi>pm i:pi>pm interpretation, the Kelly formula becomes Thus we want p−p f∗ = 1−pmm. p1 (cid:88) p1i−−ppmwi = (cid:88) p1i−−ppmwi + m m m The optimal action for the agent is to trade q∗ = f∗w/pm i:pi>pm i:pi>pm shares, where q∗ > 0 is a buy order and q∗ < 0 is a sell (cid:88) pm−piw , or order, or a bet against the outcome. pm i Note that q∗ is the optimum of expected log utility i:pi<pm 1−pm (cid:88) pi−pmw = (cid:88) pm−piw , or pln((1−pm)q+w)+(1−p)ln(−pmq+w). pm 1−pm i pm i i:pi>pm i:pi<pm Thisisnotacoincidence: Kellybettingisidenticaltomax- (cid:88) (cid:88) (p −p )w = (p −p )w , or imizing expected log utility. i m i m i i i:pi>pm i:pi<pm (cid:88) (cid:88) 3. MARKETMODEL p w = p w . i i m i Suppose that we have a prediction market, where partic- i i ipant i has a starting wealth wi with (cid:80)iwi = 1. Each Using (cid:80)iwi =1, we get the theorem. participantiusesKellybettingtodeterminethefractionf∗ i 4.2 Logutilitymaximization oftheirwealthbet,dependingontheirpredictedprobability pi. An alternate derivation of the market prediction utilizes We model the market as an auctioneer matching sup- the fact that Kelly betting is equivalent to maximizing ex- ply and demand, taking no profit and absorbing no loss. pectedlogutility. Let q=x(b+1) be the gross profitof an We adopt a competitive equilibrium concept, meaning that agent who risks x dollars, or in prediction market language agentsare”pricetakers”,ordonotconsidertheirowneffect the number of shares purchased. Then expected log utility on prices if any. Agents optimize according to the current is priceanddonotreasonfurtheraboutwhatthepricemight E[U(q)]=pln((1−p )q+w)+(1−p)ln(−p q+w). reveal about the other agents’ information. An exception m m ofsortsisthefractionalKellysetting,whereagentsdocon- The optimal q that maximizes E[U(q)] is sider the market price as information and weigh it along w p−p with their own. q(p )= · m. (1) A market is in competitive equilibrium at price pm if all m pm 1−pm agents are optimizing and (cid:80)iqi∗ = 0, or every buy order Proposition 2. In a market of agents each with log utility andsellorderarematched. Wediscussnextwhatthevalue and initial wealth w, the competitive equilibrium price is of p is. m (cid:88) p = w p (2) m i i 4. MARKETPREDICTION i (cid:80) In order to define the prediction market’s performance, where we assume w =1, or w is normalized wealth not i i we must define its prediction b, or the equilibrium payoff absolute wealth. (cid:80) Proof. These prices satisfy q = 0, the condition for as i i competitiveequilibrium(supplyequalsdemand),bysubsti- T tution. (cid:50) L≡(cid:88)I(y =1)log 1 +I(y =0)log 1 . t p t 1−p t t This result can be seen as a simplified derivation of that t=1 by Rubinstein [13, 14, 15] and is also discussed by Pennock Similarly,wemeasurethequalityofmarketparticipantmak- and Wellman [11, 10] and Wolfers and Zitzewitz [18]. ing prediction p as it T (cid:88) 1 1 5. LEARNINGPREDICTIONMARKETS Li ≡ I(yt =1)logp +I(yt =0)log1−p . it it t=1 Individualparticipantsmayhavevaryingpredictionqual- ities and individual markets may have varying odds of pay- So after T rounds, the total wealth of player i is off. Whathappenstothewealthdistributionandhencethe quality of the market prediction over time? We show next w (cid:89)T (cid:18)pit(cid:19)yt(cid:18)1−pit(cid:19)1−yt, that the market learns optimally for two well understood i pt 1−pt t=1 senses of optimal. wherew isthestartingwealth. Wenextproveawell-known i theorem for learning in the present context (see for exam- 5.1 Wealth redistributed according to Bayes’ ple [4]). Law Theorem3. Forallsequencesofparticipantpredictionsp In an individual round, if an agent’s belief is p > p , it i m then they bet pi−pmw and have a total wealth afterward and all sequences of revealed outcomes yt, 1−pm i dependent on y according to: 1 L≤minL +ln . i i wi (cid:18) 1 (cid:19)p −p p Thistheoremisextraordinarilygeneral,asitappliestoall If y=1, −1 i mw +w = i w p 1−p i i p i market participants and all outcome sequences, even when m m m these are chosen adversarially. It states that even in this p −p 1−p If y=0, (−1)1i−pmwi+wi = 1−pi wi worst-casesituation,themarketperformsonlyln1/wiworse m m than the best market participant i. (cid:80) Proof. Initially, we have that w = 1. After T rounds, i i Similarly if pi <pm, we get: the total wealth of any participant i is given by If y=1, (−1)pmp−mpiwi+wi = ppmi wi wi(cid:89)T (cid:18)ppit(cid:19)yt(cid:18)11−−ppit(cid:19)1−yt =wieL−Li ≤1, (cid:18) 1 (cid:19)p −p 1−p t=1 t t If y=0, −1 m iw +w = i w , 1−p p i i 1−p i where the last inequality follows from wealth being con- m m m served. Thus lnw +L−L ≤0, yielding i i which is identical. 1 If we treat the prior probability that agent i is correct L≤Li+lnw . as w , Bayes’ law states that the posterior probability of i i choosing agent i is P(i|y=1)= P(y=1|i)P(i) = piwi = (cid:80)piwi , 6. FRACTIONALKELLYBETTING P(y=1) p p w m i i i Fractional Kelly betting says to invest a smaller fraction λf∗ ofwealthforλ<1. FractionalKellyisusuallyjustified which is precisely the wealth computed above for the y=1 on an ad-hoc basis as either (1) a risk-reduction strategy, outcome. Thesameholdswheny=0,andsoKellybettors since practitioners often view full Kelly as too volatile, or redistribute wealth according to Bayes’ law. (2) a way to protect against an inaccurate belief p, or both 5.2 MarketSequences [17]. Herewederiveanalternateinterpretationoffractional Kelly. In prediction market terms, the fractional Kelly for- ItiswellknownthatBayes’lawisthecorrectapproachfor mula is integrating evidence into a belief distribution, which shows p−p that Kelly betting agents optimally summarize all past in- λ m. 1−p formation if the true behavior of the world was drawn from m the prior distribution of wealth. With some algebra, fractional Kelly can be rewritten as Often these assumptions are too strong—the world does p(cid:48)−p notbehaveaccordingtotheprioronwealth,anditmayact m 1−p inamannercompletelydifferentfromanyonesingleexpert. m Inthatcase,astandardanalysisfromlearningtheoryshows where thatthemarkethaslowregret,performingalmostaswellas p(cid:48) =λp+(1−λ)p . (3) the best market participant. m Foranyparticularsequenceofmarketswehaveasequence In other words, λ-fractional Kelly is precisely equivalent to p ofmarketpredictionsandy ∈{0,1}ofmarketoutcomes. fullKellywithrevisedbeliefλp+(1−λ)p ,oraweightedav- t t m We measure the accuracy of a market according to log loss erageoftheagent’soriginalbeliefandthemarket’sbelief. In this light, fractional Kelly is a form of confidence weighting wheretheagentmixesbetweenremainingsteadfastwithits ownbelief(λ=1)andaccedingtothecrowdandtakingthe market price as the true probability (λ=0). The weighted average form has a Bayesian justification if the agent has a BetaprioroverpandhasseentindependentBernoullitrials toarriveatitscurrentbelief. Iftheagentenvisionsthatthe market has seen t(cid:48) trials, then she will update her belief to λp+(1−λ)p ,whereλ=t/(t+t(cid:48))[9,10,12]. Theagent’s m posteriorprobabilitygiventhepriceisaweightedaverageof its prior and the price, where the weighting term captures her perception of her own confidence, expressed in terms of the independent observation count seen as compared to the market. 1.0 7. MARKETPREDICTIONWITHFRACTIONAL 0.9 KELLY When agents play fractional Kelly, the competitive equi- 0.8 libriumpricenaturallychanges. Theresultingmarketprice is easily compute, as for fully Kelly agents. 0.7 Theorem4. (FractionalKellyMarketPricing)Forallagent beliefs p , normalized wealths w and fractions λ 0.6 i i i (cid:80) λ w p pm = (cid:80)ilλilwili. (4) (a) 20 40 60 80 100 120 140 Prices retain the form of a weighted average, but with 0.035 weightsproportionaltotheproductofwealthandself-assessed 0.030 confidence. 0.025 0.020 Proof. TheproofisastraightforwardcorollaryofTheorem1. In particular, we note that a λ-fractional Kelly agent of 0.015 wealth w bets precisely as a full-Kelly agent of wealth λw. 0.010 Consequently, we can apply theorem 1 with wi(cid:48) = (cid:80)λiiλwiiwi 0.005 and p(cid:48) =p unchanged. i i 0.000 0.2 0.4 0.6 0.8 1.0 (b) 8. MARKET DYNAMICS WITH STATION- ARYOBJECTIVEFREQUENCY Figure 1: (a) Price (black line) versus the observed Theworst-caseboundsaboveholdevenifeventoutcomes frequency (gray line) of the event over 150 time pe- are chosen by a malicious adversary. In this section, we riods. The market consists of 100 full-Kelly agents examine how the market performs when the objective fre- with initial wealth wi = 1/100. (b) Wealth after quency of outcomes is unknown though stationary. 15 time periods versus belief for 100 Kelly agents. The market consists of a single bet repeated over the The event has occurred in 10 of the 15 trials. The course of T periods. Unbeknown to the agents, each event solid line is the posterior Beta distribution consis- unfoldsasanindependentBernoullitrialwithprobabilityof tent with observing 10 successes in 15 independent successπ. Atthebeginningoftimeperiodt,therealization Bernoulli trials. of event E is unknown and agents trade until equilibrium. t Thentheoutcomeisrevealed, andtheagents’holdingspay offaccordingly. Astimeperiodt+1begins,theoutcomeof E isuncertain. Agentsbetonthet+1periodeventuntil t+1 equilibrium, the outcome is revealed, payoffs are collected, and the process repeats. In an economy of Kelly bettors, the equilibrium price is a wealth-weighted average (2). Thus, as an agent accrues relatively more earnings than the others, its influence on priceincreases. Inthenexttwosubsections,weexaminehow this adaptive process unfolds; first, with full-Kelly agents andsecond,withfractionalKellyagents. Intheformercase, prices react exactly as if the market were a single agent updating a Beta distribution according to Bayes’ rule. 8.1 Marketdynamicswithfull-Kellyagents Figure 1.a plots the price over 150 time periods, in a market composed of 100 Kelly agents with initial wealth w = 1/100, and p generated randomly and uniformly on i i (0,1). In this simulation the true probability of success π is 0.5. For comparison, the figure also shows the observed frequency, or the number of times that E has occurred di- vided by the number of periods. The market price tracks the observed frequency extremely closely. Note that price changesaredueentirelytoatransferofwealthfrominaccu- rate agents to accurate agents, who then wield more power in the market; individual beliefs remain fixed. Figure 1.b illustrates the nature of this wealth transfer. Thegraphprovidesasnapshotofagents’wealthversustheir belief p after period 15. In this run, E has occurred in 10 i outofthe15trials. Themaximuminwealthisnear10/15or 0.5 2/3. The solid line in the figure is a Beta distribution with parameters 10+1 and 5+1. This distribution is precisely 0.4 the posterior probability of success that results from the observation of 10 successes out of 15 independent Bernoulli 0.3 trials, when the prior probability of success is uniform on (0,1). Thefitisessentiallyperfect,andcanbeprovedinthe 0.2 limitsincetheBetadistributionisconjugatetotheBinomial distribution under Bayes’ Law. 0.1 Although individual agents are not adaptive, the mar- ket’s composite agent computes a proper Bayesian update. Specifically, wealth is reallocated proportionally to a Beta (a) 20 40 60 80 100 120 140 distribution corresponding to the observed number of suc- cesses and trials, and price is approximately the expected value of this Beta distribution.1 Moreover, this correspon- 0.5 denceholdsregardlessofthenumberofsuccessesorfailures, 0.4 or the temporal order of their occurrence. A kind of collec- tiveBayesianityemerges fromtheinteractionsofthegroup. 0.3 We also find empirically that, even if not all agents are Kelly bettors, among those that are, wealth is still redis- 0.2 tributed according to Bayes’ rule. 0.1 8.2 MarketdynamicswithfractionalKellyagents In this section, we consider fractional Kelly agents who, as we saw in Section 2, behave like full Kelly agents with (b) 20 40 60 80 100 120 140 belief λp+(1−λ)p . Figure 2.a graphs the dynamics of m price in an economy of 100 such agents, along with the ob- Figure 2: (a) Price (black line) versus observed fre- servedfrequency. Overtime,thepriceremainssignificantly quency (gray line) over 150 time periods for 100 more volatile than the frequency, which converges toward agents with Kelly fraction λ=0.2. As the frequency π =0.5. Below, we characterize the transfer of wealth that converges to π =0.5, the price remains volatile. (b) precipitatesthisaddedvolatility;fornowconcentrateonthe Price (black line) versus discounted frequency (gray price signal itself. Inspecting Figure 2.a, price changes still line), with discount factor γ = 0.96, for the same exhibitamarkeddependenceoneventoutcomes,thoughat experiment as (a). anygivenperiodtheeffectofrecenthistoryappearsmagni- fied,andthepastdiscounted,ascomparedwiththeobserved frequency. Working from this intuition, we attempt to fit thedatatoanappropriatelymodifiedmeasureoffrequency. Define the discounted frequency at period n as (cid:80)n γn−t(1 ) dn = (cid:80)n γn−t(1t=1 )+(cid:80)nE(t)γn−t(1 ), (5) t=1 E(t) t=1 E(t) where1 istheindicatorfunctionfortheeventatperiod E(t) t,andγ isthediscount factor. Notethatγ =1recoversthe standard observed frequency. 1Astgrows,thisexpectedvaluerapidlyapproachestheob- served frequency plotted in Figure 1. 0.020 For example, if you allocate an initial weight of 0.5 to your predictions and 0.5 to the market’s prediction, then the re- gretguaranteeofsection5.2impliesthatatmosthalfofall 0.015 wealth is lost. 0.010 10. DISCUSSION We’ve shown something intuitively appealing here: self- 0.005 interestedagentswithlogwealthutilitycreatemarketswhich learn to have small regret according to log loss. There are 0.000 two distinct“log”s in this statement, and it’s appealing to 0.2 0.4 0.6 0.8 1.0 consider what happens when we vary these. When agents havesomeutilityotherthanlogwealthutility,canwealter Figure3: (a)Wealthw versusbeliefp atperiod150 thestructureofamarketsothatthemarketdynamicsmake i i of the same experiment as Figure 2 with 100 agents the market price have low log loss regret? And similarly if with Kelly fraction λ=0.2. The observed frequency we care about some other loss—such as squared loss, 0/1 is 69/150 and the solid line is Beta(69 + 1,81 + 1). loss,oraquantileloss,canwecraftamarketplacesuchthat The wealth distribution is significantly more evenly logwealthutilityagentsachievesmallregretwithrespectto dispersed than the corresponding Beta distribution. these other losses? What happens in a market without Kelly bettors? This can’tbedescribedingeneral,althoughacouplespecialcases Figure2.billustratesaveryclosecorrelationbetweendis- are relevant. When all agents have constant absolute risk counted frequency, with γ = 0.96 (hand tuned), and the aversion, the market computes a weighted geometric aver- same price curve of Figure 2.a. While standard frequency age of beliefs [10, 11, 13]. When one of the bettors acts provides a provably good model of price dynamics in an according to Kelly and the others in some more irrational economy of full-Kelly agents, discounted frequency (5) ap- fashion. Inthiscase,thebasicKellyguaranteeimpliesthat pears a better model for fractional Kelly agents. the Kelly bettor will come to dominate non-Kelly bettors Toexplaintheclosefittodiscountedfrequency,onemight with equivalent or worse log loss. If non-Kelly agents have expect that wealth remains dispersed—as if the market’s a better log loss, the behavior can vary, possibly imposing composite agent witnesses fewer trials than actually occur. greaterregretonthemarketplaceiftheKellybettoraccrues That’struetoanextent. Figure3showsthedistributionof the wealth despite a worse prediction record. For this rea- wealthafter69successeshaveoccurredin150trials. Wealth son, it may be desirable to make Kelly betting an explicit issignificantlymoreevenlydistributedthanaBetadistribu- option in prediction markets. tionwithparameters69+1and81+1,alsoshown. However, the stretcheddistribution can’t be modeled precisely as an- 11. REFERENCES other, less-informed Beta distribution. [1] L. Breiman. Optimal gambling systems for favorable games. In Berkeley Symposium on Probability and 9. LEARNINGTHEKELLYFRACTION Statistics, I, pages 65–78, 1961. In theory, a rational agent playing against rational oppo- [2] N. Cesa-Bianchi, Y. Freund, D. Helbold, D. Haussler, nents should set their Kelly fraction to λ = 0, since, in a R. Schapire, and M. Warmuth. How to use expert rationalexpectationsequilibrium[6],themarketpriceisby advice. Journal of the ACM, 44(3):427–485, 1997. definition at least as informative as any agent’s belief. This [3] T. M. Cover and J. A. Thomas. Elements of isthe crux of the no-tradetheorems[8]. Despitethetheory Information Theory, Second Edition. [5], people do agree to disagree in practice and, simply put, Wiley-Interscience, New Jersey, 2006. tradehappens. Still,placingsubstantialweightonthemar- [4] Y. Freund, R. Schapire, Y. Singer, and M. Warmuth. ketpriceisoftenprudent. Forexample,inanonlinepredic- Using and combining predictors that specialize. In tion contest called ProbabilitySports, 99.7% of participants Proceedings of the Twenty-Ninth Annual ACM were outperformed by the unweighted average predictor, a Symposium on the Theory of Computing, pages typical result.2 334–343, 1997. In this light, fractional Kelly can be seen as an experts [5] J. D. Geanakoplos and H. M. Polemarchakis. We can’t algorithm [2] with two experts: yourself and the market. disagree forever. Journal of Economic Theory, We propose dynamically updating λ according to standard 28(1):192–200, 1982. experts algorithm logic: When you’re right, you increase [6] S. J. Grossman. An introduction to the theory of λ appropriately; when you’re wrong, you decrease λ. This rational expectations under asymmetric information. givesalong-termprocedureforupdatingλthatguarantees: Review of Economic Studies, 48(4):541–559, 1981. [7] J.Kelly.Anewinterpretationofinformationrate.Bell • Youwon’tdotoomuchworsethanthemarket(which System Technical Journal, 35:917–926, 1956. by definition earns 0) [8] P. Milgrom and N. L. Stokey. Information, trade and • Youwon’tdotoomuchworsethanKellybettingusing common knowledge. Journal of Economic Theory, your original prior p 26(1):17–27, 1982. [9] P. A. Morris. An axiomatic approach to expert 2http://www.overcomingbias.com/2007/02/how_and_when_to.html resolution. Management Science, 29(1):24–32, 1983. [10] D. M. Pennock. Aggregating Probabilistic Beliefs: Market Mechanisms and Graphical Representations. PhD thesis, University of Michigan, 1999. [11] D. M. Pennock and M. P. Wellman. A market framework for pooling opinions. Technical Report 2001-081, NEC Research Institute, 2001. [12] E. Rosenblueth and M. Ordaz. Combination of expert opinions. Journal of Scientific and Industrial Research, 51:572–580, 1992. [13] M. Rubinstein. An aggregation theorem for securities markets. Journal of Financial Economics, 1(3):225–244, 1974. [14] M. Rubinstein. Securities market efficiency in an Arrow-Debreu economy. Americian Economic Review, 65(5):812–824, 1975. [15] M. Rubinstein. The strong case for the generalized logarithmic utility model as the premier model of financial markets. Journal of Finance, 31(2):551–571, 1976. [16] E. O. Thorp. Optimal gambling systems for favorable games. Review of the International Statistical Institute, 37:273–293, 1969. [17] E. O. Thorp. The Kelly criterion in blackjack, sports betting, and the stock market. In International Conference on Gambling and Risk Taking, Montreal, Canada, 1997. [18] J. Wolfers and E. Zitzewitz. Interpreting prediction market prices as probabilities. Technical Report 12200, NBER, 2006.