Commodity Price Prediction using An Artificial Prediction Market based Approach

Transcription

1 Commodity Price Prediction using An Artificial Prediction Market based Approach Rohith D. Vallam Ramasuri Narayanam Gyana R. Parija July 18 th, 2017

2 Agenda Prediction Markets: Introduction Problem Definition Artificial Prediction Market: A Vanilla Version Results & Comparison Next Steps 2 India Research Lab 2016 IBM Corporation

3 How do people predict? Opinion Polls Delphi methods Peer Prediction Methods Wagering mechanisms Prediction Markets (Our Focus) Source of images : 3 India Research Lab 2016 IBM Corporation

4 What is a Prediction Market? Tool for collecting and aggregating opinion using market principles Price: probability of event occurring Value: leading indicator, expose hidden information Pay-off: monetary, reputational, indirect Iowa Electronic Markets: 2008 US Democratic Convention Market Clinton Obama Accuracy: better than conventional forecasting 1 Image: Defintion: A place where information is aggregated via market (or other) mechanisms for the primary purpose of forecasting events, or the probability that an event will occur 1 Source: Arrow, K.J. et al The Promise of Prediction Markets Science, 2008, 320, 5878, IBM Corporation

5 Prediction Markets: Scientific Background 2017 IBM Corporation

6 When to use Prediction Markets? Complexity (ecosystem) Uncertainty Many decision points Clear outcomes Market liquidity Diversity of opinion 2017 IBM Corporation

7 Advantages of Prediction Markets with Other Approaches of Information Aggregation 2017 IBM Corporation

8 Prediction Markets At Work: Consumer Prediction Markets Hollywood Stock Exchange Viral Loop (Prediction market mobile app) Intrade Prediction Market 2017 IBM Corporation

9 Prediction Markets At Work: Consumer Prediction Markets (Cont.) LongBets Inkling and many more.. New kid on the block: Blockchain Prediction Markets (Wikipedia) The Augur project seeks to leverage the open, global, peer-to-peer ledger functionality that blockchain technology provides, as well as game theory and financial incentives, to better explore the concept of the wisdom of crowds (also known as collective intelligence) and try to get more accurate predictions about future events IBM Corporation

10 Prediction Markets At Work: Corporate Prediction Markets PMs in use at US organizations (July 2010) Ex: HP, BestBuy, Electronic Arts, Boeing, Amazon, Harvard, GM, Hallmark, P&G, Ford, Microsoft, Chevron, Lockheed Martin, CNN, Adobe, American Express, Bosch Applications Project management, risk management Revenue forecasting, demand planning, capital budgeting Idea management (rate, filter, prioritize ideas) 2017 IBM Corporation

11 Agenda Prediction Markets: Introduction Problem Definition Artificial Prediction Market: A Vanilla Version Results & Comparison Next Steps 11 India Research Lab 2016 IBM Corporation

12 Price Prediction for Raw Materials Raw Material & Key Information which Experts seek from High Impact Factors Feedstock Types Feedstock Suppliers Raw Material Supply (Countries / Regions / Company) Raw Material Manufacturing (Plants, Process, Capacity) Port of Origin / Destination Port Raw Material Orders Raw Material Inventory Mfg. End Product (Products, Demand, Regions) Raw Material Prices Knowledge Search & Results Expert 1 Knowledge Type 1 Expert 2 Knowledge Type 2 Expert 3 Knowledge Type 3 Artificial Prediction Market based Approach (to predict the price for raw materials) 12 India Research Lab 2016 IBM Corporation

13 Prediction Markets: High Level Outline Expert 1 s Prediction Price: 1560 Confidence Level: 5 Expert 2 s Prediction Price: 1620 Confidence Level: 7... Expert n s Prediction Price: 1650 Confidence Level: 9 Prediction Market based System Output: Market Prediction of Raw Material 13 India Research Lab 2016 IBM Corporation

14 Agenda Prediction Markets: Introduction Problem Definition Artificial Prediction Market Results & Comparison Next Steps 14 India Research Lab 2016 IBM Corporation

15 ARCHITECTURAL DIAGRAM of ARTIFICIAL PREDICTION MARKETS (for Price Prediction) Market Participants Raw Material PRICE DATA SOURCE Agent_IRL_ML Agent_IRL_Feed Stock_ML Betting Strategy (Q Learning) Betting Strategy (Q Learning) β 1 Budget β2 Budget (Prediction 2, bet 2 ) Prediction Market Market Price Prediction (from market price equations) Agent_IRL_EM Betting Strategy (Q Learning) β3 Budget Budget Updation for all agents β 1 <- β 1 bet 1 + revenue 1 β 2 <- β 2 bet 2 + revenue IBM Corporation β 3 <- β 3 bet 3 + revenue 3

16 Timeline of Artificial Prediction Market (with 2 Agents) Player 1 bets 20$ on the prediction Player 2 bets 56$ on the prediction Market Maker reveals current market prediction Prediction market closes. Ground Truth is revealed.. Week 1 starts Week 2 starts Player 1 places a revised bet 12$ on the prediction after observing the market prediction Player 2 places revised bet 35$ on the prediction based on market prediction Players are rewarded based on their predictions and the realized outcome. Note: 1. We can run the above market using the Data whose time duration: 1-May-15 to 31-Mar-17 (weekly data) 2015 IBM Corporation

17 Artificial Prediction Market Idea to Raw Material Price Prediction Algorithm sketch based on the paper on Continuous Artificial Prediction Market (c-apm). Details of reference given below: Online Prediction via Continuous Artificial Prediction Markets - IEEE Intelligent Systems (2017) - Fatemeh Jahedpari, Talal Rahwan, Sattar Hashemi, Tomasz Michalak, Marina De Vos, Julian Padget, Wei Lee Woon IBM Corporation

18 Agenda Prediction Markets: Introduction Problem Definition Artificial Prediction Market Results & Comparison Next Steps

19 Preliminary Results (using the Proposed Artificial Prediction Market) Artificial Prediction Market Dynamic Opinion Formation Model Metrics For Entire Data (05-Jun-15 to 30-Jun-17) RMSE Score = MAPE Score = Metrics For Q (Apr-Jun 2017) RMSE Score = MAPE Score = Metrics For Q (Jan-Mar 2017) RMSE Score = MAPE Score = Metrics For Q (Oct-Dec 2016) RMSE Score = MAPE Score = Metrics For Entire Data (05-Jun-15 to 30-Jun-17) RMSE Score = MAPE Score = Metrics For Q (Apr-Jun 2017) RMSE Score = MAPE Score = Metrics For Q (Jan-Mar 2017) RMSE Score = MAPE Score = Metrics For Q (Oct-Dec 2016) RMSE Score = MAPE Score = IBM Corporation

20 Preliminary Results (using Vanilla Version of Artificial Prediction Market) Maximum Absolute Percentage Deviation of the proposed Artificial Prediction Markets based approach below 4% for the last 3 Quarters IBM Corporation

21 Agenda Prediction Markets: Introduction Problem Definition Artificial Prediction Market Results & Comparison Next Steps

22 Next Steps Engineering with ``Parameter Configurations to improve the performance of ``Vanilla Version of Artificial Prediction Market Explore advanced mathematical constructs to prediction performance Work with different ``proper scoring rules to determine payments to the agents Work with different strategies for the ``market maker to define the market prediction Work with different learning algorithms for the agents to improve their own predictions after observing the market prediction Design an ``artificial expert / agent who can observe other agents predictions and then predict most accurate price of raw material: Based on ``deep reinforcement learning paradigms Based on algorithms in online learning especially ``prediction with expert advice literature Design of Artificial Prediction Markets to derive predictions in the form of Probability Distribution for a given task IBM Corporation

23 References Online Prediction via Continuous Artificial Prediction Markets - IEEE Intelligent Systems (2017) - Fatemeh Jahedpari, Talal Rahwan, Sattar Hashemi, Tomasz Michalak, Marina De Vos, Julian Padget, Wei Lee Woon. An Introduction to Artificial Prediction Markets for Classification Adrian Barbu and Nathan Lay Journal of Machine Learning Research (2012) Artificial Prediction Markets for Online Prediction of Continuous Variables-A Preliminary Report - Fatemeh Jahedpari et.al. (2015) Simulating Prediction Markets that Include Human and Automated Agents - Wendy Chang (Masters thesis, MIT 2009) Betting and Belief: Prediction Markets and Attribution of Climate Change - John J. Nay et. al (2016) 2015 IBM Corporation

24 2015 IBM Corporation Thank You