Incentives in Crowdsourcing

Transcription

1 Incentives in Crowdsourcing Troy Kravitz 1 University of California, San Diego June 11, Joint with J. Aislinn Bohren, UCSD Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

2 Fixing Ideas A clothing manufacturer wants to create an online database tagging each individual attribute of its products A public website must review and remove objectionable postings A secretarial service needs to transcribe the recorded minutes from a recent company meeting Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

3 Fixing Ideas A clothing manufacturer wants to create an online database tagging each individual attribute of its products A public website must review and remove objectionable postings A secretarial service needs to transcribe the recorded minutes from a recent company meeting Think of tasks computers have difficulty handling but people do well Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

4 What is Crowdsourcing? The process of delegating work to an undefined group of people (a crowd) through an open call Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

5 What is Crowdsourcing? The process of delegating work to an undefined group of people (a crowd) through an open call Workers have no relationship with firm Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

6 What is Crowdsourcing? The process of delegating work to an undefined group of people (a crowd) through an open call Workers have no relationship with firm Crowdsourcing offers a cost-conscious, elastic workforce [... that can] complete massive volumes of simple tasks and eliminate the lead time and overhead associated with traditional hiring or outsourcing. Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

7 What is Crowdsourcing? The process of delegating work to an undefined group of people (a crowd) through an open call Workers have no relationship with firm Crowdsourcing offers a cost-conscious, elastic workforce [... that can] complete massive volumes of simple tasks and eliminate the lead time and overhead associated with traditional hiring or outsourcing. (We focus on) exclusively financially-motivated workers or work lacking non-pecuniary benefits Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

8 Crowdsourcing Marketplace Macro features (Frei, 2009) > 2,000,000 workers Gross payments $1b Vendor revenue $500m Demographics ( Behind Enemy Lines, 2010) 68 countries 45% US, 34% India Younger and more educated Wages (same) Day or less working on AMT: jobs 1 to $10 per task $20 or less per week, some over $1,000 per month Worker income profile (same) 20% of Indian workers report AMT as primary source of income (10% for Americans) 35% of Indians report as secondary source (60% for Americans) 2 Indian men to every Indian woman (reverse for Americans) 55% Indians have incomes < $10,000 (66% of Americans < $60,000) Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

9 Foundations Unknown state of the world a firm wants to discover Costly, noncontractible effort Unverifiable output Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

10 Foundations Unknown state of the world a firm wants to discover Costly, noncontractible effort Unverifiable output Traditional reputation mechanisms have no bite Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

11 Foundations Unknown state of the world a firm wants to discover Costly, noncontractible effort Unverifiable output Traditional reputation mechanisms have no bite Rational choice theory borne out (Mason and Watts, 2009) Understanding and expectation that output matters for earnings (same; Shaw, Horton and Chen, 2011) Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

12 Main Results Incentives generated by comparing message from multiple workers Characterize set of feasible contracts Determine optimal contract within feasible set Must employ the threat of a monitor Don t inform monitor Monitor disconfirming output more Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

13 Model Actors Arbitrarily large number of potential workers Single firm Actions, etc. Firm wants to hire agents to complete a job with J individual tasks Stage game for each task j: Unknown state ω j {0, 1} Prior π = P(ω j = 1) 1/2 Agent i chooses (unobservable) effort level ej i {0, 1} Exerting effort costs c and perfectly reveals state Agent chooses message m i j {0, 1} Firm observes agents messages and chooses action Aj {0, 1} Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

14 Model cont. Payoffs per task j Firm wants to choose action Aj that matches ω j and pays each agent i hired wage wj i: N = 1 A=ω w i (m j ) u F j i=1 Agents payoffs do not depend on the state u i j = w i (m j ) c 1 e i j =1 Principal-Agent setting: point-of-view of the firm Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

15 Comments Signal structure Collusion Limited liability Firm has commitment power Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

16 Implementation Characterize feasible and optimal implementation: Simultaneous Uninformed Informed (monitor) Sequential General incentive constraint: Agent i is willing to exert effort if: w i c Pr(paid e i = 1) Pr(paid e i = 0) In an equilibrium with agents exerting effort, the principal chooses the lowest wage that satisfies this constraint Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

17 Uninformed Simultaneous Implementation Given prior π, effort cost c, maximum job size J and monitoring probability q, optimal wage is w = c 1 [ 1 q+2qπ 1+q Payment conditional upon matching output Never hire more than two workers Optimal monitoring probability always interior Wage is: Decreasing in maximum job size J Decreasing and convex in monitoring probability q Increasing and convex as prior π becomes more pronounced ] J Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

18 Uninformed Simultaneous Implementation Given prior π, effort cost c, maximum job size J and monitoring probability q, optimal wage is w = c 1 [ 1 q+2qπ 1+q Payment conditional upon matching output Never hire more than two workers Optimal monitoring probability always interior Wage is: Decreasing in maximum job size J Decreasing and convex in monitoring probability q Increasing and convex as prior π becomes more pronounced ] J Virtual monitoring (Rahman, 2010): always feasible, optimal only for arbitrarily large job size Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

19 Informed Simultaneous Implementation (asymmetric) Two agents hired with probability q, but second agent is told he s the monitor ; asymmetric wages Optimal wages are: w 1 = c 1 [ 1 q + qπ ] J ; w 2 = c 1 π J Optimal monitoring probability is again interior Not informing the monitor of his role is better for the firm Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

20 Sequential Implementation Choose to hire workers sequentially depending on first worker s output Continuum of equilibria parameterized by (q 0, q 1 ) The optimal contract gives monitor least incentive to shirk Monitor action corresponding to less likely state with higher probability Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

21 Extending Model Nonbinary state and action spaces Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

22 Extending Model Nonbinary state and action spaces Heterogeneous effort costs Difficulties Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14

23 Extending Model Nonbinary state and action spaces Heterogeneous effort costs Difficulties Results Feasible contracts: continuum of cutoffs c s.t. all workers with cost below c exert effort Shirking occurs in equilibrium Wage need not be monotonic in the fraction of workers exerting effort in equilibrium Again, don t inform the monitor that he s the monitor Kravitz (UCSD) Crowdsourcing Incentives June 11, / 14