A Payment Scheme in Crowdsourcing

Transcription

1 A Paymen Scheme n Crowdsourcng Xao Chen 1, Kaq Xong 2 1 Deparmen of Compuer Scence, Texas Sae Unversy, San Marcos, TX Florda CyberSecury Cener and College of Ars and Scences, Unversy of Souh Florda, Tampa, FL 3362 Emal: xc1@xsae.edu, xongk@usf.edu Absrac Crowdsourcng coordnaes a large group of workers onlne o do self-conaned small asks ha are publshed by job requesers on a crowdsourcng plaform. Many papers propose ncenve sraeges o movae workers o parcpae n crowdsourcng. In hs paper, we shf he focus from he workers o he job requesers by addressng wo of her ssues: how o desgn a good paymen scheme o maxmze prof and how o selec qualfed workers o do he job. We use a wdely-adoped paymen formula conssng of a base salary and exra bonus. We frs formulae he problem as an opmzaon problem and hen provde a general soluon n whch we show ha he pay rae can be he same o all he workers. Nex we nsanae he soluon wh a concree example o derve more concree resuls and propose a worker selecon algorhm WS. In WS, we no only consder workers workload demands bu also her pas workng performance o guaranee crowdsourcng qualy. Smulaon resuls show ha a job requeser can pay much less o ge he job done n a crowdsourcng envronmen and our worker selecon algorhm s effcen n ha only searches a ny space o fnd he soluon o he opmzaon problem. Our effor here provdes an evdence o suppor he benefs of usng crowdsourcng n our daly lves. Index Terms crowdsourcng, opmzaon, pay rae, workload I. INTRODUCTION Crowdsourcng coordnaes a crowd (a large group of people onlne) o do self-conaned mcro-work (small asks) ha solves problems ha sofware or one user canno easly do. Busnesses use crowdsourcng o accomplsh her asks, fnd soluons o problems, or gaher nformaon. These nclude he ably o offload peak demand, access cheap labor and nformaon, generae beer resuls, access a wder array of alen han mgh be presen n one organzaon, and underake problems ha would have been oo dffcul o solve nernally [15]. Crowdsourcng s wdely used n vong, nformaon sharng, game, creave sysems [19], and moble crowd sensng (MCS) [7]. MCS s a new paradgm enabled by he wdespread avalably of smar phones equpped wh a rch se of bul-n sensors for collecng and sharng sensng daa from surroundng envronmen over a large geographcal regon. There are hree basc componens n crowdsourcng as shown n Fg. 1: requesers who publsh asks on a plaform, workers Fg. 1. Crowdsourcng componens who carry ou he asks, and a plaform such as Amazon Mechancal Turk [1] ha manages jobs. There are wo classes of crowdsourcng applcaons: parcpaory crowdsourcng [16], [18] where parcpans are acvely nvolved, and opporunsc crowdsourcng [5] where asks are done auomacally by moble devces wh mnmal worker nervenon. Malone e al. [14] defne wo varaons of worker conrbuon: collecon and collaboraon. A collecon occurs when dfferen members of a crowd conrbue ndependenly of each oher and a collaboraon occurs when here exs dependences beween he conrbuons of a se of crowd workers such as when hey work on dfferen pars of a problem or mprove a soluon eravely. In hs paper, we address he class of parcpaory crowdsourcng and assume he worker s conrbuon s ndependen. One mporan elemen o make crowdsourcng praccal s how o movae workers o conrbue o he asks, whch has been exensvely suded [6], [9], [17], [18], [2]. In hs paper, we shf gears o look a he problems on he sde of he requeser whch have no been dscussed as much o supplemen crowdsourcng research. We wll address wo ssues of he job requeser. One s how o desgn a good paymen scheme o maxmze prof and he oher s how o selec qualfed workers based on he paymen scheme. In our problem, we assume ha he workers are movaed by he money pad for her work so her ncenves are no a major ssue here. In our model, boh he requeser and workers are raonal. The requeser wans o desgn a good paymen scheme o maxmze hs prof and he workers wll choose he correspondng workload o maxmze her happness whch s defned as her ncome less he cos. Therefore, n our problem, he paymen scheme s he drvng force. Dfferen from he exsng crowdsourcng paymen schemes whch pay a ceran amoun of money for each ask [1], we use a paymen formula b+kx wdely used n our daly lves. I s composed of wo pars: a base salary b for base workload x and an exra paymen kx based on he pay rae k and exra workload x. Tha s, a worker wll ge an ncome of b f he fnshes he base workload x and an exra paymen of kx f he akes exra x asks. To help he requeser decde b and k, we formulae hs problem as an opmzaon problem on he requeser s sde o maxmze hs prof. And once he paymen scheme s decded, he workers wll reac accordngly o choose her opmal workload demand o maxmze her happness. Correspondngly, he requeser realzes hs and wll facor n he workers reacon o fnd ou he values for b and

2 k. To solve he opmzaon problem, we frs gve a general soluon leadng o he concluson ha he pay rae k can be he same o all he workers. Then we nsanae he general soluon wh a concree example so ha more concree resuls can be derved. Once he paymen scheme has been decded, a worker selecon algorhm can be desgned accordngly. So far, very few worker selecon algorhms for crowdsourcng have consdered he ssue of qualy conrol. Qualy conrol s a cenral ssue for crowdsourcng because wh he ncrease of workload and asks sze, he qualy of he work provded by workers rases many concerns as plaforms are connuously challenged by workers wh nsuffcen skllses o solve a gven problem [11]. In our proposed worker selecon algorhm named WS, we ake he workers pas workng performance no accoun o guaranee he qualy of he work o be done. The workers prevous workng sascs can be obaned from he crowdsourcng plaform as many crowdsourcng servces on he web requre workers o mee ceran requremens before hey can parcpae n a job [8]. For example, ses such as Amazon Mechancal Turk and GURU.com manan dealed sascs rackng he performance of he workers. The former evaluaes workers based on wheher her work was acceped or rejeced by job requesers [1] and he laer keeps he echncal skll, creavy, melness, and communcaon capables of a worker hrough a sar-based rang sysem [2]. The dfferences of our work from ohers and he key conrbuons of our work are as follows: We sudy he paymen scheme desgn and worker selecon algorhm for job requesers usng a paymen formula ha has no been dscussed n crowdsourcng. We formulae he paymen scheme desgn problem as an opmzaon problem and provde a general soluon o show ha he pay rae can be he same for all he workers. We nsanae he soluon framework wh a concree example and hen derve a specfc soluon and hereafer a worker selecon algorhm. We consder workers prevous workng sascs n worker selecon o guaranee crowdsourcng qualy. We conduc smulaons o show ha he job requeser can pay much less o ge he job done n a crowdsourcng envronmen and our worker selecon algorhm searches a ny space o fnd a soluon o he opmzaon problem. We provde an evdence hrough our heorecal analyss and smulaons o jusfy he benefs of usng crowdsourcng n our daly lves. The res of he paper s organzed as follows: Secon II references he relaed works. Secon III gves prelmnary nformaon. Secon IV defnes he problem. Secons V and VI presen a general soluon and a concree example, respecvely. Secon VII descrbes he smulaons we have conduced, and he concluson s n Secon VIII. II. RELATED WORKS Crowdsourcng covers a wde specrum. Here we survey he papers on ncenve mechansms n crowdsourcng. Many exsng ncenve algorhms use aucons o model rewards. The work n [16] proposes ProMoT, a Prof Maxmzng Truhful aucon mechansm for moble crowdsourcng sysem amng o provde sasfyng rewards o he smarphone users. Feng e al. propose TRAC [6] o smulae smarphone users o jon moble crowdsourcng wh a ruhful aucon mechansm ha akes locaon nformaon no consderaon. And he works presened n [17], [2] propose aucon mechansms for moble crowdsourcng, when a lmed budge s assgned for sensng asks and he plaform performs a subse of asks accordng o s budge consran. Some works provde moneary rewards o generally cooperave users [9], [1]. In [9], a subse of users are greedly seleced accordng o her locaons subjec o he coverage and budge consrans. In [1], he auhors desgn and evaluae a reverse aucon-based dynamc prce ncenve mechansm focusng on mnmzng and sablzng ncenve cos whle mananng adequae number of parcpans o conrbue o he ask. Dfferen from hese mul-wnner mechansms, Luo e al. n [12], [13] propose a wnner-ake-all mechansm, where a sngle bes or desgnaed user ges all he rewards. Chen e al. n [4] desgn ncenve schemes by consderng no only he exrnsc rewards such as money bu also he nrnsc rewards such as a sense of sasfacon, socal saus, or honor. Our work here s from a dsnc perspecve. We address he ssues of a job requeser o desgn a good paymen scheme usng a wdely-adoped paymen formula ha has no been dscussed before n crowdsourcng and hen selec qualfed workers o work on a job accordngly. III. PRELIMINARY We formulae he crowdsourcng opmzaon problem as follows: Suppose a job requeser has a job conanng W ndependen asks and he wans o crowdsource o N workers. There are wo pares n hs problem. On he job requeser s sde, he wans o maxmze he prof of he job and on he workers sde, hey wan o maxmze her happness. On he requeser s sde: Le us assume ha each ask of he job can brng n a revenue of P. Then he oal revenue from hs job s PW. The job requeser needs o pay he workers o do he job. He decdes o use he wdely-used paymen formula b +k x, n whch he subscrp n each varable denoes worker and he superscrp s he specfed me perod. The frs par b represens he base salary for worker a me perod f he fnshes he base workload x and he second par s he exra paymen wh a pay rae of k f he worker akes x exra asks over he base workload. In he paymen formula, all he hree varables b, k, and x are relaed o boh me and ndvdual worker. The frs wo varables b, k are conrolled by he requeser and he hrd varable x s chosen by worker. The oal cos of he job s he summaon of he paymens over me and ndvdual workers, whch s (b +k x ). Now, he goal of he job requeser s o maxmze he oal prof by solvng he followng maxmzaon problem for b and k.

3 maxmze b,k PW ( (b +k x )) (1) Here, PW s a consan. So he problem s equvalen o mnmzng he second par of he objecve, whch s mnmze (b b +k x ) (2),k On he worker s sde: For each ndvdual worker, hs happness s he ncome less he cos. Suppose worker akes a load of x + x, where x s he base workload o receve he base ncome b and x s he exra load o earn he exra paymen k x. Then he ncome of worker a me s b +k x. Obvously, x can be zero, meanng he worker does no wan o ake any exra load. The cos of a worker can be any expendure on he worker s sde o do he work. For example, can be he energy consumed by he worker s devce, or fague fel by he worker. The more workload he worker akes, he more he ncome bu also he more he cos. Thus, he cos funcon of worker s defned as α f (x ), where f s an ncreasng funcon manly relaed o he exra workload x worker s wllng o pck up and he added facor α makes he cos funcon o be dependen on me bu allows he shape off o say he same over me. Now for each worker, he wans o maxmze hs happness by maxmzng he followng hrough pckng he opmal amoun of exra work x. maxmze x b +k x α f (x ) (3) IV. PROBLEM FORMULATION In hs secon, we analyze he condons, make he requremens more explc, and formulae he problem. Le s frs look a he workers sde. For each worker, he wans o maxmze he expresson n (3). To worker, b and k are decded by he job requeser, and α s a known varable. So he only unknown varable s x. To maxmze he expresson n (3), we can dfferenae he expresson wh respec o x and se he resul equal o zero. Thus, we have k = α f (x ) So he opmal exra workload x o maxmze worker s happness s: x = f 1 (k/α ) (4) Also, each worker wans hs happness o be greaer or equal o zero. Thus, he expresson n (3) should be greaer or equal o zero. Furhermore, he oal number of asks ncludng he base workload and exra aken by all he workers a any me should be less or equal o he oal number of asks W. Now o pu he requeser s objecve and all of he consrans ogeher, we can formulae he opmzaon problem wh varables b, k, x as follows: mnmze b,k,x subjec o (b +k x ) x + x W, b +k x α f (x ),, x = f 1 (k/α ),, Here, he varables b and k are decded by he requeser and x s he reacon from worker o he paymen scheme. V. A GENERAL SOLUTION In hs secon, we gve a general soluon o problem (5). Frs, n problem (5), b + k x can be mnmzed f a worker s happness s drven o zero, ha s, b + k x α f (x ) =. In ha case, a worker does no feel very happy bu he s sll movaed o do he work because of ncome. Then b + k x = α f (x ), whch can be realzed by any combnaon of b and k ha can suppor an exra workload of x. Thus, b = α f (x ) k x (6) Nex, we denoe f 1 (k /α ) as d (k /α ), where funcon d represens he demand funcon of worker correspondng o pay rae k. From Equaon (4), we have x = f 1 (k /α ) = d (k /α ) (7) Now problem (5) s reduced o an opmzaon problem wh only one varable k : mnmze (α f (d (k/α ))) k subjec o d (k/α )+ (8) x W, Ths opmzaon problem can be deal wh usng he Lagrange dualy mehod [3]. We defne he LagranganL(k,λ) assocaed wh he problem as L(k,λ) = (α f (d (k/α )))+λ( d (k/α ) + x W) The parameer λ s he Lagrange mulpler assocaed wh he nequaly consran d (k /α ) W. We defne he Lagrange dual funcon g as he mnmum value of he Lagrangan over k : g(λ) = nfl(k k,λ) = nf( (α f (d (k/α ))) k + λ( d (k /α )+ x W)) The above funcon g(λ) s he nfmum of an affne funcon n erms of λ, so s concave. Accordng o he lower bound propery [3], f λ, hen g(λ) p, where p s he opmal value of he orgnal problem (8). Tha s, he Lagrange dual (5)

4 funcon ha depends on λ gves us a lower bound on he opmal value p of problem (8). Now we can work on he Lagrange dual problem ha gves he bes lower bound of p. Tha s, maxmze g(λ) λ (9) subjec o λ Ths s a convex opmzaon problem, snce he objecve g(λ) o be maxmzed s concave and he consran s convex. So o maxmze he objecve, we dfferenae g(λ) wh respec o λ and se he resul o zero. Thus, we have d (k/α )+ x = W (1) In Equaon (1), d s a demand funcon of worker a me correspondng o he k se by he job requser, and α s a known varable. For k, suffces for he job requeser o se he same o all he workers as long as he summaon of all he workload aken by he workers s equal o W. Thus, we can drop and make k = k (11) Now he pay rae k s only relaed o me and no each ndvdual worker any more. Afer k s se, b = α f (x ) k x from Equaon (6). Snce x = d (k /α ), b = α f (d (k /α )) k d (k /α ) (12) From Equaons (11) and (12), we can conclude ha he job requeser can use a unform pay rae k for all he workers f hey work exra hours, bu can use a dfferen base salary b for each ndvdual worker. VI. A CONCRETE EXAMPLE In hs secon, we nsanae he general soluon wh a concree example so as o beer explan he problem and derve he worker selecon algorhm for he job requeser. In hs example, we assume ha we look a he opmzaon problem durng a me perod n whch we can remove he superscrp from each varable. In oher words, b,k,x,α become b,k,x,α. From he above general soluon, we know ha k can be he same o all he workers. So k = k. We se a worker s cos funcon α f (x ) = α e x. Though a lnear funcon can also be assumed, we use an exponenal funcon o emphasze a worker s rapdly growng fague or expendure as he exra workloadx ncreases. The coeffcen α reflecs he dfferen cos (e.g. fague) level of each worker. Once he job requeser decdes he pay rae k, each worker can fnd hs opmal exra workload demand x o maxmze hs happness accordng o Equaon (7). In hs example, he demand funcon d s an ln funcon whch s he nverse of f. So x s x = f 1 (k/α ) = d (k/α ) = ln(k/α ) (13) We plug x no Equaon (6) and ge b as follows: b = α f (x ) kx = α e x kx = k kln(k/α ) (14) Now we plug b,x, and k no problem (2), he mnmzaon problem becomes: mnmze k (15) In hs problem, k s no relaed o me nor ndvdual worker. So f we focus on a perod of me, hen we can remove. Suppose we wan o recru m ou of a oal of N workers, hen problem (15) becomes: mnmze mk (16) A he same me, sll needs o sasfy he consran n Equaon (1). Pung hem ogeher, he opmzaon problem n hs concree case s smplfed o: Demand x mnmze m,k subjec o mk m ln(k/α )+mx = W (17) 7 ln(k) Now we use Fg. 2 o explan 6 ln(k/2) ln(k/3) he meanng of prob- 5 ln(k/4) lem (17). Suppose here 4 x 1 * 3 x 2 * are four workers havng a x 3 * 2 x 4 * cos funcon α e x and a 1 coeffcen α (1 4) as1,2,3,4, respecvely Pay rae k The base workload of each Fg. 2. Job demand x of dfferen worker o ge he base paymen workers s x. Accordng o Equaon (13), he exra workload demand of each worker s x = d (k/α ) = f 1 (k/α ) = ln(k/α ), whch s shown as a curve n Fg. 2. Now f he job requeser ses k o be 4, hen he correspondng exra opmal job demands of he workers are x 1,x 2,x 3,x 4, respecvely. If some workers are chosen for he job, hen her oal workload should add up o W. For example, f all of hese four workers are pcked, hen x 1 +x 2 +x 3 +x 4 +4x = W. When we selec workers, we wan o mnmze he mulplcaon of he number of workers m and he pay rae k. Nex we derve he worker selecon algorhm WS for he job requeser. Please noe ha he dea of WS s no lmed o hs example. I s applcable o oher parameer sengs as well. We frs come up wh algorhm named WSFK n Fg. 3 o selec workers when he pay rae k s fxed by he job requeser. The coeffcen α n he cos funcon can be obaned from he workers pas workng performance recorded n he crowdsourcng plaform. Algorhm WSFK frs calculaes a worker s opmal exra workload demand x, hen sors he workers accordng o he exra workload demands n a non-ncreasng order, and hen pcks he workers n hs order unl he oal workload s equal o he oal number of asks W. Before we come up wh a worker selecon algorhm whou fxng k, we need o fgure ou he lower and upper bounds of he pay rae k. Upperbound of k Assume α mn s he mnmum value among all he coeffcens α (1 N) n he workers cos funcons. Then

5 Algorhm WSFK: Worker Selecon wh a Fxed k Requre: A pay rae k decded by he job requeser, each worker s job cos funcon α f, and base workload x 1: calculae each worker s opmal exra workload demand x accordng o Equaon (13). 2: sor he workers accordng o her demand x n a nonncreasng order. 3: pck he workers one by one n hs order unl he oal of her workload equals W. Fg. 3. The worker selecon algorhm when k s decded Algorhm WS: Worker Selecon Algorhm Requre: A pay rae k decded by he job requeser and each worker s job cos funcon α f 1: k = k lower and (mk) mn = 2: whle k k upper do 3: call algorhm WSFK o selec m workers wh he curren k and calculae mk; 4: f he frs soluon o problem (17) s found hen 5: k upper = m k; 6: end f 7: f m k < (mk) mn hen 8: (mk) mn = m k 9: end f 1: k = k +ncremen 11: end whle Fg. 4. The worker selecon algorhm for a gven k(k > ), accordng o Equaon (13), he opmal job demand correspondng o α mn s he larges. Therefore,k reaches s maxmum value when ln(k/α mn ) = W. Solvng hs equaon, k upper = α mn e W. Lowerbound of k Agan assume α mn s he mnmum value among all α (1 N). Then for a gven k(k > ), he job demand correspondng o α mn s he larges. If we consder all he N workers, hen N ln(k/α mn ) W. Oherwse, he job wll be oo bg for all he workers o fnsh. Solvng hs nequaly, we ge he lowerbound of k o be k lower = α mn e W/N. Now we have a worker selecon algorhm WS shown n Fg. 4 o oban he mnmum mk wh he consran n problem (17) by searchng n he range [k lower,k upper ] of k. The algorhm sars from k = k lower o selec m workers ou of N wh he fxed curren k usng algorhm WSFK n Fg. 3. Then calculaes mk and compares wh he mnmum mk. If mk s less han he mnmum mk, hen he mnmum mk s updaed o mk. And hen he algorhm ncremens k by some amoun and repeas he process. The ncremened amoun can be decded by he job requeser. For example, some job requeser lkes o ncrease pay rae by $1./hour each me. Wh each fxed k [k lower,k upper ], we have a mulplcaon of mk. Afer he whole process s over, we ge he mnmum mk, he pay rae k, and he seleced workers. Tme Complexy of Algorhm WS The loop n algorhm WS has k upper k lower + 1 rounds. Wh each fxed k, we call algorhm WSFK o selec m workers. The mos meconsumng par n algorhm WSFK s o sor he N workers accordng o her work demands. If we use qucksor, he me complexy of sorng s O(N logn) on average. So he oal me complexy of algorhm WS s O(N logn(k upper k lower +1)) = O(N logn(α mn e W α mn e W/N +1)). VII. SIMULATION In hs secon, we presen he smulaons we have conduced o evaluae he performance of our proposed worker selecon algorhm WS. Snce our algorhm s one of a knd, we wroe a cusomzed smulaor n Malab o show s properes. In our expermens, we se boh he base workload x for each worker and he ncremen of k o 1. We randomly generaed each worker s α n he range of [1,1]. We ran algorhm WS 3 mes and averaged he generaed oupus. In he frs expermen, we waned o check he relaonshp beween he pay rae k and he oal workload W when he number of workers N s fxed. We se he number of workers N o 2 frs and hen o 5. We ncreased he oal workload W from2 o1. The resuls are shown n Fg. 5(a). From he resuls, we can see ha when he number of workers s fxed, wh he ncrease of he oal workload, he pay rae k should also be ncreased. Ths s because wh he fxed number of workers, he job requeser has o mprove pay rae o movae he workers o pck up more workload. We can also see ha when he number of workers s large (N = 5) relave o he oal workload, he pay rae k s small and ncreased slowly. And when he number of workers s small (N = 2) relave o he oal workload, he pay rae k s large and ncreased very quckly. Ths means ha f here are a lo of workers, benefs he job requeser because he can use a lower pay rae o ge he job done. On he oher hand, f here are no many workers, he job requeser has o pay a hgher ncenve. Ths proves ha crowdsourcng allows a job requeser o pay a low rae o recru parcpans from a large number of workers. In he second expermen, we nended o show he relaonshp beween he pay rae k and he number of workers N when he oal workload W s fxed. We se he oal workload W o 5 frs and hen o 1. We ncreased he number of workers N from 1 o 6. The resuls are shown n Fg. 5(b). From he resuls, we can see ha when he oal workload s fxed, wh he ncrease of he number of workers, he pay rae wll be decreased for he same reason ha more workers can make he pay rae lower. Also, when he workload s large relave o he number of workers (when W = 1 and N = 2), he pay rae s very hgh. Wh he ncrease of he number of workers, he pay rae s brough down quckly. When he workload s small relave o he number of workers (when W = 5), he pay rae s sable wh he ncrease of he number of workers. Ths means ha f here are enough workers for he job, he pay rae can be lower and f here are no enough workers, he requeser has o ncrease pay rae o movae he exsng workers. Agan, hs jusfes he use of crowdsourcng o ge a job done.

6 Pay rae k N=5 N= Toal workload W (a) Pay rae vs. oal workload Pay rae k W=1 W= Number of workers N 2 N=5 N= Toal workload W (b) Pay rae vs. numbers of workers (c) Workers sasfacon rae Fg. 5. Smulaon resuls In he hrd expermen, we amed o check he sasfacon rae of he workers. The sasfacon rae s calculaed as he rao of he number of seleced workers (m) versus he oal number of workers (N). We fxed he number of workers N o 2 frs and hen o 5. We ncreased he oal workload W from 2 o 1. The resuls of workers sasfacon are shown n Fg. 5(c). From he resuls, we can see ha when he number of workers s large (N = 5) relave o he sze of a job, he workers sasfacon rae s low and when he number of workers s small (N = 2) relave o he sze of a job, he workers sasfacon rae s hgh. Ths s because when he pool of he workers s small, mos of hem wll be chosen. So more workers wll ge pad and be sasfed. If he pool s large, hen only a few of hem wll be chosen. Thus, s beer for workers o parcpae n more crowdsourcng jobs o ncrease her chances. In he fourh expermen, we evaluaed he runnng me of our algorhm WS by checkng he sze of s searchng space. In WS, f he oal workload W ncreases, he upperbound of k wll be ncreased exponenally because k upper = α mn e W. Bu acually we do no search he full range of k lower and k upper. We sop he search long before ha. When he frs soluon o he consran m ln(k/α ) + mx = W n problem (17) s found, we can subsanally reduce k upper by updang o m k, where m and k are he number of workers seleced and he pay rae, respecvely n ha soluon. If here are more soluons when k > m k, he mulplcaon of k and m a ha me mus be greaer han m k. The search effcency followng hs dea s demonsraed n Fg. 5(d) where he percenage of he whole [k lower,k upper ] space searched s shown when W = 5 and N ncreases from 2 o 6. We can see ha WS only searches a ny fracon ( 1 21 ) of he whole searchng space o fnd he soluon. VIII. CONCLUSION In hs paper, we have addressed wo job requeser s ssues of how o desgn a good paymen scheme o maxmze prof and selec qualfed workers o do he job on an open crowdsourcng plaform. We have adoped a wdely-used paymen formula ha has no been dscussed n crowdsourcng o formulae an opmzaon problem. We have frs pu forward a general soluon and hen nsanaed o ge more concree resuls and he correspondng worker selecon algorhm WS. In WS, we have consdered he workers pas workng performance o guaranee crowdsourcng qualy. Smulaon resuls have shown ha a job requeser can pay much less o ge he job done n a crowdsourcng envronmen and our worker Workers sasfacon rae (%) The fracon of he space searched (x1-21 ) W= Number of workers N (d) Fracon of space searched selecon algorhm s effcen n ha only searches a ny space o fnd he soluon o he opmzaon problem. Our effor here has provded an evdence o suppor he benefs of usng crowdsourcng n our daly lves. In he fuure, we wll furher nvesgae workers characerscs or performance ndcaors usng daa mnng and machne learnng echnques o refne he worker selecon algorhm and hereafer provde more rgorous guaranee of crowdsourcng qualy. ACKNOWLEDGMENTS Ths research was suppored n par by NSF under CNS13532, ACI144637, and CNS165665, and NSF/BBN under CNS for projec REFERENCES [1] Amazon mechancal urk. hp://murk.com. [2] Freelancer nework. hp://guru.com. [3] S. P. Boyd and L. Vandenberghe. Convex Opmzaon. Cambrdge Unversy Press, 24. [4] Y. J. Chen, B. C. L, and Q. Zhang. Incenvzng crowdsourcng sysems wh nework effecs. In IEEE INFOCOM, 216. [5] Y. Chon, N. D. Lane, Y. Km, F. Zhao, and H. Cha. Undersandng he coverage and scalably of place-cenrc crowdsensng. In UbComp, pages 3 12, 213. [6] Z. Feng, Y. Zhu, Q. Zhang, L. M. N, and A. V. Vaslakos. TRAC: Truhful Aucon for Locaon-Aware Collaboraon Sensng n Moble Crowdsourcng. In IEEE INFOCOM, 214. [7] R. K. Gan, F. Ye, and H. Le. Moble crowdsensng: Curren sae and fuure challenges. IEEE Comm. Magazne, 49(11):32 39, 211. [8] J. Howe. The rse of crowdsourcng. Wred Magazne, 14(6), 26. [9] L. G. James, I. Vergara-Laurens, and M. A. Labrador. A locaon-based ncenve mechansm for parcpaory sensng sysems wh budge consrans. In IEEE PERCOM, 212. [1] J. S. Lee and B. Hoh. Sell your experences: a marke mechansm based ncenve for parcpaory sensng. In IEEE PERCOM, 29. [11] C. Lof and K. E. Maarry. Desgn Paerns for Hybrd Algorhmc- Crowdsourcng Workflows. In IEEE Busness Informacs, 214. [12] T. Luo, S. S. Kanhere, S. K. Das, and H.-P. Tan. Opmal przes for all-pay coness n heerogeneous crowdsourcng. In IEEE MASS, 214. [13] T. Luo, S. S. Kanhere, H.-P. Tan, F. Wu, and H. Wu. Crowdsourcng wh ullock coness: A new perspecve. In IEEE INFOCOM, 215. [14] T. W. Malone, R. Laubacher, and C. Dellarocas. Harnessng crowds: Mappng he genome of collecve nellgence. Techncal repor, MIT, 29. [15] B. S. Noveck. Wk Governmen: How Technology Can Make Governmen Beer, Democracy Sronger, and Czens More Powerful. Brookngs Insuon Press, 29. [16] H. Shah-Mansour and V. W. S. Wong. Prof maxmzaon n moble crowdsourcng: A ruhful aucon mechansm. In IEEE ICC, 215. [17] J. Sun and H. Ma. A behavor-based ncenve mechansm for crowd sensng wh bedge consrans. In IEEE ICC, 214. [18] D. J. Yang, G. L. Xue, X. Fang, and J. Tang. Crowdsourcng o smarphones: ncenve mechansm desgn for moble phone sensng. In ACM MOBICOM, 212. [19] M. Yuen, I. Kng, and K. Leung. A survey of crowdsourcng sysems. In IEEE PASSAT and IEEE SocalCom, 211. [2] D. Zhao, X.-Y. L, and H. Ma. How o crowdsource asks ruhfully whou sacrfcng uly: Onlne ncenve mechansms wh budge consran. In IEEE INFOCOM, 214.