Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile Devices

Transcription

1 Bayesphone: Precomputaton of Context-Senstve Polces for Inqury and Acton n Moble Devces Erc Horvtz 1, Paul Koch 1, Raman Sarn 1, Johnson Apacble 1, and Muru Subraman 1 1 Mcrosoft Research, One Mcrosoft Way Redmond, Washngton USA {Horvtz, Paulkoch, Ramans, Johnsona, Murus}@mcrosoft.com Abstract. Inference and decson makng wth probablstc user models may be nfeasble on portable devces such as cell phones. We hghlght the opportunty for storng and usng precomputed nferences about deal actons for future stuatons, based on offlne learnng and reasonng wth the user models. As a motvatng example, we focus on the use precomputaton of call-handlng polces for cell phones. The methods hnge on the learnng of Bayesan user models for predctng whether users wll attend meetngs on ther calendar and the cost of beng nterrupted by ncomng calls should a meetng be attended. 1 Introducton Over the last decade, there has been ncreasng research on the use of probablstc user modelng for nferrng user goals and states of the world [6,7] under uncertanty. The user models have been appled typcally n desktop settngs, where desgners can assume that a personal computer s avalable for performng nferences. We focus n ths paper on the precomputaton of deal decson-theoretc polces from probablstc user models and the cachng of the polces on a cell phone for decson makng n a moble settng. We beleve that such precomputaton and cachng of polces wll enable probablstc learnng and reasonng to be appled to the large and growng number of devces and applances n the world wth lmted computatonal abltes. We focus on the example of usng probablstc models to gude the handlng of telephone calls, so as to delberate about the cost of nterrupton versus the cost of deferral of an ncomng call. Such decsons can be made locally at cell phones, based on a consderaton of context and multple propertes of meetngs. We analyze the case of local decson makng based on sensed propertes of meetngs on a user s calendar and on propertes of callers based on caller dentfcaton, as well on real-tme sensng of moton and ongong conversaton. We frst dscuss the learnng of predctve models of attendance and of nterruptablty. We present the computaton of the expected cost of nterrupton from the output of

2 these models. Then we dscuss the computaton of value of nformaton to reason about the value of acqurng addtonal nformaton from users n real tme. We revew how we can precompute and cache polces on cell phones that consder whether calls should nterrupt users. We dscuss how the methods have been used to feld a prototype call handlng system that we call Bayesphone. 2 Learnng Models of Interruptablty and Attendance Efforts over the last several years have demonstrated that relatvely accurate models can be constructed for predctng the nterruptablty of users from such contextually relevant observatons as sensed actvty and calendar propertes [2,3,4,5,8]. We shall focus frst on the constructon of two Bayesan network models va supervsed machne learnng. One model predcts the nterruptablty of a user. More specfcally, we buld a model that s used to nfer a probablty dstrbuton over the cost of nterrupton of users. The second model outputs the probablty that users wll attend meetngs that appear on ther electronc calendar. Inferences from both models are used to predct the expected cost of nterrupton at dfferent tmes for a user. Addtonally, we show how we compute from the output of the models, the value of nformaton assocated wth askng users n real tme about ther stuaton. Such an analyss consders the nferences from the models as well as the frequency and types of calls comng n over tme. 2.1 Models of a User s Interruptablty We have been nvestgatng predctve models of the cost of nterrupton from evdence assocated wth a user s context, ncludng a stream of sensed data generated by a user s nteracton wth a desktop computer and propertes of tems on a user s electronc calendar [4,5]. Onlne calendars are central for coordnatng meetngs n many enterprses. For example, the Outlook calendarng subsystem s used unversally at our organzaton for extendng nvtatons to meetngs, montorng responses about planned attendance, and schedulng and trackng daly agendas. As part of the Coordnate proect, we constructed an appontment crawler and assessment tool that searches through users onlne calendars, as represented n the Mcrosoft Outlook messagng and calendarng applcaton [3]. The appontment crawler sfts through onlne appontments and records sets of propertes assocated wth each appontment. The appontment crawler notes, for each appontment, a set of propertes drawn from the Outlook applcaton, ncludng the tme of day and day of week of the meetng, meetng duraton, subect, locaton, organzer, the response status of the user (responded yes, responded as tentatve, dd not respond, or no response request was made), whether the meetng s recurrent or not, whether the tme s marked as busy or free on the user s calendar, whether the user was requred or optonal, the number of nvtees, the organzatonal relatonshps of the nvtees to the user, and the role of the user (user was the organzer versus a requred or optonal nvtee). The system accesses the Mcrosoft Actve Drectory servce to dentfy the organzer of the meetng and nvtees and notes whether the organzer and attendees are organzatonal peers, drect reports, managers, or managers of the user s manager.

3 The crawled data s used to buld an assessment vew that dsplays a form to users. The form conssts of a lst of ttles of meetngs and provdes felds for ndcatng the state of nterruptablty of users. Fg. 1 shows the assessment palette for assgnng a cost of nterrupton to each crawled calendar tem and a form used to defne the meanng of hgh, medum, and low cost of nterrupton states. Users use ths form to assgn scalar values to each state of nterruptablty. For ths assessment, we ask users to estmate the cost assocated wth a rngng phone durng states of hgh, medum, and low cost of nterrupton. To ground the semantcs of cost throughout the system, we consder the decson-analytc noton of wllngness to pay; and assess dollar values that users would be wllng to pay to avod a call n each settng. Gven the nterruptablty tags and appontment propertes, we buld a lbrary of cases, and then employ a Bayesan structure search procedure, based on methods developed by Chckerng, et al. [1], to buld a Bayesan network. The methods employ a greedy search across dfferent structures to dentfy the probablstc dependency structure that best explans the data, based on a score known as the Bayesan Informaton Crteron. The resultng Bayesan network can be used later to nfer probablty dstrbutons over the states of nterruptablty for prevously unseen meetngs, based upon a consderaton of a set of observatons consstng of the propertes of meetngs. Fg. 1. Cost of nterrupton assessment palette that enables users to vew a lst of pror appontments and to assess the cost of a phone call durng each meetng (lef. Overall dollarvalued costs are assgned to each state (rgh. Fg. 2 dsplays a Bayesan network model learned from a set of cases tagged by cost of nterrupton. The model can be used to nfer a probablty dstrbuton over states of nterruptablty, outputtng for each prevously unforeseen appontment the lkelhood that the meetng has a hgh, medum, or low cost of nterrupton. A study of a model constructed from the same 559 appontments and tested on 100 hold out cases showed a classfcaton accuracy of 0.81 for assgnng nterruptablty. 2.1 Models of Meetng Attendance Beyond models of the cost of nterrupton assocated wth a context assocated wth the attendance of a meetng, we also assess and learn n an analogous manner

4 Bayesan network models that predct the lkelhood that the meetngs wll be attended, based on meetng propertes. Fg. 3 shows a sample Bayesan network learned from tranng data for nferrng the lkelhood that a user would attend meetngs, based on meetng propertes. The model was traned wth the same appontments as were used to tran the model for the cost of nterrupton. In use, the personalzed attendance model generates, for prevously untagged meetngs, the lkelhood that users wll attend the meetngs. For ths model, a study of the accuracy on 100 cases held out for testng found that attendance was classfed at an accuracy of Fg. 2. Bayesan network learned from case lbrary that can be used to nfer the probablty dstrbuton over states of a varable representng the nterruptablty of a user, gven attendance of a meetng wth partcular propertes. The most nfluencng varables and ther probablstc dependences are hghlghted wth shadng. Fg. 3. Bayesan network learned from case lbrary that can be used to nfer whether a user wll attend a meetng or not, based on meetng propertes. The most nfluencng varables and ther probablstc dependences are hghlghted wth shadng.

5 3 Computng Expected Cost of Interrupton We can employ the Bayesan networks for predctng attendance and the cost of nterrupton to compute the expected cost of nterrupton (ECI) assocated wth calls that rng through to users who are attendng dfferent knds of meetngs. To perform the computaton of expected cost of nterrupton, we consder the probablty dstrbuton over the cost assocated wth the meetng at hand, as provded by the nterruptablty model, and the lkelhood that a user wll attend the meetng ndcated on the user s calendar, as provded by the attendance model. To compute the ECI of nterruptng a user when a meetng on a user s calendar s recognzed as beng n progress, we need one addtonal pece of nformaton the default cost assocated wth recevng a phone call when a user does not attend a meetng ndcated on a user s calendar. Such a default cost s typcally a functon of the tme of day, as recevng a call durng the early hours of the mornng or very late at nght s lkely to be dfferent than recevng a call durng busness hours, and the cost of nterrupton may also be dependent on the day of week. To assess default costs of nterrupton, we allow users to sweep over default hgh, medum, and low cost regons wthn a seven day by twenty-four hour tme palette, and to defne default costs of nterrupton to each value. Fg. 4 dsplays the palette for assessng the default of nterrupton by tme. Fg. 4. Tme palette for assessng costs of nterrupton by tme va a sweepng out of regons of tme, and assessng default costs for non-meetng tmes assgned hgh, medum, and low costs. Gven (1) an nferred probablty dstrbuton over the nterruptablty of a meetng, (2) the lkelhood that a user wll attend a meetng on ther calendar, and (3) the default cost assocated wth the no-meetng stuaton, the ECI at any moment s computed by weghtng the cost of nterrupton for the no-attendance and attendance stuatons n accordance wth the lkelhoods of these states. Takng the expectaton, the ECI s, b ECI = p( A E) p( c E) c + (1 p( A E)) c ( S) (1) where p(a E) s the lkelhood that users wll attend a meetng, gven evdental propertes E assocated wth the meetng, obtaned va Outlook appontment propertes, p(c E) s the probablty that users wll assgn a cost c to the meetng,

6 where ndexes the meetng as beng ether n low, medum, or hgh cost, and c b (S) s the background cost of beng nterrupted n the default stuaton S, representng the case where a user does not attend a meetng, as captured by the tme of day and day of week. The default cost can be extended to be dependent on multple aspects of a user s overall context S. Also, specal mutually-exclusve contexts can be consdered as actve n a prorty-order relatonshp. In the current Bayesphone prototype, the specal contexts of user drvng (stop-and-go versus smooth hghway drvng) and local conversaton n progress are sensed from a Bluetooth-based GPS system and headset, respectvely. If nether of these stuatons s sensed, the meetng and default day and tme context s consdered as actve. Otherwse the costs of nterrupton assessed for the specal contexts are assumed. 4 Performng Cost-Beneft Analyss n Real Tme We can balance a computed cost of nterrupton wth the cost of deferrng a conversaton untl later. A key pece of the decson s the cost of deferrng calls from dfferent callers. We thus obtan from users the dollar-value cost assocated wth delayed communcaton when a call s routed to vocemal rather than a real-tme conversaton. For such an assessment, we allow users to defne groups of callers, based on propertes of people, so as to provde a manageable set of classes. The deferral-cost assessment tool allows users to create groups of people based on sets of propertes of people ncludng organzatonal relatonshps and actvtes. The tool allows users to create such organzaton-related groups as peers, drect reports, manager, poston hgher-up n the organzatonal chart, person wthn organzaton, and people dentfed n a user s lst of contacts. Users can also pck actvty-based groups, so as to have ther devce recognze people who are scheduled n meetngs n the next hour, on the same day, or later n the same week. Another actvty-based group provded by the system s people I have called today and people I have called ths week. The tool can also be used to buld ad hoc groups lke crtcal assocates, and close frends. We employ Equaton 1 to precompute the expected cost of nterrupton based on meetng propertes for any tme durng the day. We shall return to the desktop and moble devce applcaton n Secton 5. Frst, we revew the precomputaton of value of nformaton for makng decsons about when to acqure addtonal nformaton from users. 4 Precomputng Ideal Interactons wth Users Beyond storng polces for makng the best decson based on nformaton that s currently avalable to a system, we have extended the basc cost-beneft analyss wth precomputaton about whether t s worthwhle for the phone to ask users at run tme to assst wth resolvng key uncertantes about the user s stuaton. More specfcally, we precompute the value of askng users for nformaton about whether they are attendng a meetng that appears on ther calendars. Answers to such queres can resolve key uncertantes used n the ECI computaton, potentally ncreasng the value of the call-handlng polces.

7 To dentfy deal queres, we compute the expected value of nformaton (EVI) of askng users a queston. EVI s a decson-theoretc measure of the value of gatherng addtonal nformaton that consders the current uncertantes, the lkelhood of dfferent answers to a query for more nformaton, and the ultmate nfluence of the dfferent answers on deal polces. For the case of Bayesphone, we precompute the value of askng a user f they ntend to attend a meetng before a meetng s scheduled to begn. The queston tself ncurs a cost of nterrupton that must be balanced wth the gans n value based n the new nformaton. To compute the value of askng the user about attendance, we must consder the ECI before and after askng, and cost of queryng the user. Gven an answer, the ECI wll be ether be the expected cost assocated wth the meetng (the frst term n Equaton 1), or the background cost of the tme of day (the second term Equaton 1). To compute the value of nformaton, we ntroduce the concept of the overall communcaton cost over a perod of tme. The expected communcaton cost (ECC) for a perod of tme s the cost of deferral and cost of nterrupton for all ncomng calls durng the perod. We wsh to nteract wth a user only f the reducton n ECC s greater than the cost of askng. Bayesphone precomputes the value of nformaton for all meetngs and uses ths nformaton to drve selectve queston askng. The ECC s computed by mantanng a log of ncomng calls. Bayesphone records a log of ncomng calls by group. Ths log s segmented nto calls that arrve at dfferent tme perods. For the current prototype, we consder eght perods: mornngs, afternoons, evenngs, and late nght for weekdays and weekends. For each perod, we compute the rates at whch calls assocated wth dfferent caller groups arrve each hour. Gven ths nformaton, we can compute the ECC for any value of the cost of nterrupton. We smply note the expected number of calls that wll be deferred and the calls that wll rng through to a user gven the computed ECI. The expected numbers of each class of calls are computed as a product of the stored rates for each caller group and the duraton of the perod. The ECC for a meetng of duraton t based on a consderaton of current evdence E only about propertes of the upcomng meetng s, defer rng ECC( E, = ( f c + f c ) t (2) where f s frequency of calls n each caller group that has a cost of deferral lower than the cost of nterrupton, f s the frequency of calls n each caller group that has a cost of deferral hgher than the cost of nterrupton, and c defer and c rng are the costs of deferral and cost of nterrupton of each of these caller classes, respectvely. We note that c rng s ust the current expected cost of nterrupton, ECI, as computed wth Equaton 1, so we can rewrte Equaton 2 as, defer ECC( E, = ( f c + f ECI ( E) ) t (3) To compute the EVI of askng the user a queston, we recompute ECI and ECC separately for the answers of attendng and not attendng, dentfyng the changes n the numbers of calls n the deferral and the rng-through classes for the updated values of ECI, and fnally combne these two ECC values together, weghted by the

8 probablty of hearng each answer. The communcaton cost for the answer, attendng meetng consders the expected cost assocated wth beng at the meetng, + defer ECC( E, a, = ' ' ( f c f p( c E) c t (4) k k ) k The communcaton cost for the answer, not attendng takes as the cost of nterrupton, the background cost assocated wth the tme of day, '' defer '' b ECC( E, a, = ( f c f c t (5) + ) Puttng these terms together, we can compute the expected value of askng the user as, EVI ( E, = ECC( E, [ p( A E) ECC( E, a, + (1 p( A E)) ECC( E, a, ] (6) a C where C a s the cost of askng the user before the meetng, ust the ECI before the meetng begns. The system also consders the added value of drectly askng a user about the nterruptablty of a meetng, gven that the user has answered that the meetng wll be attended, usng an analogous value of nformaton computaton. Users are asked to optonally answer a second queston about the cost of nterrupton, f acqurng that nformaton s worth the ncremental cost of askng the second queston. 5 Bayesphone Desktop and Moble Applcatons Bayesphone conssts of two applcatons: (1) a desktop applcaton, runnng on WndowsXP, that performs nference, cost-beneft analyses, and value of nformaton precomputaton of deal real-tme actons and nqures, and (2) an applcaton runnng on Smartphones that downloads the precomputed polcy fle from the desktop va a devce synchronzaton program. The Bayesphone desktop applcaton analyzes each forthcomng meetng, makng nferences wth the Bayesan network models for both attendance and nterruptablty. The clent applcaton consders these nferences along wth the costs of deferral of calls from callers n dfferent groups, the expected cost of nterrupton wth takng calls for each meetng, and the hstory of ncomng calls n the user s call log, and precomputes the deal call-handlng actons and nteractons for each meetng. The desktop system creates an XML-encoded fle, whch ncludes for each meetng, the meetng ttle, date, and tme, whether the user should be asked wth an alert about meetng attendance before the meetng, and the lst of caller groups who are allowed to breakthrough to the user durng the meetng for the no-nteracton or no-answer case, and for each answer. In use, a user may be asked before a meetng occurs about whether they plan to attend the forthcomng meetng. A specal alert tone s used to nform the user about the queston, and a screen appears that allows the user to specfy whether they wll attend the meetng. The maxmum lkelhood answer s dsplayed on the devce, allowng the user to ether confrm or to change the guess. If no answer s avalable wth three mnutes, the queston tmes out and the ttle of the meetng appears on the screen, along wth the groups who can breakthrough. Users can drectly change ther

9 attendance status or the cost of nterrupton drectly at any tme va a menu, and the deal precomputed polcy for the state wll be accessed and dsplayed. Fgure 6 dsplays two screens of the Bayesphone applcaton executng the call-handlng polces of one of the authors. In ths case, the system has alerted the user to the value of answerng a queston about attendance before a meetng. The system has guessed that the user wll not attend the meetng, and the user confrms ths guess. After the nteracton, the system shows the users the caller groups that wll be allowed to breakthrough. At run tme, Bayesphone ntercepts ncomng calls and takes control of the rngng of the phone. The applcaton checks caller ID, examnes the lst of callers allowed va the precomputed cost-beneft analyss, and decdes whether to rng the phone versus transfer the call to vocemal. Although the prmary ntent of ths paper s to share wth the User Modelng communty methods for precomputng user models for feldng deal polces on moble devces that do not have the computatonal power of desktop machnes, we are nterested also n the value of these methods for the call-handlng doman. The ntal Bayesphone prototype has been used by two people on our team for four months. We have not yet performed a formal valdaton of satsfacton wth call handlng, but the system has been reported by both users to perform well overall n a qualtatve survey. Both users provded us wth feedback on the effort wth settng up the system. The users found that the assessments of caller groups, costs of deferral, and costs of nterrupton were straghtforward, takng under 15 mnutes to complete. However, they found the assessment of the Bayesan models to be more burdensome, takng about two hours of tme to assess crawled events from ther onlne calendars. We are workng on means for easng ths burden va experence samplng along the lnes of [5], and on the use of lghter-weght, but less precse models. Such an approach ncludes the relance on the drect assessments of probablty dstrbutons for attendance and nterruptablty for classes of appontments. 5 Summary We have descrbed a proect hghlghtng the opportunty for precomputng nferences from Bayesan networks and couplng these nferences wth cost-beneft polces for feldng polces for acton and dalog wth users on smple end-pont devces lke cell phones. We revewed the constructon of probablstc models that can nfer the expected cost of nterrupton and the lkelhood that users wll attend meetngs on ther calendar. We showed how these models can drve a cost-beneft analyss of callhandlng polces and revewed a prototype applcaton. We are now studyng the dffcultes that users may have n buldng probablstc models for the prototype, and the overall experence wth usng the system. We are also workng to extend the evdental consderatons beyond meetng propertes and tme, to nclude such observatons as local sensng of locaton, moton, and ambent acoustcal sgnals, such as those representng a nearby conversaton n progress.

10 Fg. 5. Bayesphone applcaton, showng the case where t s best to ask the user about attendance of a forthcomng meetng. When the meetng starts, the applcaton dsplays the ttle of the meetng n progress, the nput from the user, and the callers who can break through. Movng beyond the motvatng example we selected to explore the precomputaton of personalzed polces, we are excted about the prospects for precomputng user models for feldng adaptve behavor that can be executed on a varety of small devces especally on moble devces that may have mnmal computatonal power. References 1. Chckerng, D.M., Heckerman, D. and Meek, C. (1997). A Bayesan approach to learnng Bayesan networks wth local structure. In Proc. of UAI 1997, pp Fogarty, J., Hudson, S.E., and La, J. (2004). Examnng the Robustness of Sensor-Based Statstcal Models of Human Interruptablty, Proc. of CHI Horvtz, E. Koch, P., Kade, C.M. Jacobs, A. (2002). Coordnate: Probablstc Forecastng of Presence and Avalablty. Proc. of UAI 2002, pp Horvtz, E. and Apacble, J. (2003) Learnng and Reasonng about Interrupton, Proc. of ICMI 2003, pp Horvtz, E. Apacble, J. and Koch, P. (2004) BusyBody: Creatng and Feldng Personalzed Models of the Cost of Interrupton, Proc. of CSCW Jameson, A. (1996) User Modelng and User-Adapted Interacton, Volume 5, pp Jameson, A. (2003). Adaptve Interfaces and Agents In J. Jacko and A. Sears (Eds.), Human-computer nteracton handbook (pp ). Erlbaum Publshers. 8. Mynatt, B. and Tullo, J. (2001). Inferrng Calendar Event Attendance, Proc. of Intellgent User Interfaces 2001, pp ACM Press.

11