User Experiences on a Community-based Music Voting Service

Transcription

1 User Experiences on a Community-based Music Voting Service Timo Koskela, Janne Julkunen, Ville Keränen, Nonna Kostamo, and Mika Ylianttila Department of Electrical and Information Engineering University of Oulu Oulu, Finland firstname.lastname@ee.oulu.fi Abstract In this paper, we introduce and evaluate a community-based music voting service that relies on a DHTbased peer-to-peer network. Each community is created as a separate DHT overlay that is connected to a specific entertainment premise such as a cafeteria. A small-scale user testing was conducted in a laboratory environment to examine the attitudinal, social and perceived behavioral control factors associated with the community-based music voting service. Data was collected using questionnaires and short group interviews. The results indicate that the community-based services are seen attractive, and the service as such a very interesting and applicable idea, but the functionality would need to be enhanced. Keywords-user testing; peer-to-peer; music voting I. INTRODUCTION The emergence of new mobile services has grown fast, but not as quickly as it was predicted at the turn of the millennium [1][2]. On one hand, the development has been the delayed due to the lack of high-end mobile devices that hold the processing capabilities and fast wireless interfaces required for more versatile m-services [3][4]. With the introduction of 3G networks, the situation has however changed. For instance, in Finland, the share of 3G-capable mobile handsets has become a majority only in a few years time period due to the handset bundling [5]. On the second hand, mobile business has been seen as a very challenging area because users perceptions of the added value of the emerging m-services are hard to predict. This reflects the fact that the most of the m-services are totally new to the users [6]. In this paper, we present a mobile community-based music voting service (later referred as music voting service) and the results of a small-scale user evaluation. In short, the music voting service enables users to vote for music in location-based communities such as cafeterias and restaurants. The user evaluation was conducted in a fictitious cafeteria that was established in a laboratory environment. As a research model, a decomposed version of the Theory of Planned Behavior (TPB) was utilized [7][8][9]. TPB model focuses on the attitudinal, social and perceived behavioral control factors that were associated with the intentions to use the music voting service. The rest of the paper is organized as follows: Section II introduces the music voting service and Section III the research model along with the related hypotheses. Section IV presents the results of the user evaluation, and finally, Section V the discussion and some conclusions. II. THE MUSIC VOTING SERVICE A. Description of the Architecture The music voting service has been built upon a converged P2P and Web service architecture [10]. The community management and service discovery functionalities are provided by a P2P overlay network that relies on the Kademlia DHT algorithm [11]. Each community is created as a separate overlay that is then connected to a location-based community and the provided services. The actual end-user application is implemented as a web service. The communication between the web service and the P2P network takes place through a middleware component called P2P Daemon. More details about the P2P Daemon have been described in [10]. The P2P Daemon has a web-based configuration interface which provides means for managing the communities and services manually. For instance, the owner of a cafeteria could create a new community by entering the identifier or the name of the community and information regarding the semi-automatic community joining mechanisms. The community joining procedure can be conducted either with Bluetooth or RFID technologies. After the creation of the community, the configuration interface can also be used for publishing community-specific services only available to the members. Figure 1. Service Architecture. An external, but also a core component of the music voting service is a system called DJOnline that is a commercial virtual disc jockey system. In brief, DJOnline offers means for controlling the music played in the

2 entertainment premises. DJOnline also implements a SOAP interface that can be used to get information about the music played, playing and to be played in a particular entertainment premise. Furthermore, the information about the voting events is retrieved from DJOnline. The music voting service also utilizes the functionalities of a general voting service for registering the votes and for retrieving the results of the voting events. The architecture of the music voting service is illustrated in Fig. 1. B. Description of the Application The end-user application of the music voting service provides a playlist of the music played, playing and to be played in the user s current community. In the playlist, only the names of the songs are shown, but by selecting a song, a user can get more information such as the name of the artist, the genre of the music and the release year of that particular song. In addition, the actual voting events are also presented in the playlist. By selecting a voting event, the user is shown the voting options to choose from. After the user has provided her/his vote to one of the available songs, she/he can observe the progress of the voting event in real-time. The end-user application is mainly implemented with PHP and AJAX to achieve a desktop-like application that is interactive and easy to use. The user interface of the end-user application can be seen in Fig. 2. Figure 2. The End-user Application. DTPB examines more thoroughly the dimensions of the three main constructs. The defined belief structure in the DTPB is illustrated in Fig. 3. Figure 3. Decomposed Theory of Planned Behavior. B. Hypotheses and Statements Playfulness playfulness (PPF) stands for an individual s subjective experience concerning the human-computer interaction [12]. When dealing with an entertainmentoriented service, PPF has an important role in both encouraging the usage and developing an attitude towards the system [13]. The PPF has also been discovered a significant factor affecting the usage of WWW for entertainment purposes [14]. Thus, we expect that the following hypothesis will apply: III. RESEARCH MODEL AND HYPOTHESES H1: A user s perception of playfulness is positively associated with attitude. A. Theory of Planned Behavior TPB has been extended from the Theory of Reasoned Action (TRA) by adding the construct of perceived behavioral control as a determinant of behavioral intention [9]. The roots of TRA are in the social psychology, but both TRA and TPB have also been successfully applied in engineering to examine the individual s acceptance of technology. The other two constructs defined by TPB are attitude towards the behavior and the subjective norm. [7] More detailed descriptions about the three constructs are given in the subsection III.B. In this study, the Decomposed Theory of Planned Behavior (DTPB) is utilized. What distinguishes DTPB from TPB is the decomposition of attitude, subjective norm, and perceived behavioral control into an underlying belief structure within technology adoption contexts [7]. Thus, PPF1: It was fun to vote for music and to follow the progress of the voting results. PPF2: Voting for music has a positive impact on my enjoyment of the cafeteria. PPF3: While voting for music, I did not realize the elapse of time. PPF4: Because of the music voting, the cafeteria appeared a more interesting place to me. Usefulness In the literature, Usefulness (PU) has been defined in slightly different scales. According to the Technology Acceptance Model (TAM) [15], PU stands for the degree to which the system is seen to improve the job performance. This definition is rather limited and would exclude PU from our case, since the developed service is

3 clearly more entertainment- than professionally-oriented. However, also a broader view can be taken, where PU is seen as some value that the product or the service introduces to the individual user [16]. Based on the latter definition, we expect that following hypothesis will apply: H2: A user s perception of usefulness is positively associated with attitude. PU1: The music voting service is useful to me. PU2: It is important to me that I can affect the music played in a cafeteria. PU3: The music voting service would be a useful auxiliary service to the offerings of a cafeteria. PU4: The availability of the music voting service would positively influence my choice of a cafeteria. Image Image (IM) can be seen as the degree to which the usage of an innovative product is seen to contribute to an individual s image or status in one s social system [17]. Mobile device manufacturers have noticed the significance of image and have identified different user groups that value different aspects of the mobile devices [8]. We predict that the same trend also fits to mobile services, and thus expect that following hypothesis will apply: H3: A user s perception of image is positively associated with attitude. IM1: The music voting service is trendy. IM2: Usage of the music voting service would polish my image among other people. IM3: Usage of the music voting service would be considered positively in general. IM4: It would be considered youthful to use the music voting service. Price Level The price of the service could be seen as a sacrifice that is given to be able to take advantage of a particular service. It has been recognized that the price is clearly a subjective concept and very often described with qualitative measures instead of pure numbers. [18] Therefore, users individual perceptions on the relation of the perceived price level (PPL) and the predicted benefits greatly affect their attitude towards the service [13]. H4: A user s perceptions of perceived price level are negatively associated with attitude. PPL1: The expenses for the music voting service would be a barrier to me. PPL2: The benefits gained from the music voting service are not of monetary value. PPL3: The expenses of the music voting service should be included in the prices of other offerings of a cafeteria. Friend Influence When adopting a new innovation or a service, the beliefs of individual users are strongly influenced by their reference groups [19]. When related to adoption of Internet technology, these reference groups can be, for instance, peers, friends, supervisors and top management [8]. However, in the case of a mainly leisure-time entertainment service, only friend influence (FI) was seen relevant to the context. Thus, the following hypothesis is expected to apply: H5: Friend influence is positively associated with subjective norm. FI1: My friends would assume that I used this kind of music voting service. FI2: I would like to use the music voting service together with my friends. Cultural Influence Cultural influence (CI) takes friend influence to the next level, and is more concerned about the values and norms of the whole society. Cultural values do not only determine the behavior that is acceptable in particular situations, but also influence how we interact with other members of our society [20]. In addition, culture has been seen to have a strong influence on motivations, lifestyles, and even product choices [21]. Thus, we expect that following hypothesis will apply: H6: Cultural influence is positively associated with subjective norm. CI1: Using the music voting service in cafeterias would not be generally disturbing. CI2: The music voting service would fit well into the cafeterias. CI3: I think that people would spend their time with the music voting service in cafeterias. Internet Experience When interacting with a computer system for the very first time, people are inclined to feel embarrassed and stressed. After becoming more familiar with the system, interaction is likely to become easier and more spontaneous [22]. It has also been recognized that people take more positive stand towards systems that are compatible with the ones earlier adopted and used [23]. Since people are less reluctant to adopt new systems and services that utilize familiar the technologies, we expect that earlier experience on the Internet (IE) will have an important effect: H7: Internet experience is positively associated with perceived behavioral control. IE1: I think that I am an experienced user of Internet. IE2: I also use other Internet services than . IE3: I use Internet services such as Messenger, Skype, and IRC to keep in touch with my friends and family.

4 IE4: I use Internet community services such as Facebook, myspace, and IRC-galleria. Mobile Experience The theory presented in context of internet experience applies also to mobile experience (ME), but it has to be noted that familiarity in the fixed domain does not guarantee competence in the mobile domain. Thus, Internet and mobile experience needs to be examined separately. The two domains are distinguished by different screen sizes, input mechanisms and processing capabilities [24]. Therefore, mobile handsets and services may appear challenging to use for beginners. Thus, we expect that following hypothesis will apply: H8: Mobile experience is positively associated with perceived behavioral control. ME1: I think that I am an experienced user of small mobile devices such as PDAs, Communicator etc. ME2: I use Internet daily with my mobile phone. ME3: I use Internet with other mobile devices such as PDA, Communicator etc. Attitude, Subjective Norm and Behavioral Control Finally, the behavioral intention (BI) to adopt the music voting service is defined with three factors: attitude (ATT), subjective norm (SUN) and perceived behavioral control (PBC). ATT describes a person s feeling or perception towards the usage of the music voting service [9]. SUN describes the social influence that may have effect on the intentions to adopt the music voting service [8]. PBC, in turn, reflects the person s perception of the ease or difficulty to use the music voting service [9], and also whether the person holds the resources or skills needed to use the service. Thus, we expect that following hypotheses will apply: H9: Attitude is positively associated with behavioral intention. H10: Subjective norm is positively associated with behavioral intention. H11: behavioral control is positively associated with behavioral intention. ATT1: The music voting service was, in every way, a positive experience for me. ATT2: I followed the success of the song I had voted for with interest. SUN1: My friends would think positively of me when I use the music voting service. SUN2: I believe that the most of my best friends and my family members would use the music voting service. PBC1: It was not hard for me to use the music voting service with the given mobile device. PBC2: I think that I am an experienced user of mobile phones. BI1: I intend to use the music voting service if it becomes available. BI2: I intend to use the music voting service frequently if it becomes available. C. Collection of Data An online questionnaire was used to collect the respondents background information. The experiences gained were collected right after the user testing using a questionnaire, now with 5-point Likert scale ranging from Strongly disagree to Strongly agree. Additionally, a group interview was held in order to backup the questionnaire and to provide some additional development ideas. The group interviews were recorded and lettered afterwards. IV. RESULTS A. Testing Environment The small-scale user testing took place in November 2008, in the city of Oulu, Finland. The user testing was conducted in a fictitious cafeteria that was established in a laboratory environment. In addition to the tables and chairs, the room was equipped with a sound system, an LCD television and Nokia N810 Internet Tablets. The sound system was naturally used for playing the music the respondents could affect by voting, LCD television for showing the playlist in a larger public screen and the mobile devices for using the music voting service. The test users were divided into groups of 3 or 4 people and were given 25 minutes time period to spend in the fictitious cafeteria. Within that time period, they could vote for the music at least four times, since the songs played were shorter versions. B. Demographics There were 11 test users in total, 4 female and 7 male ones. The demographic profile in Table 1 indicates that the test users were mainly young Finnish students from the age group of (82%). TABLE I. Gender DEMOGRAPHIC PROFILE Frequency Percentage Male 7 63,6 Female 4 36,4 Age , , , ,1 Nationality Finnish 10 90,9 Other 1 9,1 Occupation Student ,0

5 Technology adoption Innovator 1 9,1 Early adopter 2 18,2 Early majority 3 27,3 Late majority 4 36,4 Late laggard 1 9,1 The technology adoption was estimated using two questions concerning the usage routines of mobile devices and services. The major part of the test users, according to our evaluation, belongs to the groups of early majority and late majority. The test users were not especially technologyoriented according to their occupation. C. Validity of the Research Model Before further analyzing the constructs of the research model, the internal consistency of the constructs was examined using the Cronbach s alpha coefficients. The Cronbach s alpha coefficients are used to test the stability of individual measurement items across replications from the same source of information [13]. It is recommended that the Cronbach s alpha coefficients should not be less than 0.80 [25]. The construct validity was also tested with a factor analysis. The construct validity indicates whether the constructs really measure what they intended to measure. In the exploratory factor analysis, an Eigen value of more than 1.0 was used as a determinant criterion. For each factor, the loading should exceed the value 0.60 [13]. Due to internal consistency and construct validity testing, some of the items were dropped. Testing results are summarized in Table 2. TABLE II. Constructs INTERNAL CONSISTENCY AND CONSTRUCT VALIDITY TESTING Loadings Cronbach alpha Items eliminated PPF PPF4 PU PU4 IM IM1, IM4 PPL PPL1 FI CI CI1 IE IE2, IE4 ME ME2 ATT SN PBC D. Hypotheses Testing In order to test the claimed hypotheses, we conducted multiple regression analyzes using SPSS The regression analysis was conducted by testing each item of each construct to clarify the overall effect of the construct in the research model. After defining the influence of each of the constructs, the achieved values of the standardized coefficient β, t-value, and the adjusted R-square were combined into Table 3. TABLE III. ANALYSIS OF THE HYPOTHESES Hypotheses β t-value H1: PPF to ATT Adjusted R- Square H2: PUF to ATT H3: IM to ATT H4: PPL to ATT H5: FI to SUN H6: CI to SUN H7: IE to PBC H8: ME to PBC H9: ATT to BI H10: SUN to BI H11: PBC to BI Hypotheses H1-H4 examine the constructs affecting ATT. According to the regression analysis, PPF (β = 0.540) and IM (β = 0.547) seem to have a clear influence on ATT. However, PUF (β = ) unexpectedly have a dominating negative effect on ATT. Thus, H2 do not apply. The effect of PC on ATT was negative as expected, but the t-value was so low that the effect could not be treated statistically significant. In addition, the group interviews revealed a basic assumption that the potential service fees for voting should be automatically included in the prices of other services provided, for instance, in a cafeteria. Hypotheses H5-H6 investigate the constructs affecting SUN. Both FI (β = 0.878) and CI (β = 0.041) have positive effect on SUN as predicted, however, FI clearly dominating. Again, the t-value of CI was too low to consider the effect of CI on SUN statistically relevant. In any case, it also appeared in the group interviews that the test users did not recognize or seem to care much about the cultural aspects. In turn, the importance of using the music voting service together with friends appeared several times in every group interview. Hypotheses H7-H8 study the links related to PBC. Again, both constructs IE (β = 0.135) and ME (β = 0.621) have a positive effect on PBC as expected. However, ME have clearly a dominating effect with much better statistical significance. The hypothesized relationships H9-H11 between BI and ATT, SUN as well as PBC bring forth some interesting aspects. SUN (β = 0.450) and PBC (β = 0.641) have an explicit positive effect on BI, but ATT (β = ), on the contrary, a negative one. Thus, H9 so not apply. ATT is clearly dominated by PUF that have a strong negative effect. Even though PPF and IM were considered important, the value of PUF also reflects the test user comments gained through the group interview. The general opinion was that the music voting service is a very interesting and applicable idea, but in its current composition, not useful enough. The

6 standardized coefficients in the research model are shown in Fig. 4. playfulness usefulness Image price level Friend influence Cultural context Internet experience Mobile experience Attitude Subjective norm behavioral control Behavioral Intention Figure 4. Research model with standardized coefficients. V. DISCUSSION AND CONCLUSION The aim of this paper was to present and evaluate a mobile community-based music voting service that enables users to vote for music in location-based communities such as cafeterias and restaurants. The music voting service has been built upon a converged P2P and Web service architecture. For the evaluation, the decomposed Theory of Planned Behavior was utilized. The findings from the small-scale user testing indicate that even though the playfulness and the image of an entertainment service are important, also the usefulness aspect needs to be considered. Thus, it is highly important to consider the actual value the user is provided with. It was also interesting to recognize that the test users had so clear opinion concerning the pricing of the service; the price should be included in the pricing of other services provided. We predict that this was the cause for the very low influence of the perceived price level on the attitude towards the service usage. The attractiveness of novel community-based services was clearly visible in the strong influence of friends on the usage intentions. However, cultural norms did not seem to have a significant influence on the test users. Finally, it appears that the people have become so familiar with the Internet that the adoption of novel web-based services is not seen as an obstacle. In addition, the web-based applications created with technologies such as AJAX bear similarity with the familiar desktop applications. However, it has to be noted that the median age of test users was 25 years, and the young people tend to be more accustomed with Internet technology. In any case, the experience with mobile devices still seems to have a significant effect on the usage intentions. As a conclusion, the mobile community-based music voting service was seen as an interesting and applicable idea, but the actual service needs further improvements. Since the number of test users was quite small (11) for quantitative analysis, the results should be considered preliminary providing guidelines for the future development of community-based multimedia services. For the future work, we plan to develop the service further and conduct a user testing with larger group of people in a real usage environment. ACKNOWLEDGMENT The financial support from the National Technology Agency of Finland (Tekes), Foundation of Technology (TES), Nokia Foundation, Foundation of Walter Alhström, and Research and Educational Foundation of TeliaSonera as well as the support of ITEA2 Expeshare project consortium are gratefully acknowledged. REFERENCES [1] H. Verkasalo, Empirical Observations on the Emergence of Mobile Multimedia Services and Applications in the U.S. and Europe, Proc. the 5th International Conference on Mobile and Ubiquitous Multimedia, Dec [2] B. Anckar, and D. D Incau, Value creation in mobile commerce: Findings from a consumer survey, Journal of Information Technology theory & application, vol. 4, no. 1, 2002, pp [3] T. Koivumäki, Consumer attitudes and mobile travel portal, Electronic Markets, vol. 12, no. 1, 2002, pp [4] X. Zhang, and V.R. Prybutok, How the mobile communication markets differ in China, the U.S., and Europe, Communications of the ACM, ACM, vol. 48, iss. 3, 2005, pp [5] H. T. Verkasalo, Handset-based measurement of smartphone service evolution in Finland, Journal of Targeting, Measurement and Analysis for Marketing, vol. 16, no. 1, 2007, pp [6] M. Kesti, A. Ristola, H. Karjaluoto, T. Koivumaki, Tracking consumer intentions to use mobile services: empirical evidence from field trial in Finland, E-Business Review, IV, 2004, pp [7] V. Venkatesh, M.G. Morris, G.B. Davis, and F.D. Davis, User Acceptance of Information Technology: Toward a Unified View, MIS Quarterly, ABI/INFORM, vol. 27, no. 3, 2003, pp [8] T.S.H Teo, and S.H. Pok, Adoption of WAP-enabled mobile phone among Internet users, Omega, Elsevier, vol. 31, no. 6, 2003, pp [9] M.K. Chang, Predicting Unethical Behavior: A Comparison of the Theory of Reasoned Action and the Theory of Planned Behavior, Journal of Business Ethics, Kluwer Academic Publishers, vol. 17, 1998, pp [10] T. Koskela, J. Julkunen, J. Korhonen, M. Liu & M. Ylianttila, Leveraging Collaboration of Peer-to-Peer and Web Services, Proc. the 2nd International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, Sept.-Oct [11] T. Koskela, O. Kassinen, J. Korhonen, Z. Ou & M. Ylianttila, Peerto-Peer Community Management using Structured Overlay Networks, Proc. the International Conference on Mobile Technology, Applications and Systems, Sept [12] J. Moon, and Y. Kim, Extending the TAM for World-Wide-Web context, Information and Management, vol. 38, no. 4, 2001, pp

7 [13] J. H. Cheong, and M.-C. Park, Mobile internet acceptance in Korea, Internet Research, vol. 15, no.2, 2005, pp [14] M.A. Atkinson, and C. Kydd, Individual characteristics associated with world wide web use: an empirical study of playfulness and motivation, Database for Advances in Information Systems, vol. 28, no. 2, pp [15] F. Davis, usefulness, perceived ease of use, and user acceptance of information technology, MIS Quarterly, vol. 13, no. 3, pp [16] J. Nielsen, Usability Engineering. Academic Press, 1993, 358 p. [17] E.M. Rogers, Diffusion of innovations, 3rd edition. The Free Press, 1983, 453 p. [18] V.A. Zeithaml, Consumer perceptions of price, quality and value: a means-end model and synthesis evidence, Journal of Marketing, vol. 52, 1988, p [19] R.L. Thompson, C.A. Higgins, and J.M. Howell, Influence of experience on personal computer utilization: testing a conceptual model, Journal of MIS, vol. 11, no. 1, 1994, pp [20] N. Singh, and H. Matuso, Measuring Cultural Adaption on the Web: A Content Analytical Study of U.S and Japanese Web Sites, Journal of Business Research, vol. 57, iss. 8, 2004, pp [21] D.K. Tse, R.W. Belk, and N. Zhou, Becoming a consumer society: a longitudinal and cross-cultural content analysis of print ads from Hong Kong, The People's republic of China, and Taiwan, Journal of Consumer Research. vol. 15 iss. 4, Mar. 1989, pp [22] G. Hackbarth, V. Grover, and M.Y. Yi, Computer playfulness and anxiety: positive and negative mediators of the system experience effect on perceived ease of use, Information and Management, vol. 40, no. 3, 2003, pp [23] J. Dearing, and G. Meyer, An exploration tool for predicting adoption decisions, Science Communications, vol. 16, no. 1, 1994, pp [24] R. Bridle, and E. McCreath, Inducing shortcuts on a mobile phone interface, Proc. the International Conference on Intelligent User Interfaces, Jan. 2006, pp [25] D. Straub Validating instruments in MIS research, MIS Quarterly, vol. 13, 1989, pp