Splitting Approaches for Context-Aware Recommendation: An Empirical Study

Transcription

1 Splitting Approaches for Context-Aware Recommendation: An Empirical Study Yong Zheng, Robin Burke, Bamshad Mobasher ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014

2 Contents ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014 Context-aware Recommender Systems Context-aware Splitting Approaches Empirical Study & Evaluation Results Discussions, Conclusions & Future work 2

3 Contents ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014 Context-aware Recommender Systems Context-aware Splitting Approaches Empirical Study & Evaluation Results Discussions, Conclusions & Future work 3

4 Recommender Systems Recommender Systems (RS) Two-dimension rating space: Users Items Ratings M1 M2 M3 U1 U2 U3 U4? 4

5 Context-aware Recommender Systems Context-aware Recommender Systems (CARS) Multi-dimensional space: Users Items Contexts Ratings 5

6 Context-aware Recommender Systems Context-aware Recommender Systems (CARS) Assumptions and Viewpoints in CARS: Users preferences or decisions usually differ from contexts to contexts, even towards the same item. E.g. buy a gift for someone. It s better to infer user s preferences by rating profiles within the same or similar contexts. E.g. look at music others choose within same contexts Context is defined as any information that can be used to characterize the situation of an entity by Dey, Anind K. (2001).However, the actual contexts in CARS and the contextual effects are domain specific. Movie domain: time, location, companion, mood, etc Music domain: time, activity, mood, etc Travel domain: season, weather, companion or trip type, etc 6

7 Context-aware Recommender Systems Context-aware Recommender Systems (CARS) Example of CARS applications: Tour Plan Recommender Yu, Chien-Chih and Chang, Hsiao-ping, "Towards Context-Aware Recommendation for Personalized Mobile Travel Planning". International Conference on Context-Aware Systems and Applications,

8 Context-aware Recommender Systems How to incorporate contexts into RS? There are two methods to categorize those incorporations. 1).In terms of how contexts interacted with the RS algorithms 8

9 Context-aware Recommender Systems How to incorporate contexts into RS? There are two methods to categorize those incorporations. 2).In terms of whether new CARS algorithms required to be developed It can be simply categorized into: a).transformation Algorithms A transformation is required, then all traditional RS algorithms can be applied to. Do NOT need to develop new CARS algorithms. such as Dimensions as Virtual Items (DaVI) and context-aware splitting approaches (CASA). b).adaptation Algorithms CARS algorithms are required, traditional algs can be modified. Such as context-aware matrix factorization (CAMF). 9

10 Contents ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014 Context-aware Recommender Systems Context-aware Splitting Approaches Empirical Study & Evaluation Results Discussions, Conclusions & Future work 10

11 Context-aware Splitting Approaches (CASA) Context-aware Splitting Approaches (CASA) In terms of the two categorizations (i.e. how to incorporate contexts into recommender systems) above, CASA belongs to pre-filtering and transformation algorithms. There are three context-aware splitting approaches: 1). Item splitting by L. Baltrunas, F. Ricci, ACM RecSys, ). User splitting by A. Said et al, RecSys, ). UI splitting by Y. Zheng et al, RecSys, 2013 User splitting and UI splitting are two approaches derived from Item splitting, examined and evaluated by different authors. 11

12 Context-aware Splitting Approaches (CASA) Item Splitting The underlying idea is that the nature of an item, from the user's point of view, may change in different contextual conditions, hence it may be useful to consider it as two different items. (L. Baltrunas, F. Ricci, RecSys'09) In short, contexts are dependent with items. At Cinema At Home At Swimming Pool Any dependent patterns involved in those ratings? 12

13 Context-aware Splitting Approaches (CASA) Item Splitting -- Example User Item Location Rating U1 M1 Pool 5 U2 M1 Pool 5 U3 M1 Pool 5 U1 M1 Home 2 U4 M1 Home 3 U2 M1 Home 2 High Rating Significant difference? Let s split it!!! Low Rating Same movie, different IDs. M11: being seen at Pool M12: being seen at Home M1 13

14 Context-aware Splitting Approaches (CASA) Item Splitting Step 1. Choose a contextual condition to split each item; The selection process is done by measuring significance of rating differences (such as the two-sample t test); Step 2. Contexts are fused to items and removed from original multidimensional matrix. We get a 2D rating matrix, then traditional algorithms like CF, MF can be applied to; How to select an appropriate contextual conditions for splitting? a). Binary contextual condition b). Impurity criteria and significance test See example in the next. 14

15 Context-aware Splitting Approaches (CASA) Item Splitting Binary Contextual Condition User Item Loc Rating User Item Rating U1 M1 Pool 5 U2 M1 Pool 5 U3 M1 Pool 5 U1 M1 Home 2 U4 M1 Home 3 U2 M1 Cinema 2 Transformation U1 M11 5 U2 M11 5 U3 M11 5 U1 M12 2 U4 M12 3 U2 M12 2 A binary contextual condition for splitting: Pool vs. Non-Pool If there is qualified split, one item will be split to two new ones. Why use a binary condition? To alleviate or avoid cold-start problems! 15

16 Context-aware Splitting Approaches (CASA) Item Splitting Impurity Criteria There could be several binary context conditions, for example, Pool vs Non-Pool, Home vs Non-Home, Weekend vs Non-Weekend. Impurity criteria and significance test are used to make the selection. There are 4 impurity criteria for splitting by L. Baltrunas, et al, RecSys'09; tmean (t-test), tprop (z-test), tchi (chi-square test), tig (Information gain) Take tmean for example, tmean, is defined using the two-sample t test and computes how significantly different are the means of the rating in the two rating subsets, when the split c (c is a context condition, e.g. location = Pool) is used. The bigger the t value of the test is, the more likely the difference of the means in the two partitions is significant (at 95% confidence value). Choose the largest one! 16

17 Context-aware Splitting Approaches (CASA) User Splitting and UI Splitting Similarly, the splitting approach can be applied to user too! User Splitting: is a similar one. Instead of splitting items, it may be useful to consider one user as two different users, if user demonstrates significantly different preferences across contexts. (A. Said et al., CARS@RecSys 2011) In short, contexts are dependent with items. UI Splitting: simply a combination of item splitting and user splitting both approaches are applied to create a new rating matrix new users and new items are created in the rating matrix. (Y. Zheng, et al, Decisions@ACM RecSys 2013). In short, it fuses dependent contexts to users and items simultaneously at the same time. 17

18 Context-aware Splitting Approaches (CASA) An Example of Three CASA After transformation: Item Splitting: User + NewItem; User Splitting: NewUser + Item; UI Splitting: NewUser + NewItem; UI Splitting fuses contexts to both users and items, where it may enlarge the contextual effects, but it also increases sparsity. It is hard to say whether UI splitting will outperform the other two algs or not. It varies from data to data. 18

19 Contents ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014 Context-aware Recommender Systems Context-aware Splitting Approaches Empirical Study & Evaluation Results Discussions, Conclusions & Future work 19

20 Empirical Study and Evaluations Experimental Goals a> Comparison Among Three Splitting Approaches Which one performs the best? Which splitting criteria is the best appropriate one? Any underlying patterns to indicate which one should be used? b> Comparison Between CASA and other Contextual Algorithms Which one performs the best? How about CASA competing with other CARS algorithms? 20

21 Empirical Study and Evaluations Data Sets Contextual variables in the three survey data sets: Food data: degree of hungriness in real and supposed situations; Movie data: Location (home/cinema), Time (weekend/weekday), Companion (family, etc); LDOS-CoMoDa: Location, Time, Companion, Weather, Emotions, Seasons, etc We use a 5-fold cross validation for all data sets and examined algorithms. 21

22 Empirical Study and Evaluations Baseline Algorithms We choose two other context-aware algorithms as the baselines: 1). Differential Context Modeling (DCM) by Y. Zheng, et al, 2012 There are two approaches falling into this category: Differential Context Relaxation (DCR) Differential Context Weighting (DCW) Basic idea: Using rating profiles with same or similar contexts for rating predictions; Take user-based collaborative filtering for example: Segment alg to various components; Apply context filter to each component; Filters could be different, and not necessary to be the same. Filter could be realized by context relaxation to find same contexts, or context weighting to find similar contexts. Generally, DCW works better than DCR. 22

23 Empirical Study and Evaluations Baseline Algorithms We choose two other context-aware algorithms as the baselines: 2). Context-aware Matrix Factorization (CAMF) by L. Baltrunas, et al, 2011 There are three approaches falling into this category: CAMF_C, CAMF_CI, CAMF_CU CAMF_C: Assume contextual effect is associated with each contextual condition only. CAMF_CI: Assume contextual effect is associated with item-context interactions. CAMF_CU: Assume contextual effect is associated with user-context interactions. CAMF is a kind of contextual modeling approach, where context-aware splitting approaches are contextual pre-filtering approaches. Both of them take advantage of the dependency between contexts and users or items. 23

24 Empirical Study and Evaluations Context-aware Splitting Approaches (CASA) They are pre-filtering approaches. Any traditional recommendation algorithms can be applied to, after the original multi-dimensional rating matrix was transformed to a 2D rating matrix. We evaluate three CASA based on the configuration as follows: 1). Evaluated by different traditional RS algorithms User-based Collaborative Filtering (UBCF), Item-based Collaborative Filtering (IBCF) Traditional Matrix Factorization techniques (MF) without taking contexts into consideration Implemented and evaluated by open-source Toolkit MyMediaLite v3.07 CF algorithms were tuned up by varying # of neighbors; MF algorithms were examined by varying # of factors and training iterations; 2). Evaluated by different impurity criteria in splitting processes tmean (t-test), tprop (z-test), tchi (chi-square test), tig (Information gain) 24

25 Empirical Study and Evaluations Evaluation Metrics We choose three metrics: RMSE, Precision, ROC. RMSE is used to evaluate the accuracy of predicted ratings. Prediction error, is the most popular and common used metric in CARS area, since context-aware data are usually sparse and few users rated a same item for several times. ROC = a visualization between recall and FPR by varying # of N in Top-N recommendations. = x axis is FPT, y axis is Recall In measuring Precision and ROC, we use a rating threshold to judge relevance. For Movie data, the threshold is set as 7, and it is set as 3 for the other two data sets. 25

26 Empirical Study and Evaluations Evaluation Metrics Traditional way to measure Precision and ROC: 1).We have training and testing set; 2).Train a model based on the training set and evaluate it on the testing set; 3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated>; 4).For evaluation purpose, provide a list of ranked Top-N items to each user; 5).And examine the hit ratio between the Top-N list and the list of items rated in the testing; However, in CARS, contexts should be taken into account; CPrecision and CROC curve 1).We have training and testing set; 2).Train a model based on the training set and evaluate it on the testing set; 3).In the testing set, aggregate rating profiles by <user, a list of items he/she rated, contexts>; 4).For evaluation purpose, provide a list of ranked Top-N items to each <user, contexts>; 5).And examine the hit ratio between the Top-N list and the list of items rated in the testing; NOTICE: <user, a list of items he/she rated, contexts>; the list is pretty short and even just one item, because users seldom rated items for several times within different contexts. Thus the value of CPrecision and CROC will be much smaller than traditional ones. 26

27 Empirical Study and Evaluations Evaluation Challenge in CASA (Optional Part) User Item Loc Rating User Item Rating U1 M1 Pool 5 U2 M1 Pool 5 U3 M1 Pool 5 U1 M1 Home 2 U4 M1 Home 3 U2 M1 Cinema 2 Transformation U1 M11 5 U2 M11 5 U3 M11 5 U1 M12 2 U4 M12 3 U2 M12 2 RMSE can be directly evaluated based on the transformed rating matrix in CASA It is because the number of rating profiles in data is NOT changed. CPrecision and CROC cannot be directly evaluated on the transformed rating matrix 1). # of users and # of items could be DIFFERENT 2). It is not comparable to other CARS algorithms Solution: We only use transformed matrix to predict ratings, but evaluate IR metrics on the original multi-dimensional rating matrix. 27

28 Experimental Results Experimental Results (in RMSE) Goal-1: Comparisons among the three context-aware splitting approaches (in RMSE) Q: Which one performs the best? The best impurity criteria? A: UI Splitting using MF as the recommendation algorithm. MF works better than CFs. The best choice varies from data to data. No consistent patterns. Q: Any other patterns? A: For Movie data, item splitting is better than user splitting; But user splitting is better than item splitting for the other two ones, where they have emotional or feeling contextual variables, we assume those contexts are more dependent with users. 28

29 Experimental Results Experimental Results (in RMSE) Goal-2: Comparisons with other CARS algorithms (in terms of RMSE) Q: Which one performs the best? The best impurity criteria? A: UI Splitting using MF as the recommendation algorithm in terms of RMSE. Q: Any other patterns? A: If item splitting is better than user splitting, CAMF_CI is better than CAMF_CU; If user splitting is better than item splitting, then CAMF_CU is better than CAMF_CI; It is because both of them take advantage of context-dependency patterns!! 29

30 Experimental Results Experimental Results (CPrecision & CROC) Goal-1: Comparisons among the three context-aware splitting approaches In CPrecision, UI splitting > Item splitting > User Splitting; In ROC Curve, UI splitting > User splitting > Item Splitting; Goal-2: Comparisons with other CARS algorithms In CPrecision, UI splitting > CAMF_CI > CAMF_CU > DCW > DCR; In ROC Curve, UI splitting > CAMF_CU > CAMF_CI > DCW > DCR; Patterns: UI Splitting is the best in RMSE and IR metrics for LDOS-CoMoDa; Consistent findings in context-dependency pattern in EACH METRIC; In RMSE, context is more dependent with user; 30

31 Experimental Results Experimental Results (CPrecision & CROC) Goal-1: Comparisons among the three context-aware splitting approaches In CPrecision, Item splitting > UI splitting > User splitting; In ROC Curve, same patter as above; Goal-2: Comparisons with other CARS algorithms In CPrecision, Item splitting > UI splitting > CAMF_CI > CAMF_CU > DCW; In ROC Curve, Item splitting > UI splitting > CAMF_CI > DCW > CAMF_CU; Patterns: Item Splitting is the best in RMSE and IR metrics for Movie data; Consistent findings in context-dependency pattern in EACH METRIC; 31

32 Experimental Results Experimental Results (CPrecision & CROC) Goal-1: Comparisons among the three context-aware splitting approaches In CPrecision, UI splitting > User splitting > Item Splitting; In ROC Curve, same pattern as above; Goal-2: Comparisons with other CARS algorithms In CPrecision, UI splitting > CAMF_CU > CAMF_CI > DCR > DCW; In ROC Curve, DCR > UI splitting > DCW > CAMF_CU > CAMF_CI Patterns: Overall, UI Splitting is the best in RMSE and IR metrics for Food Data; Consistent findings in context-dependency pattern; 32

33 Contents ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014 Context-aware Recommender Systems Context-aware Splitting Approaches Empirical Study & Evaluation Results Discussions, Conclusions & Future work 33

34 Conclusions Conclusions & Future Work Which one performs the best? Generally speaking, UI splitting is the best; In Movie data, UI splitting is the best on RMSE, but item splitting is the best on IR metrics; If context is not that dependent with users, merging effects by UI splitting may decrease the joint effect on recommendations. Any patterns or guidelines to choose which context-aware algorithms? In terms of choices between item splitting & user splitting, and CAMF_CI & CAMF_CU, it totally depends on which one contexts are more dependent to, user or item? Whether UI splitting performs the best depends on three factors: 1). The dependency between contexts and users and items; 2). The sparsity after rating matrix transformation cold-start problems in CASA; 3). The performance difference between user splitting and item splitting. If one of them performs bad, it is not guaranteed that the joint effect UI splitting will perform better; Future work: 1).how to judge contexts are more dependent with users or items? Any numeric metrics to validate it? PS: Impurity values? no consistent patterns. 2). How to alleviate the cold-start problems in UI splitting. 34

35 Conclusions References Item Splitting 1) L. Baltrunas, and F. Ricci. "Context-based splitting of item ratings in collaborative filtering." ACM RecSys, ) L. Baltrunas, and F. Ricci. "Experimental evaluation of context-dependent collaborative filtering using item splitting." User Modeling and User-Adapted Interaction (2013): User Splitting 1) L. Batrunas and X. Amatriain."Towards Time-Dependent Recommendation Based on Implicit Feedback." ) A. Said, E. Luca, S. Albayrak. "Inferring contextual user profiles improving recommender performance UI Splitting 1) Y. Zheng, R. Burke, B. Mobasher. "The Role of Emotions in Context-aware Recommendation" ) Y. Zheng, R. Burke, B. Mobasher, Splitting Approaches for Context-Aware Recommendation: An Empirical Study, ACM SAC, 2014 Context-aware Matrix Factorization 1) L. Baltrunas, B. Ludwig, F. Ricci. "Matrix factorization techniques for context aware recommendation." ACM RecSys Differential Context Modeling 1) Y. Zheng, R. Burke, B. Mobasher. "Differential Context Relaxation for Context-aware Travel Recommendation". EC-WEB, ) Y. Zheng, R. Burke, B. Mobasher. "Optimal Feature Selection for Context-Aware Recommendation using Differential Relaxation" ) Y. Zheng, R. Burke, B. Mobasher. "Recommendation with Differential Context Weighting". In UMAP,

36 ACM SIGAPP the 29th Symposium On Applied Computing Gyeongju, South Korea, March 26, 2014