User Behavior Recovery via Hidden Markov Models Analysis

Transcription

1 User Behavior Recovery via Hidden Markov Models Analysis Alina Maor, Doron Shaked Hewlett Packard Labs HPE Keyword(s): Hidden Markov Model; predictive analytics; classification; statistical methods; clickstream analysis Abstract: In this report, we propose a method for user-behavior profiling and user-intention prediction based on Hidden Markov Models. User behavior is described by a sequence of user-actions. Note that there is no strict correspondence assumption between action order and user purpose or business process. On the contrary, we assume actions characterizing a purpose or a business process will have order variations or be interleaved with actions emerging from different purposes. The statistical nature of user behavior and the partial order assumption fit the statistical Hidden Markov Model. Different models are trained for different user profiles or to predict predetermined behavior types. Prediction can be incorporated with other predictors in a maximum likelihood setting. The proposed technique is generic and can be applied to profiling of any given number of (predefined) profiles. The only assumption is that profiles are characterized by different dynamics. The method was successfully applied in a twogroup scenario to actions on HP.COM for predicting eventual purchase. The motivation being that the preferred type of offers (similar products vs. accessories) depend on whether a shopper is just window shopping or in a purchase process. External Posting Date: August 2, 2016 [Fulltext] Internal Posting Date: August 2, 2016 [Fulltext] Copyright 2016 Hewlett Packard Enterprise Development LP

2 User Behavior Recovery via Hidden Markov Models Analysis Alina Maor and Doron Shaked Software & Analytics Lab Hewlett Packard Labs July 20, 2016 Abstract In this report, we propose a method for user-behavior profiling and user-intention prediction based on Hidden Markov Models. User behavior is described by a sequence of user-actions. Note that there is no strict correspondence assumption between action order and user purpose or business process. On the contrary, we assume actions characterizing a purpose or a business process will have order variations or be interleaved with actions emerging from different purposes. The statistical nature of user behavior and the partial order assumption fit the statistical Hidden Markov Model. Different models are trained for different user profiles or to predict predetermined behavior types. Prediction can be incorporated with other predictors in a maximum likelihood setting. The proposed technique is generic and can be applied to profiling of any given number of (predefined) profiles. The only assumption is that profiles are characterized by different dynamics. The method was successfully applied in a two-group scenario to actions on HP.COM for predicting eventual purchase. The motivation being that the preferred type of offers (similar products vs. accessories) depend on whether a shopper is just window shopping or in a purchase process. Problem statement One of the most intriguing problems in multiuser applications analysis is timely profiling (classification) of application users. By user profiling we refer to the conceptual purpose or business process performed by users. In on-line shopping, examples of such profiles can be windowshopping, price comparison and purchase. In IT management 1

3 applications, examples of such profiles can be system initialization, code-update, commit and so on. The business implication of such profiling is enormous, providing: 1) application metering and monitoring via changes in users behavior; 2) application design efficiency; and 3) user-customized application experience. Henceforth, we concentrate on one specific example, namely profiling window shoppers and buyers in on-line shopping, focusing on HP.COM, but it should be clear that the proposed approach is applicable to other multi-user applications. User behavior is represented as a click-stream. The core problem our model solves is that user-actions are weakly unordered cluttered and interleaved: e.g., a purchase confirmation can be interrupted by additional browsing, pop-ups or sporadic actions such as an effort to contact the site administrator. As the first step to user-behavior analysis, we wish to classify users into two groups - potential buyers and window-shoppers, based on as little number of clicks as possible. This task is highly non-trivial due to the above mentioned complex nature of users-clicks. In addition the HP.COM site has additional complications including 1) HP.COM has a complex internal structure and URLs have functional duplicity such that identical user operations of different users often result in completely different click-streams. 2)A 30-minutes session termination rule. Our solution We base our solution on the long-known mechanism of Hidden Markov Models (HMMs), see [1] for details. HMM is a statistical model that assumes that there is a process generating observed outcomes (outputs). In our case, the outcomes are the user clicks (URLs). The process is governed by unknown unobserved, i.e., hidden, states. For on-line shopping, the states may refer to the implicit process states, e.g., looking for a particular product, comparing, or completing purchase. These states are defined by another hidden parameter - the user is either a potential buyer or a window-shopper. In our case, this hidden parameter is the profile type we want to uncover. Assume that a system has N hidden states and K possible outputs. By HMM, all hidden states are connected with a zero-memory Markov chain. When a system is in a certain state, there is a probability of generating every possible system output. Each output is dependent only on the current system-state and not on previous/future states or outputs. HMM definition includes also the initial probabilities of starting the system in each of the hidden states. For each user profile we train a separate HMM. For statistic validity, training and test data-sets are of the same size. The choice of N is a 2

4 difficult problem. Methods we developed based on randomization and divergence convergence indicated that N = 6 will be a good choice in this case. The convergence of HMM parameters is performed via the Baum-Welch method [1]. We regularize and normalize the model to account for rare outputs in the training sets outputs. Note that since K > 2500, rare outputs are quit common. Having trained two HMMs one for buying and one for non-buying, sequences are matched to both HMMs resulting in two likelihood values for buying or non-buying intentions. We perform two tests. The first scheme compares the likelihood values of and deciding whether the user is a buyer or not according to the higher likelihood. While this scheme is simple, it compares well against the Word-Based Algorithm (WBA). WBA is based on choosing a descriptive key-word(s) describing the desired user action and declaring a certain user behavior based on presence/absence of that key-word(s). For purchase analysis, two natural choices for such key-words are action-sets related to Cart, or Check-Out. The HMM approach always out-performs for short-medium sessions, and often for long sessions. We propose a novel algorithm, combining the advantages of both the WBA and the HMMbased approaches: Since both groups of users may perform the same sets of actions, the required correction to the HMM is the compensation for this noise. The accommodation for the noise will be both in offset and scaling - broadly speaking, buyer is declared when: LL buy αll not buy + β1 W BA + γ 0, (1) where LL is the logarithm of the likelihood, α is the LL scale, 1 W BA is the indicator function of word existence with weight β and γ is the regularization constant. Specifically, α emphasizes the difference in decisions performed by HMM and should not be too large, otherwise statistic validity of the decision will be violated. The value of γ provides constant offset compensating for noise in LL values due to high dimensionality. Finally, β is the tradeoff in decision made by HMM Vs WBA: The larger the value of β, the lesser the impact of HMM. Setting β, we degenerate to WBA. By taking β and γ to 0 and setting α = 1, we degenerate to the decision based solely on HMM. We omit further discussion on possible variation of the decision rule due to the abstract-length limitation. Evidence the solution works Data-Set: The HP.COM data-set contains a sample of session collected over a 6 month period and includes and sessions that were tagged as buyers/non-buyers, respectively, by an HP.COM coordinator, according to at least one purchase performed during the 3

5 last 2 months of the period. The number of sessions containing at least 5 clicks is buyers and 8559 non-buyers. For statistic validity, we create train-sets by randomization of 7000 buyer and non-buyer sessions out of the pool and use sessions for tests. The dimensionality of large sites is huge - in the HP.COM sample there are different URLs. Classified to K = types according to functionality (not removing functional duplicity). Limiting K to outputs appearing in the data sets more than twice reduced K to 4922 for the buyers model and 2671 for the non-buyers model. Some outputs are frequent and statistically valid while others are rare. We could omit these rare outputs, but we kept all. Starting with the first approach, we train two HMMs, and pause to address a difference in obtained behavior states: The HMM of buyers contains a clear purchasing state and other targeted browsing activities, while in case of non-buyers, purchasing activities are interleaved in almost all states with varying probability of occurrence. We proceed as following: 1) For every click in new sessions run the clicks through both HMMs and obtain both likelihood values. 2) Perform tests with these likelihood values: The simplest scheme compares the likelihoods and decides buyer/non-buyer by the higher value. The obtained results are generally better than WBA schemes as illustrated in Fig. 1. The x-axis is the number of clicks performed by user and the y-axis is accuracy, or the percentage of correct decisions. The four graphs represent four schemes: Magenta for the random-decision (RD), Red for the above HMM based decision, and Green and Blue for WBA scheme: users invoking Cart (Green) or Check-Out (Blue) are henceforth classified as buyers. As one can see, the accuracy of HMM is much higher than that of WBA and RD. Notice that the key advantage in users profiling comes from the earlier clicks. Next, we enrich the HMM method and the WBA schemes with the hybrid combination of both. The results of randomizing α, β and γ are demonstrated in Fig. 2 for decisions made at clicks 15 and 40: The x-axis and the y-axis are the recall and the precision percentage, respectively and the black graph is the break-even line. As we can see, for different values of α, β and γ the red-lines (matching different randomization values of α, β and γ) are closer to the upper rightmost corner of the plot, indicating the superiority of our hybrid combination. Competitive approaches There were numerous attempts to tackle the problem of online browsing modeling - see [2] and references therein. The main differences between the works that we are aware of and our setting is the sessions sub-division, unavailability of temporal data, and the decision making 4

6 algorithm (operating HMM per click) which is proposed herein. Current status and next steps We have developed a new tool for business intelligence analysis for the HP.COM analytic group. With slight changes, we also tested our approach on HP-SW Quality Center application focusing on profiling conceptual users-flow and flow-steps, rather than per-user categorization. We believe the HMM based tool is generically applicable to many applications and is suitable for different types of analysis. The attractive features of this solution is its implementation simplicity. The scheme can be plug-and-played in any system whose dynamics intrinsically assume hidden operational states. Specifically, HMMs-based analysis can be used for flow monitoring, system inspection and classification and we hope to integrate our scheme into an analytic platform. References [1] L.R. Rabiner, A Tutorial On Hidden Markov Models And Selected Applications In Speech Recognition, Proc. of the IEEE, vol. 77, no. 2, pp , Feb [2] A. L. Montgomery, S. Li, K. Srinivasan and J. C. Liechty, Modeling Online Browsing and Path Analysis Using Clickstream Data, Marketing Science, vol. 23, no. 4, pp , Fall

7 Figure 1: Accuracy of prediction of buying/non-buying intentions. Figure 2: Recall-Precision for non-buying intentions as function of Alpha, Beta and Gamma. 6