Online Shopping Recommendation with Bayesian Probabilistic Matrix Factorization

. Recommendation system plays a crucial role in demand prediction, arousing attention from industry, business, government and academia. Widely employed in recommendation system, matrix factorization can well capture the potential relationships between users, items and latent variables. In this paper, we focus on a specific recommendation task on the large scale opinion-sharing online dataset called Epinions. We carried out recommendation experiments with the Bayesian probabilistic matrix factorization algorithm and the final results showed the superior performance in comparison to six representative recommendation algorithms. Meanwhile, the Bayesian probabilistic matrix factorization was investigated in depth and the potential advantage was explained from the model flexibility in parameters’ adjustment. The findings would guide further research on applications of Bayesian probabilistic matrix factorization and inspire more researchers to contribute in this domain.


Introduction
The past several decades have witnessed the great necessity in designing novel recommendation systems, keeping pace with the fast development of information technologies.The increasing popularity of recommendation systems has influenced human life in many aspects.The potential value of recommendation is being excavated in fruitful applications, such as online social voting recommendation on social networks like Facebook, research paper recommendation for accelerating the speed of crucial scientific findings [1], multimedia recommendation for satisfying users' interest [2], citation recommendation for sorting relevant papers [3].Interestingly, these commercial or academic applications have generalized well and at the same time brought us convenience in a personalized way.However, the demand has already confronted difficulties and challenges from both data quality and volume, requiring more novel algorithms or frameworks to deal with.A specific instance in recommendation research lies in that the ratings on items by users tend to be sparse and imbalanced, leading to the cold start phenomenon in the domain.On the other hand, the boom of internet allows the interactive behaviors like rating the goods bought before or feeding back the preferences of items such as foods, movies or music.In China, some famous corporations including Alibaba, Xiecheng Company have perceived the importance of feed-backing comments from customers and there is no doubt that accurate predictions on customers' preferences no matter in the future or later would bring much profits.It is obvious that the potential relationships between the users we take interests in may contribute to the recommendation in some sense and such social information involved in the model forms the representative family of recommendation algorithms as collaborative filtering.
To address these issues, we firstly make a brief survey regarding the probabilistic matrix factorization (PMF) algorithm which is proved to be effective in sparse and imbalanced recommendation dataset with the social information well exploited [4].Distinguished from the traditional methodologies which mainly capture the information from the user-item ratings, we combined the social information with the rating circumstances for the performance promotion in our study and a superior adaptive Bayesian based model was employed in shopping recommendation.
The remainders of paper are arranged as follows.Section2 elaborates the related developments of probabilistic matrix factorization and Bayesian based recommendation system.The experiments on the collected dataset using Bayesian probabilistic matrix factorization are described in section3 with some detailed analysis.Section4 summarizes our work and light the direction in our future research.

Review on Probabilistic Matrix Factorization
Probabilistic matrix factorization (PMF) is regarded as one of the outstanding algorithms in several recommendation competitions like Netflix Prize, compared to several collaborative filtering methodologies.The essence of the PMF is to make use of the low rank approximation and assume the potential distribution of dataset, overcoming the shortness of matrix factorization model.After both of user and item feature matrix obtained by low-rank approximating the rating matrix, we utilize the feature matrix to complete the potential missing ratings.Such model maintains well prediction accuracies in practice.Due to the advantages of PMF, it has successfully attracted researchers' attention as well as interest.D. Dueck et al. proposed a probabilistic sparse matrix factorization algorithm, which well explained the uncertainty of noise in different extents [5].Based on the matrix factorization ideology, R. Salakhutdinov et al. made assumption that userfeature matrix, item-feature matrix and user-item matrix were Gaussian in prior distribution, and then by modelling the latent variables, the PMF were derived in first time [6].Multiple rating indexes were integrated into a user-item weighted matrix, assuming the latent matrix obeyed the Gaussian distribution and density probabilistic distribution conditionally correlated with user-feature and item-feature matrixes [7].Another adaptation on the PMF was SoRec recommendation model [8], where user-feature vectors from both PMF model and confidence based matrix factorization model were combined to enhance the performance.Apart from the theoretical research on the PMF, some applications have also driven the interest of adaptation on PMF.Kernel probabilistic matrix factorization [7][9].

A Brief introduction to the Bayesian Probabilistic Matrix Factorization
In this section, we would focus our attention on the mechanism of Bayesian probabilistic matrix factorization (BPMF) algorithm proposed by R Salakhutdinov et al. [10] and the essence of the recommendation algorithm would be elaborated.
The PMF is the basic model for the BPMF.Generally speaking, the PMF is a linear probabilistic model for solving recommendation tasks with some plausible information containing N users, M items and N*M rating matrix as R. Missing elements in R are ratings which the user never response to the item.The element r ij in R records the preference score of the user i towards the item j.
We apply the low rank approximation to the rating matrix R and then obtain two matrix in low dimensions as Where the D*N matrix U and D*M matrix V are respectively user-feature matrix and item-feature matrix in low rank representations, describing the latent representations of users and items respectively.
The probabilistic graphical model for PMF can be translated in the Fig1.Assuming that the difference between the actual rating r ij and the predicted rating r ij ~ obeys the Gaussian distribution with mean 0 and variance σ, we derive By the constant shifting, the distribution of rating r ij can be represented as Meanwhile, the assumption on the prior distribution on both latent user and latent item is the Gaussian distribution as follows: Based on the Bayesian inference, we can acquire the loss function by minimizing the log of the posterior distribution accompanied with quadratic regularization terms.
Where the equations follow During the past years in the research, the adaptations on the feature dimension D and prior distribution's assignment make great improvements on the capability of PMF's generalization.
For the BPMF, the U and V are assumed to follow the Gaussian distributions instead of constants.
Where the samples {   ,    } are generated from a Markov chain.Detailed information on MCMC's application to the BPMF refers to [10].

Evaluation Metrics and Selected Dataset
In our research, the root means square error (RMSE) is chosen as the evaluation metric.More specifically, for the rating r ij which measures the user i's preference towards the item j and the predicted rating r ij ~ derived by the recommendation system, the RMSE is computed as Due to the severe penalty on inaccurate predictions using square error in RMSE, the evaluation metric is regarded as a strict but reasonable criteria.The dataset in our experiments is from the electronic business website as Epions (www.Epinions.com).The ratings reflecting the preference of users towards items are collected, processed and prepared in user-item matrix.Another crucial information matrix describes the friend relationship between users is also provided in Table1.In comparison to the commonly used dataset like Movielens and IMBD, the Epnions behaves rather sparse in social relationships.

Results and Analysis
To verify the effectiveness of the BPMF, we selected some frequently used popular algorithms which were believed to perform well in several recommendation tasks as baselines.Specifically, they were SVD [11], NMF [12], BaselineOnly [13], Slop-erOne [14], KNNBaseline [13] and CoClustering [15].In the experiments, both of the parameters λ U and λ V were set as 0.002 and the turns of iteration was 200.Some other parameters were selected as the default.Aiming at determining the optimal feature dimension D, we ranged the parameter from 5 to 20 at the step width 5 and the 5-fold cross validation was performed in final selection.The results were arrange in the Ta-ble2.From the table, we can notice that the value of RMSE decreased steadily in the small interval while such metric fluctuated severely when D was over 15.So 15 was chosen as the optimal value for the feature dimension.
A persuasive metric is the k-fold evaluation to average the testing results, in which the k-1 folds are prepared for model fitting with the last fold for testing and such manipulation is performed k times in one turn by ranging all of k-1 folds.
Furthermore, we carried out experiments using the above-mentioned seven algorithms in 10 independent turns and provided the results in average.The fold partition ranged from 1 to 5 with the step width 1.The Table3 recorded the averaged RMSE in ten turns using various algorithms in different folds and it revealed the superior performance of BPMF with the lowest RMSE.To further validate the outstanding performance of BPMF in statistics, the paired ttest was introduced to test the results of 10 independent turns with seven algorithms in the case when the 5-fold RMSE was chosen as the metric.Specifically, the 10 results in each turn obtained by BPMF were compared with those derived by NFM, SVD, BaselineOnly, SlopeOne, KNNBaseline and Coclustering respectively with the help of SPSS.The testing results indicated the BPMF outperformed others significantly.And the compared results were mapped into three element tuple as win\tie\loss.Then the counts of win\tie\loss in 10 turns with some baseline algorithm were summarized in the Table4 as follows.The potential reason for obtaining satisfying results using BPMF was that the Bayesian framework was more flexible in searching optimal hyper parameters and the model maintained high complexity to fit the dataset of high volume.

Conclusions
In the end, the Bayesian probabilistic matrix factorization was employed in online item recommendation backgrounded in the opinion-sharing platform.The BPMF showed its superiority in recommendation task, deriving the lowest RMSE in comparison to six representative algorithms.The conclusions were supported by the paired t-test in ten independent experiments and the persuasive results motivated us to research in depth in other applications.In the future, we would design more variant matrix factorization algorithms incorporating the other social information to promote the performance in recommendations.

Table 1 .
Related information about Epinions

Table 2 .
The RMSE in different D value

Table 3 .
Averaged k-fold RMSE in different algorithms