On Applying Experience Sampling Method to A/B Testing of Mobile Applications: A Case Study

Abstract : With the advent of mobile devices, the experience sampling method (ESM) is increasingly used as a convenient and effective way to capture user behaviors of, and evaluate mobile and environment-context dependent applications. Like any field based in situ testing methods, ESM is prone to biases from unreliable and unbalanced data, especially for A/B testing situations. Mitigating such effects can in turn incur significant costs in terms of the number of participants and sessions, and prolonged experimental time. In fact, ESM has rarely been applied to A/B testing nor do existing literatures reveal its operational details and difficulties. In this paper, as a step toward establishing concrete guidelines, we describe a case study of applying ESM to evaluating two competing interfaces for a mobile application. Based on the gathered data and direct interviews with the participants, we highlight the difficulties experienced and lessons learned. In addition, we make a proposal for a new ESM in which the experimental parameters are dynamically reconfigured based on the intermediate experimental results to overcome the aforementioned difficulties.
Type de document :
Communication dans un congrès
15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. Lecture Notes in Computer Science, LNCS-9297 (Part II), pp.203-210, 2015, Human-Computer Interaction – INTERACT 2015. 〈10.1007/978-3-319-22668-2_17〉
Liste complète des métadonnées

Littérature citée [10 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01599866
Contributeur : Hal Ifip <>
Soumis le : lundi 2 octobre 2017 - 15:41:30
Dernière modification le : mardi 3 octobre 2017 - 14:45:58

Fichier

346942_1_En_17_Chapter.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Myunghee Lee, Gerard Kim. On Applying Experience Sampling Method to A/B Testing of Mobile Applications: A Case Study. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. Lecture Notes in Computer Science, LNCS-9297 (Part II), pp.203-210, 2015, Human-Computer Interaction – INTERACT 2015. 〈10.1007/978-3-319-22668-2_17〉. 〈hal-01599866〉

Partager

Métriques

Consultations de la notice

23

Téléchargements de fichiers

3