Virtual Co-Eating: Making Solitary Eating Experience More Enjoyable

. Recently, a research on eating habits of Japanese college students revealed that they have a highly desire to communicate with others through co-eating. Even though better eating experience through co-eating is important, they often tend to be alone even more because of some reasons like small households, living alone, and having no time to find others for co-eating. Therefore, we believe that it may improve eating experience by incorporating a fictional character into the real space as a partner to eat together. For validating the idea, we have developed a virtual co-eating system for solving issues caused from solitary eating, and show some insights from its user study.


Introduction
Recently, Japanese college students tend to eat alone even more because of some reasons like small households, living alone and having no time to find others for co-eating.However, they have a highly desire to communicate with others thorough co-eating [1].Co-eating gives them better eating experience that cannot gain from solitary eating, and recent researches showed that good eating experience through co-eating closely related to positive emotion such as 'enjoyable', 'family', 'talking', 'together', 'friend', and 'smile' [1].Especially, 'family' and 'talking' are strongly related to good experience [2].Therefore, we believe that a partner to eat together is essential as a factor for improving solitary eating experience.
Typically, good eating experience is comprehensively obtained by combining not only the taste of food but also the environment, memory and experience such as a condition, feelings and atmosphere while eating.Therefore, there are several investigations on whether it is possible to improve eating experience by changing a variety of information other than the taste of food.For example, in a past study to report a system that enhances eating experience, a feeling of a full stomach is maintained due to changing the meal amount, if its appearance size is not changed by an augmented reality technology [3].
In this study, we introduce an alternative approach named virtual co-eating to improve eating experience by allowing a user to talk with a virtual partner and dissolving his/her dull eating experience for college students who frequently eat food alone.

Virtual Co-Eating System
In order to improve eating experience, the virtual co-eating system is designed by focusing on the two factors: "a preferable eating partner" and "enjoyable conversation".
For considering the first factor; we adopted a 3D virtual character named Kurei Kei (Pro-Nama-Chan, https://github.com/kureikei),who is typically preferable from the main target users of the virtual co-eating system who are Japanese computer science course students as a virtual partner because the character is very popular in young Japanese information technology communities.This character can easily guide to an enjoyable conversation with various common topics with the main target users.Also, in order to increase a feeling of the reality on the character, we decided to use the mirroring effect, which is the effect of having favor with others who do the same actions as a user.As shown in Fig. 1 (a), for example, in the virtual co-eating system, the user's smile is detected while talking with the virtual partner, and the partner also smiles as a feedback.For considering the second factor: enjoyable conversation, the virtual partner answers appropriate responses when a user speaks to her as shown in Fig. 1 (b) (c).On the other hand, when the user is silent, the partner asks the user a question.The partner talks three topics; common topics with users, topics related to the meal, and typical daily conversation.Also, in order to direct the atmosphere to a pleasant dining table and to guide a user to an enjoyable conversation experience, the system offers a background music taken from a video game for making eating atmosphere calm.
In the virtual co-eating system, the partner's behavior is generated in software that can create 3D character's motions called MMD.In addition, the voice of the partner is generated by a voice synthesizing software using 500 kinds of voice materials from actual voice actors.The character's 3D model, motion materials and voice materials are loaded into Unity, which is a game development platform.We wrote C# scripts and to control the playback timing of the partner's motion, expressions and voice.
If the system requires to wear a wearable device such as a head-mounted display, it makes a user hard to eat and to see, then it makes difficult to improve eating experience.Therefore, the system uses a projection technique to present the virtual partner in the real space.The screen of the mesh fabric is adopted for the projection to make the partner to be more three-dimensional in the real space as shown in Fig. 2.

Fig. 2. Projecting the Virtual Partner
In addition, facial expression recognition is performed with OpenCV written in Python, and the recognition results can be transmitted to Unity through WebSocket.By using OpenCV's haar-like feature classifier in two stages, we cut out the face, then we implemented the smile recognition function by detecting the feature quantity of the smile from among the cut out faces.Finally, the system controls the behavior of the partner by sending the recognition results to Unity through WebSocket.

Preliminary User Study
We conducted an experiment to investigate whether it is possible to improve eating experience by changing solitary eating experience to co-eating experience through the dialogue between participants who join to the experiment and the virtual partner.The main purpose of the user study is to find potential pitfalls of the current design not to validate the effectiveness of the design.This experiment was carried out by five participants; 20's male Japanese computer science course students who like animation and game content characters and who are usually eating alone.In the experiment, each participant first silently eats white rice balls with no good taste alone.Then, he uses this system to have the conversation with the virtual partner and eats the same rice balls.
Finally, we analyzed the evaluation of each factors designed in the system through the questionnaires and the interview with him.
From the results of the questionnaires, when comparing with the eating experience to eat food alone and with the virtual partner, 4 out of 5 participants answered that their eating experience was improved.From the interviews, three participants said "I felt it was more delicious than when I was eating alone."In addition, a participant claimed "It were not tasty because I were hungry at the time when eating alone.",and also commented "I did not matter for me.." when eating with the system.On the other hand, some participants who did not change eating experience with the system answered "Because I were talking with the partner, I did not pay an attention to meal."Thus, it is essential how the system supports the conversation with the partner.
More than half participants answered "Whichever" for the background music and talking common topics in the conversation with the virtual partner.Regarding common topics, in particular, three participants commented "It is better to have a daily conversation than to talk about programming as common topics during eating."In addition, one participant commented "I want to share feelings of delicious food through conversation." Regarding a background music, one participant answered "There is no background music in my actual daily life.",and two participants answered "I did not feel loneliness when there is a background music."On the other hand, two participants answered, "It would be better to use environmental sounds such as talking sounds and tableware sounds, not a background music from a video game." Regarding a smile feedback, four participants answered "I felt that my partner was smiling if I laughed, then I became more smiling."Thus, this approach seems that participants were able to make the virtual partner feel more preferable.On the other hand, one participant commented "It looked she was always laughing."

Conclusion
In this study, we developed a virtual co-eating system to improve eating experience.
The system provides better eating experience through a conversation with an empathetic virtual partner.The current system focuses on improving eating experience of 20's Japanese computer science course students.We conducted an experiment to extract pitfalls of the current design, and extracted some insights for future improvement of the current prototype system.