e2-Mask: Design and Implementation of a Mask-Type Display to Support Face-to-Face Communication

. In Face-to-Face communication, his or her face is an important part that decides the impression of a person because a person estimates the personality of the interlocutor from the features of the face of the interlocutor. Therefore, we made a hypothesis own impression and personality of giving to others by changing the own face freely. Then we proposed the e2-Mask that a mask-type display that synchronizes with facial expressions and substitutes an avatar for a human face.


Introduction
There is a relationship between people's visual appeal and the impression of their inner personality [1].In particular, this relationship is strong in the initial stage of interpersonal relationships [2].The psychologist Mehrabian mentioned that first impressions are formed from 7% language information, 38% auditory information and 55% visual information [3].Also, Secord mentioned that people estimate a person's character from the characteristics of his or her face [4].Therefore, the better the impression of a person's face, he or she can have a positive character such as being kind and friendly to interlocutor.Also, people change the impression of the face according to the situation, such as making up makeup or shaving a beard.In conventional research, there are some systems that change face virtually.Yamamoto et al. implemented a system to enable people to relax during public speaking by substituting the audience's faces with an avatar of a smiley pumpkin [5].However, this system aims to change the face of the audience and cannot change the user's own face when the audience sees the user's face.Then we think that if we can change the face of the user seen by an unspecified number of others, we can operate the impression and atmosphere of the user more flexibly and strongly.
Therefore, we proposed the system that a mask-type display that substitutes a person's face with an avatar.We call the proposed system the e2-Mask.The user's face is substituted with the avatar synchronously with the facial expression.Also, the e2-Mask can show the motion of the user's eyes.We think that using the e2-Mask cannot only operate the impression of the user's face but also support face-to-face communication.In a conversation with the user and an interlocutor, the impression of the using's looks can be improved by using the e2-Mask.Then interlocutor can assume that the user is positive character.Therefore, the e2-Mask will have effects that relax and increase the number of utterances.In addition, eye contact will be easier because the user talks through the avatar.

Related research
In conventional research, there are some systems that change people's face virtually.Here we will introduce the three such systems.The first system is the ChameleonMask, used for telepresence [6].The ChameleonMask shows the face of remote user on a display.In telepresence, an agent can act on behalf of a remote user by wearing the ChameleonMask.Therefore, the ChameleonMask gives a sense of presence to a conversation between a remote user and interlocutor.In contract, the e2-Mask shows the avatar on the display and is used in FTF communication.The second system is the Agencyglass, which is a system, in the form of sunglasses, that acts as a substitute for the user's eyes [7].The Agencyglass a liquid crystal plate of the same size as the eye is placed on the lens of the sunglasses, then the motion of the user's eyes, recorded in advance, is shown on the liquid crystal display.The Agencyglass shows the prerecorded motion of the user's eyes.However, the e2-Mask shows an avatar synchronized with expressions in real time.The third system is a support system for presentations [5].This system proposed a method of overlapping positive responses on audiences, and implemented a system which overlaps an image of a smiling pumpkin on each audience member using see-through HMD.This system is used in presentations.However, the e2-Mask is used in FTF communication such as a conversation by turns.

System structure
We design and implement a mask-type display that substitutes a person's face with an avatar1 .FTF communication involves scenarios such as a daily conversation, a meeting and an interview.As a situation in which the e2-Mask can be used, there is an interview.For example, an interviewee is nervous by the atmosphere of interview and an interviewer who look craggy.Therefore, the interviewee cannot speak own opinion and understand interviewer s questions.We think that the interviewer uses e2-Mask to substitute the face of the interviewer with an impressive avatar for the interviewee, so that the interviewee can ease tension and make it easy to talk.The e2-Mask is assumed for use in FTF communication as mentioned above.Therefore, we set up two design policies.
(1) Transmissibility: The e2-Mask usage scenario is FTF communication such as a daily conversation, a meeting or an interview.Therefore, we need to design the mask so that the user and the interlocutor can visually recognize each other's gestures, figures and expressions.Especially, facial expression is an effective means to transmit and read feelings such as joy or surprise [8].
Thus the e2-Mask changes the expression of the avatar in real time based on the user's expression, to transmit the user's feeling to the interlocutor at once.In addition, by covering the entire head like a costume, the voice of the e2-Mask user and the voice of the interlocutor are not heard.Therefore, we should consider that each other's voice is transmitted to the other.( 2) Wearability: The e2-Mask is used in various situations, as described above.
In improving the wearability, we should consider how to prevent the user's daily activity, such as shaking hands, holding luggage, moving up and down stairs and walking, from being impeded when wearing the e2-Mask.Thus it is important that the user can use the e2-Mask hands-free and that the e2-Mask is small and lightweight.
We implemented the prototype of the e2-Mask based on these two design policies.Figure 1 shows the appearance of someone wearing the e2-Mask.Figure 2 shows the structure of the system.The weight of the e2-Mask is 3.9 kg.Table 1 shows the manufacturer's name, specifications and number of the devices used.The reason for using this display is that the size fits people's faces and it is light.The reason for using this web camera is that it is small and it recognize even if the distance between the lens and the face is short.The following is an explanation of the composition of the e2-Mask.
(1) The facial expression output unit: The e2-Mask uses the Facerig to substitute an avatar for a human face.The Facerig is software that can reflect a person's face recognized by the web camera onto an avatar and stream facial expressions to synchronize with the avatar (Figure 2).Also, it is possible to select the avatar according to the preference of the user and interlocutor because the Facerig has over the 55 kinds of avatar.The display shows picture of avatar that attached user's expression obtained from the facial expression recognition camera to the avatar by using the Facerig.(2) The visual field acquisition unit: The e2-Mask uses the display and the web camera to secure the user's view.The display for securing visual field shows the image from the visual recognition camera.(3) Each fixed part: Figure 4 shows each fixed part of the e2-Mask.
(a) The fixing of the facial expression output unit and the visual field acquisition unit: The facial expression output unit and the visual field acquisition unit were put into display cases (the upper left in Figure 4).After that, the two displays were combined back to back.Also, the lower part of the display case of the facial expression output unit was fixed the facial expression recognition camera.The upper part of the display case of the visual field acquisition unit was fixed the visual recognition camera.
(b) The fixing of arm: Between the facial expression output unit and the visual acquisition unit is fixed an acrylic board with a hole of a size through which the arm can pass.We fixed to the arm, the facial expression output unit and the visual acquisition unit by passing the arm through the hole and fixing it to the acrylic board (the upper center in Figure 4).Then the arm and the backpack were connected by fixing the arm and the backpack with metal fittings (the upper right in Figure 4).(c) The fixed part of the mobile battery and the mini PC: An acrylic plate of sufficient size to fit the mobile battery and mini PC was fixed to the backpack.The mobile battery and mini PC were attached to the plate with double-sided tape (the lower in Figure 4).

Fig.
Fig. Wearing the prototype of the proposed system

Table 1 .
Information on the devices used in the prototype of the proposed system