The Effect of Expression Geometry and Facial Identity on the Expression Aftereffect

Few studies have systematically examined the effect of expression geometry and facial identity on this cross-identity expression aftereffect. This issue is, however, critical for understanding the nature of expression adaptation. We measured expression aftereffects using a cross-identity/cross-expression geometry factorial design. The results show that expression aftereffect would reduce if adaptor and tests have different identities or expression geometries, and that identity-independent expression aftereffect is relatively more robust to variance in expression geometry in comparison to identity-dependent expression aftereffect. Based on these results, we discussed their psychophysical and physiological implication for understanding the nature of identity-dependent and identity-independent expression representations.


Introduction
Adaptation is a universal phenomenon in the human visual system, which refers to that the previous sensory experiences would affect the subsequent perception and response properties of visual system [1]. For instance, an observer adapts to a smiling face for a minute, and then looks at a middle expression within a series of ambiguous expression images morphing between smiling and angry expressions, this middle expression is tended to be perceived as an angry expression. This illusion is called expression aftereffect induced by the visual adaptation [2]. As visual adaptation can isolate and/or temporarily reduce the contribution of specific neural populations, visual adaptation is a time-honored tool for researchers to investigate the neural representation of human sensory system. Previous study has found expression aftereffect in different person condition was weaker than that in same person condition, which indicated that there are two different neural representations in facial expression system, i.e., identity-dependent neural *Corresponding Author representation and identity-independent neural representation [3]. However, two issues should be further considered. First, although variance in identity impaired expression aftereffect, when adapting faces and test faces are of different person, not only facial identity but also expression geometry is also changed. It raises a possibility that the reduction of expression aftereffect in different person condition is simply induced by the difference of expression geometry instead of the difference of face identity between adaptor and test faces. Second, as the similarity between the adaptor and the tests is an important factor to modulate the aftereffect in the low-level visual adaptation, it is necessary to examine whether the similarity of facial identities also affects the magnitude of expression aftereffect.
To clarify these two issues and systematically examine the effects of expression geometry and facial identity on the expression aftereffect, five experimental conditions were tested in current study. The interaction of facial identity and expression geometry were considered in 2X2 combinations to constitute four conditions (same or different: facial identity and expression geometry between adaptor and test face): same identity/same geometry, same identity/ different geometry, different identity/same geometry, different identity/ different geometry. In fifth condition, to examine the effect of identity similarity on the expression aftereffect, we used the computer-morphing technology to generate the artificial faces of different similarity, and tested on the same ambiguous images.

Method
There were five adapting conditions in the experiment, in which test faces were always the same among conditions, but the identity and/or the expression geometry of adaptor were manipulated. The five conditions are respectively termed as SI/SG, SI/DG, DI/SG, DI/DG, and DIDS, where the first two letters indicates whether adaptor and test face are of the same identity (SI: Same Identity) or not (DI: Different Identity), and last two letters indicates whether the expression geometry of adaptor are the same with that of test face (SG: Same Geometry) or not (DG: Different Geometry). The last DIDS refers to the condition in which the expression aftereffect is measured with the face adaptor of different similarity.

Subject
The subjects were 8 paid students with normal or corrected-to-normal vision (5 from Kochi University of Technology and 3 from Shanghai Maritime University, mean age: 20.2, SD=3.7). All 8 subjects participated SI/SG, SI/DG, DI/SG, DI/DG conditions, and 5 subjects from Kochi University of Technology participated the DIDS condition.

Stimuli and Apparatus
The face stimuli were selected from the affiliated image set of the Facial Action Coding System [4] and the Cohn-Kanade AU-Coded Facial Expression Database [5]， which are coded by Face Action Coding System and enables us to select same expression configuration for different photographic subjects. In SI/SG, SI/DG, DI/SG, and DI/DG conditions, happy, angry, surprised and disgusted expressions constituted two expression pairs, i.e., happy-angry and surpriseddisgusted expression pairs. Illustrated by the case of surprised-disgusted expression pair, we selected two female photographic subjects (F01 and F02) depicting two expression configurations (C01 and C02) of surprise and disgusted expressions, resulting in four combinations of identity and expression configuration (F01 with C01, F01 with C02, F02 with C01, and F02 and C02) (see Fig. 1a). The same method was used to select the adaptors of happy-angry expression pair, except that the two photographic subjects showing happy and angry expressions were male.
Using two expression images of the same photographic subject, we morphed a series of test faces using Abrosoft FantaMorph 5.0 for surprised-disgusted and happyangry expression pairs, respectively, with the nine middle ambiguous images, which varied from 30% to 70%, served as test stimuli.
The test faces in SI/SG, SI/DG, DI/SG, and DI/DG conditions were always the ambiguous expression images morphed between F01 with surprised expression (C01) and F01 with the disgusted expression (C01). We used the same test faces but different adaptors in the four experimental conditions. For instance, In the same identity/same configuration condition, the same images used to construct the test faces were used as the adaptors (i.e., F01 with surprised expression C01, and F01 with disgusted expression C01). In different identity/different configuration condition, the adaptors were the face images with different identities and expression configuration that differed from test faces (i.e., F02 with surprise expression C02, and F02 with disgusted expression C02).
In DIDS condition, we used computer-morphing technology to create the similar faces, because the morphing faces resembled two original faces when the features of two face are blended together. The similar faces were created by morphing the test face between F01 with G01 and F02 with G01 on smiling and angry expression pair using the Fanta morphing software. We created the nine similar faces with identity proportion of test face from zero to 100% in steps of 10% in terms of the scale of the morphing software, with zero refer to that the created adapting face is dissimilar to the test face, while one denote the adapting face is absolutely same with test face, we select those face with proportion equal with 30%, 50% serves as adapting face. The reason not to select the face images with similarity strength greater than 50% as adapting face, because these faces are too similar with test face so that subject may consider them as the same person rather than similar person.

Procedure
The experiment consisted of 10 blocks, each for one of eight experimental conditions (2 expression pair x 5 experimental conditions), performed in an order that was randomized across subjects. Each block includes 2 adapting images and each adapting image was presented 20 times in 9 test images, resulting in a total of 360 trials for each block. The trials for different test in a block were also randomized. The duration for one block is approximately 40 minutes. The subject participated one block every other day and finished all experiments within 20 days. Each subject was tested individually. They learned the experimental task through oral instruction and a short training session. In each trial, adaptor and test image were sequentially presented in the center of screen (Fig. 1b). After the presentation of test image, subjects perform a two-alternative forced choice (2-AFC) task to classify presented image into one of two categories (i.e., two expression images used to create the morphed series). The subject was instructed to attend to the face stimuli but no fixation point was given. This is to prevent subjects from overly attending to local facial features near fixation point. The duration of adapt stimuli and test stimuli were determined to 4000 and 200 ms, respectively, with 100ms noise mask between adaptation and test stage to minimize the possible apparent motion effects.

Result
All adapting conditions generated significant aftereffect (Fig. 2), confirming the expression aftereffect reported in the previous literatures [2] [6]. A two-way analysis of variance (ANOVA) has been performed on SI/SG, SI/DG, DI/SG, and DI/DG conditions. There was a significant main effect for adapting condition (F(3,18)=10.19, P<0.001), indicating that the expression aftereffect were significantly different for four different adapting conditions. There was no significant main effect for expression pairs (F(1,36)=0.11, p=0.742) and no interaction between expression pair and adapting condition, thus, the data from different expression pair was merged for the further analysis.
The size of expression aftereffect in DI/SG condition was weak relative to that in SI/SG condition (t(17)=6.57, p<0.001), indicating that the reduction of expression aftereffect across identity is held even when the adaptor and test face has the same expression geometry. This result suggests that only variance in expression geometry cannot well explain the reduction of expression aftereffect, thus confirming the inter-ference from identity system to the expression system. On other hand, the size of expression aftereffect in SI/DG condition was weaker than that in SI/SG condition (t(17)=5.84, p<0.001). In contrast, the aftereffect size in DI/SG condition was approximately the same with that in DI/DG condition (t(17)=0.19, p=0.851). These are new findings suggesting that variance in expression geometry impairs the identity dependent expression aftereffect, but not the identity independent expression aftereffect.
For DIDS condition, the result reveals that identity similarity between the adapting images and test could significantly increase the expression aftereffect. The adapting effects by 30% and 50% similar faces are 10.4% (S.E.M. =2.3%) and 22.1% (S.E.M. = 3.9%), and the adapting effect by the 50% similar faces is significantly stronger than that by the 30% similar faces (t8 = 2.843, p < 0.02). It is worth noting that the adapting effect by the similar faces is still weaker than that in the SI/SG adapting condition (100% similar faces), but stronger than that in the DI/SG adapting condition (0% similar faces) (Fig. 2b). It suggests that the expression aftereffect would increase as the function of identity strength, thus confirming that the expression aftereffect is identity-dependent.

Discussion
We firstly found that the expression aftereffect is still much reduced when adaptors and test face have the identical expression geometry. Such finding is consistent with Ellamil et al.'s work [7], which used the artificial faces of anger and surprise expression as adaptors and found the reduction of adaptation effect when adaptor and test faces have same expression morphing prototype but different facial texture and contour. Together, our and Ellamil et al.'s observation suggests that the reduced aftereffect cannot be simply attributed to the dissimilarity of expression geometry between adaptor and test faces, it does reflect a functional interference from identity system on expression system. This observation consolidates Fox's proposal of the identity dependent neural representation. We next found that identity similarity between the a) b) adapting images and tests could significantly increase the expression aftereffect, thus indicating that the expression aftereffect is identity-dependent in specific range of identity strength.
Our data fits with the hypothesis of partially overlapping neural representations for the perception of expression and identity [8] [9]. These two systems depend on the different facial component to extract the information and process the expression. As the structural reference hypothesis stated [10], the face structure information is not only important in the identity discrimination, but also used by observers as a reference to compute and recognize expressions. The identity dependent expression system may more rely on the facial shape and/or structure information (e.g., local edge, facial contour) to perform the expression geometry analysis, while the identity independent expression system seems to depend on the abstract emotional information. It would be interesting for the future researches to locate the cortex of these two different neural representations.