Eye contact is a highly salient and fundamentally social signal. This entails that the mere perception of direct gaze may trigger differentiated neurobehavioral responses as compared to other gaze directions. We investigated this issue using a visual word-spelling task where faces under different gaze directions and head orientations were displayed on-screen concomitantly with the words. We show evidence for automatic increase of skin conductance response (SCR), indicative of arousal, associated with the perception of direct gaze as compared to both averted gaze and closed eyes. Moreover, the perception of averted gaze was associated with an increase of electromyographic (EMG) corrugator activity. These effects were observed in two demanding word-spelling tasks, but not in a simple letter decision task. We propose to interpret these findings in terms of the social value of direct and averted gaze and conclude that some circumstances such as the task at hand may be essential for uncovering the neurobehavioral responses associated with the perception of others' gaze.