Recognizing human facial expression and emotion by computer is an interesting and challenging problem. In this paper, we propose a method for recognizing negative emotions through an appropriate representation of facial features from relevant face regions displayed in video streams and still images. A measure that is sensitive to facial movements is used in predefined regions of interest to detect the negative emotions. The experimentation has been performed on a standard dataset and live video streams and has showed promising results.