This paper presents an approach for the recognition of on-line handwritten mathematical expressions. The Hidden Markov Model (HMM) based system makes use of simultaneous segmentation and recognition capabilities, avoiding a crucial segmentation during pre-processing. With the seg-mentation and recognition results, obtained from the HMM-recognizer, it is possible to analyze and interpret the spatial two-dimensional arrangement of the symbols. We use a graph grammar approach for the structure recognition, also used in off-line recognition process, resulting in a general tree-structure of the underlying input-expression. The resulting constructed tree can be translated to any desired syntax (for example: Lisp, L A T E X, OpenMath. . .).