This paper presents a novel approach that combines high level and low level features for the recognition of handwritten words. Given a word image, high level features are extracted from loosely segmented words. Such features are used with an HMM word classifier in a lexicon-driven approach. This classifier produces at the output a ranked list of the N-best recognition hypotheses consisting of text transcripts, segmentation boundaries of the word hypotheses into characters, and recognition scores. Given the segmentation boundaries produced by the HMM classifier, low level features are extracted from the character hypotheses. Such features are used with a segmental neural network classifier (SNN) in a hypothesis-driven approach. At combination level, the confidence scores produced by both HMM and SNN classifier are combined through simple combination rules. Experimental results on a large database have shown that the combination of high-level and low-level features reduces the word error rate in almost 71%.