This paper presents a probabilistic Bayesian framework for object tracking using the combination of a cornerbased model and local appearance to form a locally enriched global object shape representation. A shape model is formed by corner information and it is rendered more robust and reliable by adding local descriptors to each corner. Local descriptors contribute to estimation by filtering out some irrelevant observations, making it more reliable. The second contribution of this paper consists in introducing an online feature adaptation mechanism that enables to automatically select the best set of features in presence of time varying and complex background, occlusions, etc. Experimental results on real-world videos demonstrate the effectiveness of the proposed algorithm.