Image data is growing by leaps and bounds. Machine learning based applications that run on image datasets increasingly use local image feature descriptors. In a sense, we can now visualize images as objects with features as parts. Typically there are thousands of local features per image, resulting in an explosion of feature set size for already humungous image datasets. In this paper we present a feature set consolidation strategy based on two aspects: pruning of non-discriminatory features across different object types and association of matching features for the same object type. We showcase the effectiveness of our consolidation strategy by performing classification on a building dataset. Our method reduces storage space footprint (~5%) and classification runtime (~4%), and increases classification accuracy (~2%).