In this paper, we propose a fast re-coloring algorithm to improve the accessibility for the color vision impaired. Compared to people with normal color vision, people with color vision impairment have difficulty in distinguishing between certain combinations of colors. This may hinder visual communication owing to the increasing use of colors in recent years. To address this problem, we re-map the hue components in the HSV color space based on the statistics of local characteristics of the original color image. We enhance the color contrast through generalized histogram equalization. A control parameter is provided for various users to specify the degree of enhancement to meet their needs. Experimental results are illustrated to demonstrate the effectiveness and efficiency of the proposed re-coloring algorithm.