In content-based image retrieval systems, the main approaches based on the query-by-example paradigm involve an approximate search on the whole image, which requires a global description of it. When considering tasks like object recognition or partial queries on particular area, these methods become inadequate and more local characterizations must be employed. In this context, image description based on points of interest appear best adapted. The point characterization which proved reliable is based on invariants to rotation and in particular on combinations of the Hilbert's differential invariants. For gray value images, such a description used to be considered up to third order at least. More recently, generalizations to color images were proposed for stereovision and image retrieval. Some of them propose to consider the invariants only at first order and to enrich the characterization with geometrical constraints for describing spatial relations between points, while others consider higher order invariants and compute some combinations of them to achieve illumination changes invariance. In this report, we discuss the advantages and drawbacks of these different choices, with the aim of proposing an optimal use of color points of interest for content-based image indexing and retrieval.