Product Quantization for Nearest Neighbor Search

Abstract : We propose an approximate nearest neighbor search method based on product quantization, which decomposes the space into a Cartesian product of low dimensional subspaces and quantizes each subspace separately. A vector is represented by a short code corresponding to the quantization index. The Euclidean distance between two vectors can be efficiently estimated from their codes. Our method has an advantageous asymmetric version, where an estimated distance is computed between a vector and a code. Our approach is used to search for nearest neighbors efficiently, in particular in combination with an inverted file system. Experiments performed on SIFT and GIST image descriptors show excellent search accuracy outperforming three state-of-the-art approaches. The scalability of our approach is validated on a dataset of two billion vectors.