Fully automatic localization of anatomical structures in 2D and 3D radiological data sets is important in both computer aided di- agnosis, and the rapid automatic processing of large amounts of data. We present a simple, accurate and fast approach with low computa- tional complexity to nd anatomical landmarks, based on a multi-scale regression codebook of informative image patches and encoded landmark contexts. From a set of annotated training volumes the method captures the ap- pearance of landmarks over several scales together with relative posi- tions of neighboring landmarks and a spatial distribution model. During multi-scale search in a target volume, starting from the coarsest level, each landmark model predicts all landmark positions it has encoded, with the median of all predictions yielding the nal prediction for each scale. We present results on two challenging data sets (hand radiographs and hand CTs), where our method achieves comparable accuracy to the state of the art with substantially improved run-time.