This paper focuses on the detection of small objects (e.g. vehicules in aerial images) on complex back-grounds (e.g. natural backgrounds). A key contribution of the paper is to show that, in such situations, learning a target model and a background model separately is better than training a unique discriminative model. This contrasts with standard object detection approaches for which objects vs. background classi fiers use the same types of visual features/models for both. The second contribution lies in the use of manifold learning approaches to build these models. The proposed detection algorithm is validated on the publicly available OIRDS dataset, on which we obtain state-of-the-art results.