In this communication, we address the problem of approximating the atoms of a parametric dictionary A = {a(θ) : θ ∈ Θ}, commonly encountered in the context of sparse representations in "continuous" dictionaries. We focus on the case of translation-invariant dictionaries, where the inner product between a(θ) and a(θ) only depends on the difference θ − θ. We investigate the following general question: is there some lowrank approximation of A which interpolates a subset of atoms {a(θj)} J j=1 while preserving the translation-invariant nature of the original dictionary? We derive necessary and sufficient conditions characterizing the existence of such an "interpolating" and "translation-invariant" low-rank approximation. Moreover, we provide closed-form expressions of such a dictionary when it exists. We illustrate the applicability of our results in the case of a two-dimensional isotropic Gaussian dictionary. We show that, in this particular setup, the proposed approximation framework outperforms standard Taylor approximation.