We present a perturbation theory for finite dimensional optimization problems subject to abstract constraints satisfying a second order regularity condition. We derive Lipschitz and Hölder expansions of approximate optimal solutions, under a directional constraint qualification hypothesis and various second order sufficient conditions that take into account the curvature of the set defining the constraints of the problem. We also show how the theory applies to semi-definite optimization and, more generally, to semi-infinite programs in which the contact set is a smooth manifold and the quadratic growth condition in the constraint space holds. As a final application we provide a result on differentiability of metric projections in finite dimensional spaces.