Background and purpose. Deformable image registration (DIR) is a core element of adaptive radiotherapy workflows, integrating daily contour propagation and/or dose accumulation in their design. Propagated contours are usually manually validated and may be edited, thereby locally invalidating the registration result. This means the registration cannot be used for dose accumulation. We present and validated a novel multi-modal DIR algorithm that incorporates contour information to guide the registration. This ensures that the estimated deformation vector field and warped dose are in accordance with operator-validated contours. Materials and methods. The proposed algorithm minimizes both a normalized gradient field-based data-fidelity term on the images and an optical flow data-fidelity term on the contours. The Helmholtz-Hodge decomposition was incorporated to ensure anatomically plausible deformations. The algorithm was validated for same-and cross-contrast Magnetic Resonance (MR) image registrations, Computed Tomography (CT) registrations, and CT-to-MR registrations for different anatomies, all based on challenging clinical situations. The contour-correspondence, anatomical fidelity, registration error, and dose warping error were evaluated. Results. The proposed contour-guided algorithm considerably and significantly increased contour overlap, decreasing the mean distance to agreement by a factor of 1.3 to 13.7, compared to the best algorithm without contour-guidance. Importantly, the registration error and dose warping error decreased significantly, by a factor of 1.2 to 2.0. Conclusion. Our contour-guided algorithm ensures that the deformation vector field and warped quantitative information are consistent with the operator-validated warped contours. This presents a feasible semi-automatic strategy for spatially correct warping of quantitative information even in difficult and artefacted cases.