We study the deep relations existing between differential logical relations and incremental computing, by showing how self-differences in the former precisely correspond to derivatives in the latter. We also show how differential logical relations can be seen as a powerful metatheoretical tool in the analysis of incremental computations, enabling an easy proof of soundness of differentiation.