In graph analysis, a classic task consists of computing similarity measures between (groups of) nodes. For latent space random graphs, nodes are associated to unknown latent variables. One may then seek to compute distances directly in the latent space, using only the graph structure. In this paper, we show that it is possible to consistently estimate entropic-regularized optimal transport (OT) distances between groups of nodes in the latent space. We provide a general stability result for entropic OT with respect to perturbations of the cost matrix. We then apply it to several examples of random graphs, such as graphons and geometric graphs on manifolds. Along the way, we prove concentration results for the so-called universal singular value thresholding estimator, and for the estimation of geodesic distances on a manifold, that are of independent interest.