Extracting information from multitrial magnetoencephalography or electroencephalography (EEG) recordings is challenging because of the very low SNR, and because of the inherent variability of brain responses. The problem of low SNR is commonly tackled by averaging multiple repetitions of the recordings, also called trials, but the variability of response across trials leads to biased results and limits interpretability. This paper proposes to decode the variability of neural responses by making use of graph representations. Our approach has several advantages compared to other existing methods that process single-trial data: first, it avoids the a priori definition of a model for the waveform of the neural response; second, it does not make use of the average data for parameter estimation; third, it does not suffer from initialization problems by providing solutions that are global optimum of cost functions; and last, it is fast. We proceed in two steps. First, a manifold learning algorithm, based on a graph Laplacian, offers an efficient way of ordering trials with respect to the response variability, under the condition that this variability itself depends on a single parameter. Second, the estimation of the variability is formulated as a combinatorial optimization that can be solved very efficiently using graph cuts. Details and validation of this second step are provided for latency estimation. Performance and robustness experiments are conducted on synthetic data, and results are presented on EEG data from a P300 oddball experiment.