This article explains evaluation methods for real-time au- dio to score alignment, or score following, that allow for the quantitative assessment of the robustness and precise- ness of an algorithm. The published ground truth data base and the evaluation framework, including file formats for the score and the reference alignments, are presented. The work, started for MIREX 2006, is meant as a first step towards a standardized evaluation process contributing to the exchange and progress in this field.