Mutators represent a successful strategy in rapidly adapting asexual populations, but theory predicts their eventual extinction due to their unsustainably large deleterious load. While antimutator invasions have been documented experimentally, important discrepancies among studies remain currently unexplained. Here we show that a largely neglected factor, the mutational idiosyncrasy displayed by different mutators, can play a major role in this process. Analysing phylogenetically diverse bacteria, we find marked and systematic differences in the protein-disruptive effects of mutations caused by different mutators in species with different GC compositions. Computer simulations show that these differences can account for order-of-magnitude changes in antimutator fitness for a realistic range of parameters. Overall, our results suggest that antimutator dynamics may be highly dependent on the specific genetic, ecological and evolutionary history of a given population. This context-dependency further complicates our understanding of mutators in clinical settings, as well as their role in shaping bacterial genome size and composition.