Computer architects rely on on-the-fly simulation for designing microprocessor- s. However, these simulations cannot use large workloads consuming several CPU hours since, using current software approaches, even skipping instructions, is very time consuming. This paper proposes a new cost effective trace collection and on-the-fly simulation approach. The original application code is lightly annotated to provide a fast (direct) execution mode. An embedded instruction-set emulator enables trace collection or on-the-fly simulations. At run time, dynamic switches are enabled from the fast mode to the emulation mode by the annotation code, and vice-versa. The instrumentat- ion tool, calvin2, and the instruction-set emulator, DICE, presented in this paper, exhibit low execution slowdowns (from 1.01 to 3.09) on the SPEC95 benchmarks running in fast mode. Such low execution slowdowns make it possible to use samples spread over a long running application for simulations. We compare our approach with the dynamic translation approach used in Shade. While our DICE emulator is not optimized for raw performance, our system will outperform dynamic translation provided that only a small part of the application execution is effectively simulated.