Since the gap between main memory access time and processor cycle time is continuously increasing, processor performance dramatically depends on the behavior of caches and particularly on the behavior of small on-chip caches. In this paper, we present a new organisation for on-chip caches : the semi-unified cache organization. In most microprocessors, two physically split caches are used for respectively storing data and instructions. The purpose of the semi-unified cache organization is to use the data cache (resp. instruction cache) as an on-chip second-level cache for instructions (resp. data). Thus the associativity degree of both on-chip caches is artificially increased and the cache spaces respectively devoted to instructions and data are dynamically adjusted. The off-chip miss ratio of a semi-unified cache built with two direct-mapped caches of size S is equal to the miss ratio of a unified two-way set associative cache of size 2S ; yet, the hit time of this semi-unified cache is equal to the hit time of a direct-mapped cache ; moreover both instructions and data may be accessed in parallel as for the split data/instruction cache organization. Since on-chip miss penalty is lower than off-chip miss penalty, trace driven simulations show that using a direct-mapped semi-unified cache organization leads to higher overall system performance than using usual split instruction/data cache organization.