The minimum time function $T(\cdot)$ of smooth control systems is known to be locally semiconcave provided Petrov's controllability condition is satisfied. Moreover, such a regularity holds up to the boundary of the target under an inner ball assumption. We generalize this analysis to differential inclusions, replacing the above hypotheses with the continuity of $T(\cdot)$ near the target, and an inner ball property for the multifunction associated with the dynamics. In such a weakened set-up, we prove that the hypograph of $T(\cdot)$ satisfies, locally, an exterior sphere condition. As is well-known, this geometric property ensures most of the regularity results that hold for semiconcave functions, without assuming $T(\cdot)$ to be Lipschitz.