In this paper, we present a novel approach for aerial obstacle detection using a stereo vision wearable device in the context of the visually impaired assistance. This kind of obstacles are specially dangerous because they could not be detected by the walking stick. The algorithm maintains a local 3D map of the vicinity of the user, which is estimated through a 6DOF egomotion algorithm. The trajectory is used to predict the next movement of the blind and the 3D information of the local map is used to evaluate a possible aerial obstacles in the next pose. A key stabilization algorithm is introduced in order to maintain the floor of the map continuously aligned with the horizontal plane. This is a very important task, because in a wearable 3D device, the relative transformation of the camera reference system with respect to the user and the environment is continuously changed. In the experimental section, we show the results of the algorithm in several situations using real data.