In order to facilitate the user's access to a movie film the scene structure should be constructed. A general framework for contructing scenes consists in clustering the shots into groups and then merging overlapped groups to extract scenes. The clustering of shots represents the most challenging task and should be done correctly. This paper presents a mixed approach to cluster shots using both features computed on keyframes and the corresponding localized moving objects. Two shots are matched on the basis of color-metric histograms, color-metric autocorrelograms and the number of similar objects localized into them. Using the hierarchal classification technique, a partition of shots is identified for each feature separately. From all these partitions a unified partition is deduced based on a proposed distance between their components (clusters). The components of the resulted partition are then linked together using the temporal relationsof Allen in order to construct a temoral graph of clusters from which the scenes will be extracted. The experimental results of the proposed approach on three real-world video sequences demonstrate its performance. A comparative study between the original approach which uses only one descriptor and the extended one proposed here is analysed and reported.