We address the problem of view independent object classification. Our aim is to classify moving objects of traffic scene surveillance videos into pedestrians, bicycles and vehicles. However, this problem is very challenging due to large object appearance variance, low resolution videos and limited object size. Especially, perspective distortion of surveillance cameras makes most 2D object features like size and speed related to view angles and not suitable for object classification. In this paper, we adopt the common constraint that most objects of interest in traffic scenes are moving on the ground plane. Firstly, we realize the ground plane rectification based on appearance and motion information of moving objects, which can be applied for normalization of 2D object features. An online learning framework is then described to achieve automatic object classification based on rectified 2D object features. Experimental results demonstrate the effectiveness, efficiency and robustness of the proposed method.