This paper presents an efficient approximation to the nonlinear SVM with Radial Basis Function (RBF) kernel. By employing second-order polynomial approximation to RBF kernel, the derived approximate RBF-kernel SVM classifier can take a compact form by exchanging summation in conventional SVM classification formula, leading to constant low complexity that is only relevant to the dimensions of feature. Experiments on pedestrian classification show that our approximate RBF-kernel SVM achieved classification performance close to the exact implementation with significantly low time and memory.