In this paper, we present a novel method for in-situ 3D sketching of polyhedral scenes. A video camera is used as both an interaction and a tracking device, which makes the system particularly suitable for handheld devices such as PDAs and mobile phones. The efficiency and accuracy of the method are demonstrated using a miniature scene and a real outdoor scene.