ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Publications Copernicus
Articles | Volume II-3/W2
02 Oct 2013
 | 02 Oct 2013

Interactive multi-object tracking for virtual object manipulation

Y. Guo, M. Ying Yang, and B. Rosenhahn

Keywords: Kinect, multiple object tracking, gesture recognition, ICP, 1D signature

Abstract. We present an interactive system to manipulate a virtual object by tracking multiple hands in 3D space using a Kinect device. The system segments hand shapes from a captured 3D scene by using depth information and active contours. A hand shape is recognized by a trained naive Bayes classifier, whether it belongs to a palm, a pointing hand form or both hands with simple occlusion. A plane is approximated by using RANSAC for a palm hand form, while a vector from the hand centroid to the fingertip is obtained for a pointing hand form by using ICP as tracking method. A shape of simple occluded two hands is split into a palm and an incomplete pointing hand form, whose missing data is estimated by using PCA. The system works in semi-real time.