Robust Feature Tracking in Image Sequences using View Geometric Constraints

Abstract. Interest point tracking across a sequence of images is a fundamental technique for determining scene motion characteristics. Traditionally, feature tracking has been performed with a variety of appearance-based comparison methods. The most common methods analyze intensity values of local pixels and subsequently attempt to match them to the most similar region in the following frame. This standard, though sometimes effective, lacks versatility. For example, these methods are easily confused by shadows, patterns, feature occlusion, and a variety of other appearance-based anomalies. To counteract the issues presented by a one-sided approach, a new method has been developed to take advantage of both appearance and geometric constraints in a complementary fashion. Using an iterative scheme, camera parameters are computed at each new frame and used to project a derived shape to new point coordinates. This process is repeated for each new frame until a trajectory is created for the entire video sequence. With this method, weight can be allocated as desired between both appearance and geometric constraints. If an issue arises with one constraint (e.g., occlusion or rapid camera movement), the other constraint will continue to track the feature successfully. Experimental results have shown that this method is robust to tracking challenges such as occlusion, shadows, and repeating patterns, while also outperforming appearance-based methods in tracking quality.