Keypoints

Keypoint detection (extraction), feature description, and finally matching are closely related topics in computer vision. Applications that use keypoints range from panorama creation to three-dimensional reconstruction, from visual odometry to object tracking, and many other use cases.

The concept of a keypoint reflects the fact that not all image points, but only some of them, have a high probability of being identified unambiguously during matching. These are distinctive, stable points that are easy to locate. Over the last decade, as in almost every field of computer vision, significant progress has been made in developing local invariant features: keypoints that allow applications to define local image geometry and encode it in a way that is invariant to image transformations such as translation, rotation, scaling, and affine deformations.

This chapter covers techniques for detecting keypoints. Point description and matching strategies are discussed in greater detail in subsequent chapters.

A non-exhaustive list of algorithms for detecting keypoints is

Harris Corner

Harris formalizes the concepts of edges and corners mathematically by analyzing the eigenvalues of the structure tensor in the neighborhood of a point. This analysis distinguishes uniform regions, edges, and corners. It is invariant to geometric transformations such as translation and rotation, and only partially invariant to scale changes (Section 6.2);

Shi-Tomasi

a variant of Harris that directly uses the minimum eigenvalue of the autocorrelation matrix to detect points that can be tracked reliably over time; it forms the basis of the Kanade-Lucas-Tomasi tracker (Section 8.2);

AST

The Accelerated Segment Test family (Section 6.5) identifies keypoints by examining the brightness differences of points on a circle;

SIFT

analyzes the image at multiple resolutions and is invariant to similarity transformations (Section 6.3);

SURF

a detector and descriptor inspired by SIFT, but based on approximations of the Hessian matrix and the use of the integral image to reduce computational cost (Section 6.4).