Principal curvature-based region detector

The principal curvature-based region detector, also called PCBR ^[1] is a feature detector used in the fields of computer vision and image analysis. Specifically the PCBR detector is designed for object recognition applications.

Local region detectors can typically be classified into two categories: intensity-based detectors and structure-based detectors.

Intensity-based detectors depend on analyzing local differential geometry or intensity patterns to find points or regions that satisfy some uniqueness and stability criteria. These detectors include SIFT, Hessian-affine, Harris-Affine and MSER etc.
Structure-based detectors depend on structural image features such as lines, edges, curves, etc. to define interest points or regions. These detectors include edge-based region (EBR) and scale-invariant shape features (SISF)

From the detection invariance point of view, feature detectors can be divided into fixed scale detectors such as normal Harris corner detector, scale invariant detectors such as SIFT and affine invariant detectors such as Hessian-affine.

The PCBR detector is a structure-based affine-invariant detector.

Why a new detector?

In many object recognition tasks, within-class changes in pose, lighting, color, and texture can cause considerable variation in local intensities. Consequently, local intensity no longer provides a stable detection cue. As such, intensity-based interest operators (e.g., SIFT, Harris-Affine)–and the object recognition systems based on them–often fail to identify discriminative features. An alternative to local intensity cues is to capture semi-local structural cues such as edges and curvilinear shapes. These structural cues tend to be more robust to intensity, color, and pose variations. As such, they provide the basis for a more stable interest operator, which in turn improves object recognition accuracy. PCBR detector was developed to exploit these more reliable image structural cues.

Algorithm description

Step 1: Curvilinear structure detections

As a structure-based detector, PCBR does not use edges, instead, it uses curvilinear structures, also called ridges. Curvilinear structures detection generates a single response for both lines and edges, producing a clearer structural sketch of an image than is usually provided by the gradient magnitude image. The Steger's algorithm ^[2] is modified to get the curvilinear images. As only the first step of this algorithm is used which is to calculate the principal curvature images, the principal curvature is adopted as the name of this detector. To get the principal curvature, the Hessian matrix is calculated:

$H(\mathbf {x} )={\begin{bmatrix}L_{xx}(\mathbf {x} )&L_{xy}(\mathbf {x} )\\L_{xy}(\mathbf {x} )&L_{yy}(\mathbf {x} )\\\end{bmatrix}}$

where $L_{aa}(\mathbf {x} )$ is second partial derivative of the image evaluated at point x in the $a$ direction and $L_{ab}(\mathbf {x} )$ is the mixed partial second derivative of the image evaluated at point x in the $a$ and $b$ directions. The maximum and minimum eigenvalues of this matrix form two images which correspond to white lines on black background and black lines on white background.

Step 2: Seeking characteristics and robustness in scale space

To make this detector scale invariant and improve the detection robustness, the process of David Lowe's ^[3] SIFT detector is simulated to detect principal curvilinear structure in scale space. Local maximum images of principal curvature values are used to define regions.

Step 3: Defining regions by enhanced watershed algorithms

The principal curvature images are cleaned by a morphological closing and eigenvector-flow guided hysteresis thresholding. Then traditional watershed algorithm is applied on images to acquire regions.

Step 4: Stable region selections

Similar to the process of selecting stable regions via thresholding in MSER,^[4] stable regions are selected across local scale changes. To achieve this, overlap error is computed across each triplet of consecutive scales. If the region overlap error is greater than 90%, only one region is kept. If the error is greater than 70% and less than 90%, all regions are kept. If overlap is less than 70%, discard these regions. These numbers are determined by the analysis of sensitivity of the SIFT descriptor.

How does the PCBR differ?

It is a structure-based detector.
It is designed to handle within-class variance.
It is used when local intensity is not stable.
It detects a semi-local characteristic region.

Software packages

Binary code of an implementation of PCBR can be downloaded from Tom Dietterich's webpage.^[5]

References

^ Deng, H.; Zhang, W.; Mortensen, E.; Dietterich, T.; Shapiro, L. (2007). Principal Curvature-based Region Detector for Object Recognition (PDF). IEEE Conference on Computer Vision and Pattern Recognition.
^ Steger, C. (1998). "An unbiased detector of curvilinear structures". IEEE Transactions on Pattern Analysis and Machine Intelligence. 20 (2): 113–125. CiteSeerX 10.1.1.42.2266. doi:10.1109/34.659930.
^ Lowe, D. G. (2004). "Distinctive image features from scale-invariant keypoints" (PDF). International Journal of Computer Vision. 60 (2): 91–110. doi:10.1023/B:VISI.0000029664.99615.94. S2CID 221242327.
^ Matas, J.; Chum, O.; Urban, M.; Pajdla, T. (2002). "Robust wide baseline stereo from maximally stable extremal regions" (PDF). Proceedings of the British Machine Vision Conference 2002. pp. 384–393. doi:10.5244/C.16.36. ISBN 1-901725-19-7.
^ Dietterich, Tom. "PCBR" (ZIP file).

[Hongli01-1] Deng, H.; Zhang, W.; Mortensen, E.; Dietterich, T.; Shapiro, L. (2007). Principal Curvature-based Region Detector for Object Recognition (PDF). IEEE Conference on Computer Vision and Pattern Recognition.

[Steger01-2] Steger, C. (1998). "An unbiased detector of curvilinear structures". IEEE Transactions on Pattern Analysis and Machine Intelligence. 20 (2): 113–125. CiteSeerX 10.1.1.42.2266. doi:10.1109/34.659930.

[DavidLowe01-3] Lowe, D. G. (2004). "Distinctive image features from scale-invariant keypoints" (PDF). International Journal of Computer Vision. 60 (2): 91–110. doi:10.1023/B:VISI.0000029664.99615.94. S2CID 221242327.

[4] Matas, J.; Chum, O.; Urban, M.; Pajdla, T. (2002). "Robust wide baseline stereo from maximally stable extremal regions" (PDF). Proceedings of the British Machine Vision Conference 2002. pp. 384–393. doi:10.5244/C.16.36. ISBN 1-901725-19-7.

[5] Dietterich, Tom. "PCBR" (ZIP file).

[1]

[2]

[3]

[4]

[5]