Reading 27: A Visual Approach to Sketched Symbol Recognition

Citation

Ouyang, Tom Y., and Randall Davis. "A visual approach to sketched symbol recognition." (2009).

PDF

Summary

The paper presents a image based method for sketch recognition. Earlier methods have used geometric or gesture based features for sketch recognition. This paper tries to apply ideas of computer vision to sketch recognition. The paper presents 5 novel image based features and and efficient metric for recognition which improves the accuracy significantly compared to state of the art systems.

The intuition is that images are perceived more as images by humans than as stroke points and their geometric properties. The authors present 5 new features:

1) 4 features based on the stroke direction with respect to 4 reference angles (0, 45, 90 and 135 degrees). The feature values are calculated as the difference between the stroke angle and the reference angle, and vary linearly between 1.0 (if the two are equal) and 0.0 (if they differ by more than 45 degrees)

2) 1 feature based on the endpoints of the stroke. It is equal to 1.0 if the point is at the beginning or end of a stroke and 0.0 otherwise.

The overall process is as follows:

Discussion

The distance metric the authors use to find the distance between an input sketch and a template sketch is as follows:

dx represents the shift in of a point inside a 3x3 box. The minimum distance among these shifts is taken to be the distance for that x, y. This is done for all x, y boxes. The image below shows this:

The paper applies two pruning methods:

1) Coarse Candidate Pruning, based on taking first k Principle Component features and Euclidean L2 distance metric.

2) Hierarchical Clustering.