$-Family of Gesture Recognizers

$-Family of Gesture Recognizers

We have been working on developing, evaluating and extending a suite of simple, fast, accurate pen and finger gesture recognizers, which we call the “$-family”. We are working with Jacob Wobbrock at UW and Radu-Daniel Vatavu at University Stefan Cel Mare of Suceava.

The $-family began as an extension to Wobbrock et al’s $1 unistroke gesture recognition algorithm. The goal of $N, like $1 before it, is to implement a simple, fast, reasonably accurate recognizer that can be easily incorporated into rapid prototyping processes, to give application designers the power of trying out different input methods without having to spend a lot of effort getting the recognition up and running. Try out an online demo of $N in JavaScript, and check out the detailed pseudocode listing. $N has some limitations, of course, but in our preliminary experiments, it has performed comparably to well-known recognizers, with much lower start-up costs.

$P was the next member of the $-family, and it was faster and more robust to differences in how users actually make (articulate) gestures. $P can recognize gestures made with any number of strokes in any order or direction with low storage cost, whereas $N had high storage costs since it internally represented all the same variations that $P’s matching algorithm can handle seamlessly. Try out an online demo of $P in JavaScript, and check out its detailed pseudocode listing. $P received the Best Paper Award when it was published at ICMI’2012.

The most recent incarnation of the $-family that members of the INIT Lab contributed to is $Q, a “super-quick” version of $P that uses some straightforward algorithmic optimizations to reduce calculation costs of the matching process. As such, it is particularly suited to extremely low-resource devices. Though it can run on other devices, novice programmers may choose to stick with the original $P formulation if some of the $Q features like look-up tables are a bit advanced for their skills. It was nominated for Best Paper when it was published at MobileHCI’2018. Try out an online demo of $Q in JavaScript, and check out its detailed pseudocode listing.

For help in choosing the best member of the $-family for your needs, check out the “cheat sheet” which we include in the $P paper. Also, we provide more detailed information, and C# reference implementations, on our full project websites for $N, $P, and $Q!

The Team

Dr. Lisa Anthony
Alex Shaw
Qingchuan Zhao “Bruce” (Former Member)

$-Family Papers

Refereed Conference Papers

  1. Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. 2018. $Q: A super-quick, articulation-invariant stroke-gesture recognizer for low-resource devices. Proceedings of the ACM Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’18). Barcelona, Spain (September 3-6, 2018). New York: ACM Press, to appear. [PDF]
  2. Vatavu, R., Anthony, L., and Wobbrock, J.O. 2012. Gestures as Point Clouds: A $P Recognizer for User Interface Prototypes. Proceedings of ACM International Conference on Multimodal Interaction (ICMI’2012), Santa Monica, CA, p.273-280. [PDF]
  3. Anthony, L. and Wobbrock, J.O. 2012. $N-Protractor: A Fast and Accurate Multistroke Recognizer. Proceedings of Graphics Interface (GI’2012), Toronto, Canada, p.117-120. [PDF]
  4. Anthony, L. and Wobbrock, J.O. 2010. A Lightweight Multistroke Recognizer for User Interface Prototypes. Proceedings of Graphics Interface (GI’2010), Ottawa, Canada, 02 Jun 2010, p.245-252. [PDF]
Blogposts