Skip to main content
My Department

Understanding Gestures

Human interaction behaviors span a vast range of common or rare patterns, challenging machine learning algorithms.

Details: Improving automatic machine recognition of human input like gestures depends on developing an understanding of the input behaviors we are likely to see. In our early work in this space, datasets and tools like GREAT and GHoST provided ways for researchers and developers to choose gesture recognizers, vocabulary, and interactions. We continued to investigate gesture articulation patterns but for children. Our work demonstrated that children are less consistent than adults when making gestures, leading to lower recognition rates. We explored how to develop new methods of characterizing children’s gestures, as outlined by our studies. We have shown that humans are able to recognize children’s gestures better than machine algorithms, and future work continues to attempt to improve machine algorithms to reduce that gap.

Team Members
Faculty

Dr. Lisa Anthony

In collab with Jaime Ruiz, Jacob O. Wobbrock, and Radu-Daniel Vatavu.

Current Students

project closed

Former Students

Alex Shaw

Read more on our approaches to automatic recognition of gestures here.

Publications
Refereed Conference Papers
  • Shaw, A., Ruiz, J., and Anthony, L. 2017. Comparing human and machine recognition of children’s touchscreen stroke gestures. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI 2017). ACM, New York, USA, 32-40, Best Student Paper. [Pdf]
  • Shaw, A. and Anthony, L. 2016. Analyzing the articulation features of children’s touchscreen gestures. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI 2016). ACM, New York, NY, USA, 333-340. Nominated for Best Student Paper. [Pdf]
Refereed Conference Posters
  • Shaw, A. and Anthony, L. 2016. Toward a Systematic Understanding of Children’s Touchscreen Gestures. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI’2016) , San Jose, CA, 7 May 2016, p.1752-1759. [Pdf and Poster]

Read Alex Shaw’s Ph.D. thesis based in this project here.

Funding

This work is partially supported by National Science Foundation Grant Awards #IIS-1552598, #IIS-1218395 / IIS-1433228 and IIS-1218664. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect these agencies’ views.

[Ancient] News and Updates

back to home