In our previous posts, we have noted that the pose project aims to understand how users articulate gestures to improve whole-body gesture recognition algorithms. As a first step to enable this understanding, we investigated the variations in how users move their body part when performing whole-body gestures. To facilitate this investigation, we designed a method, which we call filterJoint, that selects the key joints that users are actively moving when performing whole-body gestures. Our paper presenting the results of this investigation, titled “FilterJoint: Toward an Understanding of Whole-Body Gesture Articulation”, was accepted to the ACM International Conference on Multimodal Interaction (ICMI) conference. The paper details the design and evaluation of the filterJoint method and presents case studies showing how this method can enable an understanding of whole-body gesture articulation with corresponding implications on the design of whole-body gesture sets for both adults and children. Here is the abstract:

Classification accuracy of whole-body gestures can be improved by selecting gestures that have few conflicts (i.e., confusions or misclassifications). To identify such gestures, an understanding of the nuances of how users articulate whole-body gestures can help, especially when conflicts may be due to confusion among seemingly dissimilar gestures. To the best of our knowledge, such an understanding is currently missing in the literature. As a first step to enable this understanding, we designed a method that facilitates investigation of variations in how users move their body parts as they perform a motion. This method, which we call filterJoint, selects the key body parts that are actively moving during the performance of a motion. The paths along which these body parts move in space over time can then be analyzed to make inferences about how users articulate whole-body gestures. We present two case studies to show how the filterJoint method enables a deeper understanding of whole-body gesture articulation, and we highlight implications for the selection of whole-body gesture sets as a result of these insights.

Interested readers can find the camera-ready version (preprint) available here. The ICMI 2020 conference will be a virtual event because of the Coronavirus pandemic. The conference will be held from October 25 – October 29, where a pre-recorded presentation of the paper will be played live to the audience.

The work presented in this paper is a part of my dissertation work, so the progress I made on this work contributed significantly to advancing my dissertation research. I feel excited to continue the next steps of my dissertation work, focusing on using the results from the filterJoint method to understand children’s and adults’ natural motion qualities. I am looking forward to discussing my paper at the ICMI virtual conference. The conference will provide an avenue to gain valuable feedback from the audience regarding the paper’s conclusions.