Computer Science Location: Zoom (email presenter for link)
Add to Calendar 2020-09-04T08:30:00 2020-09-04T08:30:00 America/New_York PhD Qualifying Exam Presentation by Wenqiang Chen Continuous Micro Finger Writing for AR/VR Interaction Using Vibration on a Commodity Smartwatch Abstract: Zoom (email presenter for link)

Continuous Micro Finger Writing for AR/VR Interaction Using Vibration on a Commodity Smartwatch

Abstract:

Input is a significant problem for wearable devices, particularly for head-mounted virtual and augmented reality systems. Contemporary AR/VR systems use in-air gestures or handheld controllers for interactivity. However, mid-air handwriting provides a natural, subtle, and easy-to-use way to input commands and text. In this paper, we propose and investigate ViFin, a new technique for input commands and text entry, which tracks continuous micro finger-level writing with a commodity smartwatch through vibrations. Inspired by the recurrent neural aligner and transfer learning, ViFin recognizes continuous finger writing and works across different users and achieves an accuracy of 90\% and 91\% for recognizing numbers and letters, respectively. We quantify the accuracy of our approach through real-time system experiments in different arm positions, writing speeds, and smartwatch position displacement. Finally, a real-time writing system with two specific applications using AR smartglasses is implemented and assessed.

Committee:

  • Brad Campbell (Chair)
  • John Stankovic (Advisor)
  • Madhur Behl
  • Tariq Iqbal