Vibration Intelligence: Bringing Machine Perception Beyond the Human-Like
Machine perception/sensing (MP) is an emerging cutting-edge, which bridges the gaps between both cyberspace and the physical environments. Ubiquitous computers are currently being expanded to sense the physical world by imitating humans, such as seeing, listening and reading. Seizing the booming pervasive sensors and AI, I propose to seek Vibration Intelligence (VibInt) and bring MP beyond human-like, intending to advance MP to sense the physical world through universal vibrations. In addition to everything's natural vibration frequency, all physical activities, even a heartbeat, cause vibrations, spreading through human bodies, machines, infrastructures, and even ocean fluids. These vibrations may be captured by ubiquitous sensors, such as accelerators, cameras, and lasers. Thus, I envision that gaining insight and actionable information from vibrations accelerates seamlessly coupling between the worlds of bits and atoms. However, VibInt presents challenges that differ from those tackled in other fields, such as CV and NLP. These prominent fields manually label large amounts of data for deep learning since people can understand those data. However, detected incomprehensible vibrations are difficult to use subsequently for manual data annotation. A further difficulty is detecting subtle, fine-grained and distinctive vibrations. The challenges of sensitivity to noise and disturbances, latent variables and transients significantly complicate the vibration modeling. Additionally, vibrations entail complex, multi-scale phenomena whose understanding and control remain to a large extent unresolved. To overcome these challenges, I first detect vibrations by using amplifier circuit configuration, filter, multi-sensors accumulation, and energy-based threshold segmentation. Second, PSD, Fisher score, reliefF algorithm, and cubic spline interpolation are utilized for feature extraction. Third, Newton's laws, KNN, Siamese network, and LSTM-CTC are adopted for vibration modeling. These methods can be used for many applications, such as exploiting human body vibrations as wearable interfaces; analyzing different water flow vibrations to facilitate robotic fish of exploring and inhabiting various habitats; recognizing smart building vibrations to monitor human activities; and spying to recover passwords using keystroke vibrations.
- Brad Campbell, chair, (CS/SEAS/UVA)
- Jack Stankovic, advisor, (CS/SEAS/UVA)
- Aidong Zhang (CS/SEAS/UVA)
- Seongkook Heo (CS/SEAS/UVA)
- Tariq Iqbal (ESE/SEAS/UVA)
- Mani Srivastava (ECE, UCLA)