Interactive Perception & Graphics for a Universally Accessible Metaverse
Abstract:
With the dramatic growth of virtual and augmented reality, ubiquitous information is created from both the virtual and the physical worlds. However, it remains a challenge how to bridge the real and the virtual realms, and further, how to blend the metaverse into our daily life. In this talk, we will delve into a series of innovations in interactive perception and graphics, that aim to make the metaverse universally accessible.
With our consecutive works in Geollery.com, kernel foveated rendering, and neural compression, we presentreal-time pipelines of reconstructing a mirrored world and acceleration techniques. With DepthLab, Ad hoc UI, and Rapsai, we present real-time 3Dinteractions with depth maps, everyday objects, and how one can build suchpipelines with ease. With Montage4D, HumanGPS, and MonoAvatars, we demonstrate the great potential of digital humans in the metaverse. With CollaboVR,GazeChat, Visual Captions, ThingShare, SketchyScenes, and ProtoSound, we enrich communication with mid-air sketches, gaze-aware 3D photos, relevant visuals, object-focused views, and customized sound recognition.
We conclude the talk with highlights of the Google I/O 2022 Keynote, offering a visionary glimpse into the future of a universally accessible metaverse.
About the Speaker:
Ruofei Du is a Senior Research Scientist and Manager at Google and works on creating novel interactive technologies for virtual and augmented reality. His research covers a widerange of topics in VR and AR, including XR interactions, visual programming, augmented communication, XR social platforms, digital human, foveated rendering, accessibility, and deep learning in graphics. His research has been featured by Engadget, The Verge, PC Magazine, VOA News, cnBeta, etc. Dr. Du serves as an Associate Editor for IEEE Transactions on Circuits and Systems for Video Technology and Frontiers in Virtual Reality. He also served as a committee member in CHI 2021-2024, UIST 2022-2023, and SIGGRAPH Asia 2020 inXR. He holds 3 US patents and has published over 40 peer-reviewed publicationsin top venues of HCI, Computer Graphics, and Computer Vision, including CHI,SIGGRAPH, UIST, TVCG, CVPR, ICCV. Dr. Du holds a Ph.D. and an M.S. in Computer Science from University of Maryland, College Park; and a B.S. from ACM HonoredClass, Shanghai Jiao Tong University. Website: https://duruofei.com
SEE FUTURE SPEAKERS AT OUR WEBSITE.