Recent Publications

» List of All Publications

Generative Modeling of Shape-Dependent Self-Contact Human Poses
One can hardly model self-contact of human poses without considering underlying body shapes. For example, the pose of rubbing a belly …
PDF Cite Code
Generative Modeling of Shape-Dependent Self-Contact Human Poses
AssemblyHands-X: Modeling 3D Hand-Body Coordination for Understanding Bimanual Human Activities
Bimanual human activities inherently involve coordinated movements of both hands and body. However, the impact of this coordination in …
PDF Cite DOI
AssemblyHands-X: Modeling 3D Hand-Body Coordination for Understanding Bimanual Human Activities
Leveraging RGB Images for Pre-Training of Event-Based Hand Pose Estimation
This paper presents RPEP, the first pre-training method for event-based 3D hand pose estimation using labeled RGB images and unpaired, …
PDF Cite
Leveraging RGB Images for Pre-Training of Event-Based Hand Pose Estimation
EgoInstruct: An Egocentric Video Dataset of Face-to-face Instructional Interactions with Multi-modal LLM Benchmarking
Analyzing instructional interactions between an instructor and a learner who are co-present in the same physical space is a critical …
PDF Cite
EgoInstruct: An Egocentric Video Dataset of Face-to-face Instructional Interactions with Multi-modal LLM Benchmarking
Affordance-Guided Diffusion Prior for 3D Hand Reconstruction
How can we reconstruct 3D hand poses when large portions of the hand are heavily occluded by itself or by objects? Humans often resolve …
PDF Cite DOI
Affordance-Guided Diffusion Prior for 3D Hand Reconstruction
Cross-View Correspondence Modeling for Joint Representation Learning Between Egocentric and Exocentric Videos
Joint analysis of human action videos from egocentric and exocentric views enables a more comprehensive understanding of human …
PDF Cite Code DOI
Cross-View Correspondence Modeling for Joint Representation Learning Between Egocentric and Exocentric Videos
EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs
Transferring and integrating knowledge across first-person (egocentric) and third-person (exocentric) viewpoints is intrinsic to human …
PDF Cite Code DOI
EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs
Unveiling Egocentric Reasoning with Spatio-Temporal CoT
Egocentric video reasoning focuses on the unseen, egocentric agent who shapes the scene, demanding inference of hidden intentions and …
PDF Cite Code
Unveiling Egocentric Reasoning with Spatio-Temporal CoT
Vinci: A Real-time Smart Assistant based on Egocentric Vision-language Model for Portable Devices
We present Vinci, a vision-language system designed to provide real-time, comprehensive AI assistance on portable devices. At its core, …
PDF Cite Code DOI
Vinci: A Real-time Smart Assistant based on Egocentric Vision-language Model for Portable Devices
Egocentric Action-aware Inertial Localization in Point Clouds with Vision-Language Guidance
This paper presents a novel inertial localization framework named Egocentric Action-aware Inertial Localization (EAIL), which leverages …
PDF Cite Code
Egocentric Action-aware Inertial Localization in Point Clouds with Vision-Language Guidance
See all publications