Recent Publications

» List of All Publications

Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities
We address the challenge of unsupervised mistake detection in egocentric video of skilled human activities through the analysis of gaze …
PDF Cite
Gazing Into Missteps: Leveraging Eye-Gaze for Unsupervised Mistake Detection in Egocentric Videos of Skilled Human Activities
SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-training
We present a framework for pre-training of 3D hand pose estimation from in-the-wild hand images sharing with similar hand …
PDF Cite Code
SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-training
A Multimodal LLM-based Assistant for User-Centric Interactive Machine Learning
This paper proposes a system based on a multimodal large language model (MLLM) to assist non-expert users without prior experience in …
PDF Cite DOI
A Multimodal LLM-based Assistant for User-Centric Interactive Machine Learning
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
We propose a novel benchmark for cross-view knowledge transfer of dense video captioning, adapting models from web instructional videos …
PDF Cite
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
Learning Multiple Object States from Actions via Large Language Models
Recognizing the states of objects in a video is crucial in understanding the scene beyond actions and objects. For instance, an egg can …
PDF Cite Code DOI
Learning Multiple Object States from Actions via Large Language Models
ActionVOS: Action as Prompts for Video Object Segmentation
Delving into the realm of egocentric vision, the advancement of referring video object segmentation (RVOS) stands as pivotal in …
PDF Cite Code
ActionVOS: Action as Prompts for Video Object Segmentation
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects
We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such …
PDF Cite
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects
Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition
Compared with visual signals, Inertial Measurement Units (IMUs) placed on human limbs can capture accurate motion signals while being …
PDF Cite
Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition
WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding
In this paper, we address the challenge of fine-grained video event understanding in traffic scenarios, vital for autonomous driving …
PDF Cite
WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding
Evaluating User Experience and Data Quality in Gamified Data Collection for Appearance-Based Gaze Estimation
Appearance-based gaze estimation, which uses only a regular camera to estimate human gaze, is important in various application fields. …
PDF Cite DOI
Evaluating User Experience and Data Quality in Gamified Data Collection for Appearance-Based Gaze Estimation
See all publications