Both Juri’s papers at IEEE ISMAR 2027 and ACM SUI 2027 have received awards. Congratulations!
2026
-
Grasping Real and Virtual Objects in AR: Study of Carry-Over Effects on Grasping Anticipatory Behavior
Juri Yoneyama, Etienne Peillard, Guillaume Moreau, and Ferran Argelaguet
IEEE Transactions on Visualization and Computer Graphics, 2026
When alternatively interacting with virtual and real objects in an Augmented Reality (AR), users experience sensory discrepancies, notably differences in visual and tactile feedback. While prior work has demonstrated that interacting with both virtual and real objects leads to a decrease in grasping performance for real objects, the underlying impact on the grasping process itself remains unclear. To explore how virtual object interactions can influence subsequent interactions with real objects, we conducted a pick-and-place experiment (N=28), where participants alternated interactions between real and virtual cylinders using two common virtual grasping methods, a pinch gesture or a physics-based grasping method. We measured the grasp anticipatory process, i.e. pre-shaping, on subsequent interactions with real objects (maximum grip aperture and time to peak aperture) and transition time. Results show that virtual interactions, no matter the virtual grasping method used, influenced pre-shaping towards subsequent real-object reach motion. Moreover, the physics-based grasping method significantly showed the largest grip aperture and the earliest preparation for subsequent interactions with real objects, suggesting greater grasp uncertainty. Taken together our experimental results demonstrate the importance of investigating virtual-real interactions in AR setups, highlighting the relevance of using the pre-shaping analysis as a formative metric for quantifying the modulation of the users’ grasping process during virtual-to-real transitions.
-
Interacting with Two Dominant Hands in AR: A Study of Aimed Motions with Real and Spatially Displaced Virtual Hands
Juri Yoneyama, Etienne Peillard, Guillaume Moreau, and Ferran Argelaguet
In SUI 2026 - ACM Symposium on Spatial User Interaction, 2026
A number of works have explored non-isomorphic hand mappings to increase the interaction space or to alter the speed/accuracy trade-off when using virtual hands in virtual reality. However, when these alterations are applied in an augmented reality context, where users perceive their real hands and interact with both real and virtual objects, the impact of these mappings remains unclear. In this paper, we investigate whether users can independently control both their real hand and a spatially displaced virtual hand, adapting their use based on task demands. To explore this, we conducted an experiment (N=24) in which participants had to touch virtual and real objects alternately. We controlled the virtual hand’s representation (anthropomorphic vs. spherical pointer) and the spatial offset between the real and the virtual representation (20 cm vs. 40 cm). We measured the transition time between real and virtual objects, the dynamics of the user’s hand (trajectory, velocity), and embodiment-related metrics (proprioceptive drift and subjective questionnaire responses). This is the first study to investigate whether such switching in an AR setup featuring one to multiple hand interactions. Our findings suggest that users can switch smoothly between real and virtual hands, highlighting the potential utility of multiple virtual hands techniques for mid-range interactions.
