Project overview
Our recent VR dyad study found that observers decode where an avatar’s operator attends almost entirely from gross body orientation, with the head-on-torso rotation as the one unique cue and no measurable contribution from the rendered eyes. Everything in that study passed through an avatar-rendering pipeline. The obvious and hard question: does the same hierarchy hold between two physical people in a physical room, where real eyes with real sclera contrast are visible and nothing is retargeted or smoothed?
Project motivation
If the body-first hierarchy replicates face to face, it is a property of human social perception at these distances, and the psychophysics of peripheral gaze perception [1] would gain a strong ecological confirmation. If instead real eyes outperform rendered eyes, the gap isolates exactly what current avatar eye pipelines lose (resolution, latency, micro-movements), which is precisely what engine developers need to know [2]. Methodologically the project is a challenge in its own right: measuring shared attention without an HMD requires mobile eye tracking, motion capture without occluding the face, and a response method that does not re-introduce a display.
Project goals
- Real-world paradigm. Adapt the decoding task to a physical room: a Sender wearing mobile eye-tracking glasses and a mocap suit attends wall targets; a Receiver indicates the attended target (for example with a laser pointer or numbered grid), with the same 22.5 degree spacing as the parent study for comparability.
- Cross-medium comparison. Run matched real and VR sessions within participants and compare the cue hierarchies (body, head, eyes) between media, using the existing variance-decomposition pipeline on the mocap features.
You will
- Perform a literature review on gaze perception in real versus mediated settings
- Design the physical setup (OptiTrack, Pupil-style mobile eye tracker, response method)
- Adapt the existing preprocessing and feature pipeline to the real-world recordings
- Run the study and compare cue hierarchies across media
- Summarize your findings in a thesis and present them to an audience
- (Optional) co-write a research paper
You need
- Strong communication skills in English
- Hands-on attitude toward hardware and calibration
- Python skills (data pipelines); Unity is a plus for the VR arm
References
- Loomis, J. M., Kelly, J. W., Pusch, M., Bailenson, J. N., & Beall, A. C. (2008). Psychophysics of perceiving eye-gaze and head direction with peripheral vision. Perception, 37(9). https://doi.org/10.1068/p5896
- Waltemate, T., Senna, I., Hülsmann, F., Rohde, M., Kopp, S., Ernst, M., & Botsch, M. (2016). The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality. VRST 2016. https://doi.org/10.1145/2993369.2993381
- Bangerter, A. (2004). Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue. Psychological Science, 15(6). https://doi.org/10.1111/j.0956-7976.2004.00694.x
- The parent study on avatar-mediated shared attention in virtual reality; details are available from the advisor on request.
Keywords: Real World, Mobile Eye Tracking, OptiTrack, Shared Attention, Replication