Project overview
In our recent VR dyad study, a Receiver estimated which of 80 identical, uniformly spaced markers an avatar’s operator attended. Real scenes are nothing like that: targets are objects with semantics, salience, and unequal prior probability. When only three objects sit on a table, a coarse body cue plus a scene prior may identify the referent perfectly; when many similar objects cluster together, kinematics must do all the work. This project asks how contextual information combines with bodily cues when observers decode attention.
Project motivation
Theories of social attention describe gaze decoding as functionally sufficient inference under uncertainty rather than point-precise measurement [1], and dialogue research shows pointing and describing trade off in grounding referents [2]. If context carries much of the burden in realistic scenes, the parent study’s body-first hierarchy may be even more robust than reported (coarse cues suffice more often), or context could re-open a role for fine cues by disambiguating between close-together candidates. Quantifying this trade-off directly informs how attention-aware systems should fuse scene knowledge with kinematic estimates.
Project goals
- Context manipulation. Extend the existing Unity environment with scenes that vary target semantics and layout: uniform grid (replication baseline), sparse meaningful objects, and dense clusters of similar objects.
- Cue-fusion analysis. Model Receiver estimates as a combination of kinematic evidence and scene priors (for example with a Bayesian cue-combination or regression framework), and quantify how much decoding accuracy context contributes on top of the body.
You will
- Perform a literature review on referential grounding and social attention in context
- Design and build the contextual scenes in the existing Unity VR environment
- Run a dyad study with the existing motion-capture and eye-tracking pipeline
- Model the joint contribution of context and kinematics to decoding accuracy
- Summarize your findings in a thesis and present them to an audience
- (Optional) co-write a research paper
You need
- Strong communication skills in English
- Good knowledge of Unity (scene building)
- Interest in probabilistic modeling
References
- Stephenson, L. J., Edwards, S. G., & Bayliss, A. P. (2021). From gaze perception to social cognition: The shared-attention system. Perspectives on Psychological Science.
- Bangerter, A. (2004). Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue. Psychological Science, 15(6). https://doi.org/10.1111/j.0956-7976.2004.00694.x
- Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica, 26.
- The parent study on avatar-mediated shared attention in virtual reality; details are available from the advisor on request.
Keywords: VR, Shared Attention, Scene Context, Cue Combination, Referential Grounding