Janet Gibbs

Goldsmiths

Janet is exploring how multi-modal perceptual feedback contributes to a player's sense of presence in the virtual world. Jaron Lanier described Virtual Reality (VR) as the substitution of the interface between a person and their physical environment with an interface to a simulated environment. This interface is of particular significance in understanding how presence depends on the nature, extent and veridicality of our sensorimotor interaction with the virtual environment, and how that relates to our normal engagement with the real world. In practice, only selected parts of the interface are substituted - we are never fully removed from our physical environment.

Our perceptual apparatus evolved to make sense of changing sensations in multiple modalities originating naturally and coherently from the same event or percept. By contrast, in VR, individually crafted feedback using different technologies for each modality are coordinated to appear as if from a single source. VR benefits from a long history of visual and audio technologies, developed in harness for virtual experiences from cinema to computer games. Haptics is a relative newcomer that must be blended with them to create coherent multimodal perceptual experiences. Additionally, haptics is closely related to proprioception, and to the wide range of tactile senses—texture, heat, pain etc—that current VR systems do not address. Building on sensorimotor theory of perception, Janet aims to establish how our perceptual system responds to multi-modal feedback that almost, but not quite, matches what we are used to, in making sense of the simulated environment of VR.