Virtual Reality: Whole Body Movement, Haptics, and Locomotion

The use of virtual reality (VR) technology in education offers a unique opportunity for immersive learning experiences that tap into the user’s senses of sight, touch, movement, and hearing, rather than focusing solely on visual elements. This was explored further by Kathy Mills, Laura Scholes, and Alinta Brown (2022) and Lesley Friend and Kathy Mills (2021) as part of the ARC Future Fellowship ‘Sensory Orchestration’ project. We have already covered some of this work at Sensory Orchestration – Project Highlights, Makerspaces: soundscapes, a touch typology & the sense of smell! and most recently, Virtual Reality: Story writing & virtual painting.

Photo by Julien Tromeur on Unsplash

In VR painting, communication expands on a range of sensory perceptions. Essentially, VR uses the sensory system of the body. The research teams (Friend & Mills, 2021; Mills, Scholes, & Brown 2022) explored this concept by using Google TiltBrush™ for virtual painting to study how students created three-dimensional artworks that conveyed emotions, and body movement.

The team studied how this full-body movement through virtual painting, is a unique way of communicating. In this method, participants used a digital gaming controller to paint with virtual brushes. Instead of traditional paint and canvases, they created images using invisible ink that appears in the air.

What is fascinating about virtual painting is that it does not follow the usual rules of physics, like gravity. With the help of in-air haptics, artists can make 3D images that seem to float in the immersive environment, defying gravity. It offers a whole new world of artistic expression. Virtual painting includes a variety of body movements (Friend & Mills, 2021; Mills, Scholes, & Brown, 2022), as outlined in Table 1:

As seen in Table 1, the students fully engaged their bodies during the activity. They utilised a range of movements, such as nodding their heads, twisting their torsos, performing squats, pivoting, and stepping over objects as they immersed themselves in the experience.

Photo by Stephan Sorkin on Unsplash

Our observations highlight the increasing accessibility of virtual reality (VR) technologies as a means of expressing creative concepts. These technologies offer unique avenues for employing our physicality and senses to communicate in innovative ways. In this context, we have been able to demonstrate that the students effectively harnessed their senses and incorporated bodily movements into their creative endeavours within immersive, three-dimensional environments. Further details about this work are covered in our forthcoming final project report, and in open access publications: Immersive virtual reality (VR) for digital media making: transmediation is key, and Virtual reality and embodiment in multimodal meaning making.

References

Friend, L. & Mills, K.A. (2021). Researching children’s virtual reality. In Serafini, F. (Ed.) Beyond the Visual: Researching Multimodal Phenomena. Teachers College Press. https://www.tcpress.com/beyond-thevisual-9780807766842

Mills, K. A., & Brown, A. (2022). Immersive virtual reality (VR) for digital media making: Transmediation is key. Learning, Media and Technology, 47(2), 179–200. https://doi.org/10.1080/17439884.2021.1952428

Mills, K. A., & Brown, A. (2023). Smart glasses for 3D multimodal composition. Learning Media & Technology, 1-22. [Published first online] http://dx.doi.org/10.1080/17439884.2023.2207142

Mills, K., Heck, E., Brown, A., & Funnell, P. (forthcoming). Senses Together – Multimodal Literacy Learning in Primary Education: Final Project Report. ARC Future Fellowship. Institute for Learning Sciences and Teacher Education Australian Catholic University.

Mills, K. A., Scholes, L. & Brown, A. (2022). Virtual reality and embodiment in multimodal meaning making. Written Communication, 39(3), 335–369. https://doi.org/10.1177/07410883221083517