Virtual Reality: Story writing & virtual painting

Virtual Reality (VR) technologies have transformed the way we learn by providing immersive and interactive environments that stimulate dialogue and abstract thinking (Fernandez, 2017). According to Hussein and Nätterdal (2015), the use of VR in classrooms is set to see exponential growth across education, from early childhood to higher education. In recent years, the scope of virtual technologies has expanded to encompass storytelling, social interaction, and playful learning experiences (Huang et al., 2010; Steed et al., 2018).

Photo by Jessica Lewis on Unsplash

In another phase of our research, we delved into the world of VR to explore the multimodal learning experiences of upper primary students. Our data collection and analysis focused on the transformative potential of VR in education. Our student participants engaged in multimodal composition through two key activities:

1. Story writing: Using VR technology, students embarked on a creative journey by retelling Greek myths while crafting accompanying illustrations. They also explored the world of virtual painting using Google TiltBrush™ (Mills & Brown, 2022).
2. Virtual Painting: Equipped with head-mounted displays (HMDs) and motion sensors, students immersed themselves in the art of virtual painting. This allowed us to examine the many possibilities of embodied multimodal representation in VR, encompassing whole-body engagement, haptic feedback, and locomotive movement (Mills, Scholes, & Brown, 2022).

An objective of this research phase was to uncover both the opportunities and the limitations of virtual reality as an instrument for translating ideas from conventional forms of expression to the immersive possibilities of virtual painting. Virtual painting, with its immersive and three-dimensional capabilities, offered students a fresh perspective for expressing their ideas. See Figure 1 for an example of how students expressed mood with virtual painting.

Figure 1. An example of student virtual paintings, exploring moods such as ‘happy’ and ‘crazy’.

What made the VR activities significant was the ability to reshape the way students imagined their stories and concepts. No longer limited to two-dimensional representations, they quickly grasped the possibilities of constructing intricate three-dimensional landscapes and placing themselves right at the heart of their narratives.

Additionally, students found the appealing feature of depicting both the exterior and interior of their three-dimensional scenes. For instance, the textured surface of the Trojan horse could be presented as a detailed image. By navigating within that image, they could reveal the objects concealed inside the belly of the horse, previously hidden from the external view (see Figure 2).

Figure 2. A virtually drawn ‘Trojan Horse’, as an example where students could visualise both the interior and exterior of their image.

The consensus among the majority of our students was that this immersive experience greatly nurtured their creativity (Mills & Brown, 2022). Virtual painting in VR not only expanded their horizons but also provided a platform to use their imagination. The three-dimensional canvas of VR has the potential to revolutionise the way we explore and express our ideas, with learning occurring in unexpected ways.

Impact on learning and pedagogy with VR

In essence, virtual reality (VR) is a dynamic, interactive technology that seamlessly responds to the user’s movements, creating a fully immersive experience. Within this engaging virtual domain, lies a many opportunities and challenges when it comes to translating concepts from traditional forms of expression, such as writing and drawing, into the virtual domain. While the immersive and physically interactive nature of VR adds an exciting dimension to learning, it also presents certain hurdles, particularly in capturing fine details. In this digital landscape, there are no tangible surfaces or artificial stimuli to recreate the sensation of pressing against a physical canvas or paper (Mills & Brown, 2022).

As our study has illustrated, VR technologies hold many possibilities in facilitating the transition of information from conventional modes to immersive ones. However, it’s important to understand that this process is not merely a straightforward replication of content. Rather, it is a generative journey where learners must reimagine and reconfigure their knowledge. They must adapt to increasingly complex, multimodal, immersive, and interactive environments that exist in three dimensions, yet remain physically intangible (Mills & Brown, 2022).

VR transcends the boundaries of traditional learning, offering an exciting space where knowledge is not just transferred but transformed. It challenges us to engage with information in innovative ways, encouraging us to navigate a world where the line between the real and virtual blurs, creating many opportunities for engaging learning experiences.

For further details about this phase of our research, please check out Kathy Mills and Alinta Brown’s recent publications, Smart glasses for 3D multimodal composition , Immersive virtual reality (VR) for digital media making: Transmediation is key and Kathy Mills, Laura Scholes and Alinta Brown’s Virtual reality and embodiment in multimodal meaning making.

References

Fernandez, M. (2017). Augmented-virtual reality: How to improve education systems. Higher Learning Research Communications, 7(1), 1–15. https://doi.org/10.18870/hlrc.v7i1.373

Huang, H. M., Rauch, U., & Liaw, S. S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education, 55(3), 1171–1182. https://doi.org/10.1016/j.compedu.2010.05.014

Hussein, M., & Nätterdal, C. (2015). The benefits of virtual reality in education: A comparison study. [Bachelors Thesis]. University of Gothenburg. http://hdl.handle.net/2077/39977

Mills, K. A., & Brown, A. (2022). Immersive virtual reality (VR) for digital media making: Transmediation is key. Learning, Media and Technology, 47(2), 179–200. https://doi.org/10.1080/17439884.2021.1952428

Mills, K. A., & Brown, A. (2023). Smart glasses for 3D multimodal composition. Learning Media & Technology, 1-22. [Published first online] http://dx.doi.org/10.1080/17439884.2023.2207142

Mills, K. A., Scholes, L. & Brown, A. (2022). Virtual reality and embodiment in multimodal meaning making. Written Communication, 39(3), 335–369. https://doi.org/10.1177/07410883221083517

Steed, A., Pan, Y., Watson, Z., & Slater, M. (2018). “’We wait’—The impact of character responsiveness and self embodiment on presence and interest in an immersive news experience. Frontiers In Robotics and AI, 5: 112. https://doi.org/10.3389/frobt.2018.00112