Smart glasses for 3D multimodal composition
by Kathy A. Mills, Alinta Brown.
Extended reality technologies – mixed, augmented, and virtual reality, and future-related technologies – are rapidly expanding in many fields, with underexplored potentials for multimodal composition in digital media environments. This research generates new knowledge about the novel wearable technology – smart glasses – to support elementary students’ multimodal story authoring with 3D virtual objects or holograms. The researchers and teachers implemented learning experiences with upper elementary students from three classrooms to compose and illustrate written narratives before retelling the story with Microsoft HoloLens 2 smart glasses, selecting 3D holograms to illustrate the settings, characters, and events from the 3D Viewer software. The findings analyse how smart glasses supported students’ multimodal composition, and relatedly, the new modal resources available to students wearing smart glasses to compose 3D stories. The findings have significance for educators and researchers to understand and utilise the multimodal affordances of augmented and mixed reality environments for composing and storytelling.
Virtual Reality and Embodiment in Multimodal Meaning Making
Immersive virtual reality (VR) technology is becoming widespread in education, yet research of VR technologies for students’ multimodal communication is an emerging area of research in writing and literacies scholarship. Likewise, the significance of new ways of embodied meaning making in VR environments is undertheorized—a gap that requires attention given the potential for broadened use of the sensorium in multimodal language and literacy learning.
Literacy for Digital Futures. Mind, Body, Text
by Kathy A. Mills, Len Unsworth, and Laura Scholes.
Topics in chapters 1, 5-7, and 11 are related to research conducted through this Sensory Orchestration for Multimodal Literacy Learning in Primary Education project」, and list these chapter titles, as per below:
- Beyond Education for Industry 4.0: Next-Generation Literacies
- Embodiment and Digital Media Literacies
- Haptics and Motion in Literacy Practices with Digital Media
- Virtual, Augmented and Mixed Reality
- Conclusion: Multimaterial Literacies for Digital Futures
The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualized, embodied, multimodal, and digitally mediated.
In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes—Mind and Materiality; Body and Senses; and Texts and Digital Semiotics—to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice.
In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes—Mind and Materiality; Body and Senses; and Texts and Digital Semiotics—to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice.」
Sensing place: Embodiment, sensoriality, kinesis, and children behind the camera
by Mills, K., Comber, B. & Kelly, P. (2013). Sensing place: Embodiment, sensoriality, kinesis, and children behind the camera. English Teaching,12(2), 11-27. New Zealand: University of Waikato.
This article is a call to literacy teachers and researchers to embrace the possibility of attending more consciously to the senses in digital media production. Literacy practices do not occur only in the mind, but involve the sensoriality, embodiment, co-presence, and movement of bodies. This paper theorises the sensorial and embodied dimension of children’s filmmaking about place in two communities in Australia. The films were created by pre-teen Indigenous and non-Indigenous children in Logan, Queensland, and by Indigenous teenagers at the Warralong campus of the Strelley Community School in remote Western Australia. The films were created through engagement in cross-curricular units that sensitised the students’ experience of local places, gathering corporeal information through their sensing bodies as they interacted with the local ecology. The analysis highlights how the sensorial and bodily nature of literacy practice through documentary filmmaking was central to the children’s formation and representation of knowledge, because knowledge and literacy practices are not only acquired through the mind, but are also reliant on embodiment, sensoriality, co-presence, and kinesics of the body in place.
Immersive virtual reality (VR) for digital media making: transmediation is key
by Mills, K., Brown, A. (2021)
The rapid evolution of virtual reality (VR) technologies and their adoption for learning opens up new possibilities for shifting semiotic content across modes, with underexplored scope for transmediating content in visual, haptic, and auditory ways in immersive media literacy practices. This research investigated users’ creative digital designing involving a popular, three-dimensional virtual painting program with upper elementary students who used a VR head mounted display and sensors. The analysis attended to how students transmediated the same story across written, verbal, and virtual painting modes, tracing key themes of the students’ virtual experience: (i) immersion and three-dimensionality, (ii) subjective presence, (iii) sensory illusion, and (iv) interactivity with motion tracking. Students reassembled and shifted narrative content, sometimes seamlessly, while experiencing ambiguity and complexity about three-dimensional representation in an immersive world. Producing stories across modes invited adaptation and generative thinking to overcome the lack of equivalence between writing, drawing, and virtual painting modes.
Towards a typology of touch in multisensory makerspaces
by Friend, L. & Mills, K.
This research examined the role of touch in creative media production in the context of educational and community makerspaces. Touch, while only recently explored in digital media production, is a crucial perceptive sense through which to experience the world, particularly in two- and three-dimensional making, and to explore texture, temperature, and vibration. As an embodied experience through the hands, fingers, and other body parts, touch affords knowledge and agency. This paper describes research that investigated how students, ages 8–13, used touch to make media. The findings illustrate how different touch types—explorative, creative, auxiliary, evocative, orchestrated, and transformative – emerged as central to the students’ media practices for making products. These findings are important given the recent applications of embodiment theory and its relevance to creative digital media making in education and society.
Chapter “Big Data in Qualitative Research Methods” in SAGE Research Methods
by Kathy A. Mills
Big data has emerged in an era of heightened digital change, referred to in the literature and research since the turn of the twenty–first century. Big data refers to rapidly produced, digitally encoded information of variety, velocity, volume, value, and veracity (5 V’s) that is used as evidence for a phenomenon, commonly having relationality within data networks (Mills, 2019). While increasingly relevant to qualitative, quantitative, and mixed methods research, this article discusses big data in qualitative research—a field in which a large variety of nonstatistical data have always been central. Here, the meaning and significance of big data is established, providing examples for qualitative and mixed methods research. A brief history of big data is described, while identifying some of the key potentials and challenges of big data in qualitative research.
Chapter 13 “The digital mediation of emotion in late modernity”, in Emotions in Late Modernity
This international collection discusses how the individualised, reflexive, late modern era has changed the way we experience and act on our emotions. Divided into four sections that include studies ranging across multiple continents and centuries, it is the first collection to investigate the changing nature of emotional experience in contemporary times.
Chapter 7 Digital literacies in Literacy Theories for the Digital Age
Forward by David Howes and Chapter 7 Sensory Literacies
by Kathy A. Mills
Literacy Theories for the Digital Age insightfully brings together six essential approaches to literacy research and educational practice. The book provides powerful and accessible theories for readers, including Socio-cultural, Critical, Multimodal, Socio-spatial, Socio-material and Sensory Literacies. The brand new Sensory Literacies approach is an original and visionary contribution to the field, coupled with a provocative foreword from leading sensory anthropologist David Howes. This dynamic collection explores a legacy of literacy research while showing the relationships between each paradigm, highlighting their complementarity and distinctions. This highly relevant compendium will inspire researchers and teachers to explore new frontiers of thought and practice in times of diversity and technological change.
Handbook of Writing, Literacies, and Education in Digital Cultures
by Kathy A. Mills, Amy Stornaiuolo, Anna Smith, Jessica Zacher Pandya
At the forefront of current digital literacy studies in education, this handbook uniquely systematizes emerging interdisciplinary themes, new knowledge, and insightful theoretical contributions to the field. Written by well-known scholars from around the world, it closely attends to the digitalization of writing and literacies that is transforming daily life and education.
Conferences and Keynotes
Future Directions for Critical literacy for CritLit Research Symposium
Mills presented an invited keynote called Future Directions for Critical literacy for CritLit Research Symposium, University of South-Eastern Norway, Sept 23, Tonsberg, Norway.
Mexico, SIELL Conference
For colleagues in Guadalajara, Mexico, Professor Kathy Mills was invited to present from the Senses Together project on The Future of the Digital Turn in the New Literacy Studies at del Instituto Transdisciplinar en Literacidad
At the University Centre of Art, Architecture and Design (CUAAD) on June 12th, 2023. The keynote was followed by a lively discussion about the challenges of being truly literate the digital age
University College London, Knowledge Lab and Research in Media Arts and Play
Professor Mills was invited to present from the Senses Together project on Smart Glasses for 3D Multimodal Design, Knowledge Lab in London, UK on June 28th, 2023. This event was well-attended, both in person and online, and was generously organised by Professor John Potter, the Director of ReMap.
Sheffield Hallam University, UK
In the UK, Professor Kathy Mills also presented from the Senses Together project on Using Mixed Reality for 3D Multimodal Literacy to the Language and Literacy in Education department at Sheffield Hallam University on June 22nd, 2023. The event was followed by a networking dinner and sparked new questions about extended reality technologies in early childhood and primary education in a digital age
ALEA Digital Synergy Symposium: Language, Literature and Literacy in a Digital Age, Queensland
Professor Kathy Mills was invited to present a Keynote on Smart glasses for 3D Multimodal Composition at the ALEA Digital Synergy Symposium: Language, Literature and Literacy in a Digital Age on Saturday 13th May, 2023, University of Southern Queensland, Springfield.
Contemporary Humanism International PhD Program Seminar, Italy
Professor Kathy Mills presented at the Contemporary Humanism International PhD Program Seminar hosted by the University of Rome, LUMSA, Italy, on Wednesday 22nd February, 2022.
Her virtual presentation shared from her research was entitled: The Future of the Digital Turn in the New Literacy Studies.
ALEA/AATE 2022 National Conference, Darwin
Professor Kathy Mills presented at the ALEA 2022 Conference Darwin, on Friday July 8th Her virtual presentation was shared on Day 1 from her ARC Future Fellowship was entitled: Virtual Reality Technology for indigenous Heritage Media.
Digital Reading and Writing Symposium
We hosted the “Digital Reading and Writing Symposium” and think tank to address key issues of digital change for literacy education, held at ILSTE on Thursday 14th July. This whole day event was enthusiastically attended by Department of Education, QLD staff, Big Picture Industries Inc, and academics from multiple ACU campuses nationally.
Digitality and Normativity in Informal and Formal Learning Contexts, Research Exchange Day, Germany.
Mills, K.A. (2022, March 31) Keynote Speaker: What is the Future of the Digital Turn in the New Literacy Studies?
Digital Symposium: Friedrich-Alexander-University, Erlangen-Nurnberg.
Multimodal Research International Conference, Auckland, New Zealand
Mills, K. A. (2019, December). Transmediation: Story writing to multimodal design in in virtual reality. Multimodal Research International Conference, Auckland, New Zealand.
Symposium conducted at the meeting of American Educational Research Association, Toronto, Canada
Mills, K. A. (2019, April). Transmediation in children’s virtual reality (VR) art. In D. Low (Chair). Symposium conducted at the meeting of American Educational Research Association, Toronto, Canada.
Critical Digital Literacies in a Sensory World. USN, Norway
Mills, K.A. Keynote Speaker (2020, Sept). Critical Digital Literacies in a Sensory World. USN, Norway.
Transmediation using Virtual Reality Technology, Cyprus
Mills, K.A. Keynote Speaker (2021, March 12-13). Transmediation using Virtual Reality Technology. 4th International Conference Literacy and Contemporary Society, Cyprus.
AATE/ALEA National Conference, Challenge & Change: Contemporary Literacy & English Teaching, Brisbane.
Mills, K.A. Keynote Speaker (2021, July 7-9) . The Future of the Digital Turn in the New Literacy Studies. AATE/ALEA National Conference, Challenge & Change: Contemporary Literacy & English Teaching, Brisbane.
Australian Literacy Educators Association
Mills, K.A. Keynote Speaker (2020, March). The Role of the Senses in Multimodal Design: Stories from Culturally Diverse Classrooms, Australian Literacy Educators Association, June 29.