Narrativity in Mobile Computing with Augmented Reality
Introduction
It is commonly accepted that computer games constitute a distinct phenomenological category of narrative beyond the classical literary diegetic and mimetic modes. In the classical sense narrative is either told or enacted, and therefore heard or seen; furthermore, classical modes of narrative follow a linear path from start to end. Cybertextual narrative in computer games requires an agent’s wilful participation in the direction and determination of narrative events. The spectator becomes entwined within the story through play and skill development thus experiencing the medium through their own agency within it. My research interests have led me to want to understand more about the processes and determining factors behind the concept of ludological narrative to evaluate the extent to which mobile computing combined with augmented reality has already, or will through future innovation, enhance the existing narrative mode of computer games or whether this emerging medium may constitute a distinct narrative mode of its own.
Background
The Athenian philosopher, Plato, introduces the idea of dual literary modes of storytelling in the third book of Replublic written around the fourth century BCE; in which the characterisation of Socrates explains how fabulists (storytellers) and poets may convey their story through either pure narration:
Is not everything that is said by fabulists or poets a narration of past, present, or future things? Plato, Republic, 394b-394c
Or through imitation (drama, enactment, etc) :
There is one kind of poetry and tale-telling which works wholly through imitation, as you remarked, tragedy and comedy Plato, Republic, 394b-394c
Or through both together:
…and there is again that which employs both, in epic poetry and in many other places Plato, Republic, 394b-394c
These two styles of storytelling described in Republic are commonly referred to as diegetic and mimetic modes in academic literature.
Trending AR VR Articles:
1. How VR could bring transhumanism to the masses
2. How Augmented Reality (AR) is Reshaping the Food Service Industry
3. ExpiCulture — Developing an Original World-Traveling VR Experience
Cybertext
Before continuing, I’d like to add a note about what is cybertext. The term was popularised in Aarseth’s 1997 book — Cybertext: Perspectives on Ergodic Literature. Hypertext was the given term for electronic texts; from which the web page scripting language hypertext markup language (html) is derived. Aarseth suggested that while hypertext was useful for referring to structures of links and nodes but less so if it includes all other digital texts as well (Aarseth p75). Cybertext, is dynamically produced narrative whereas hypertext is static. The dynamic nature of cybertext allows for the creation of ergodic narrative as opposed to linear. While linear narrative always tells the same story, repeatedly. Ergodic, or non-linear narrative allows for a story to unfold in different ways based on an interactive process between the text and the receiver.” When you read from a cybertext,” says Aarseth, “you are constantly reminded of inaccessible strategies and paths not taken, voices not heard” (p3). It is this information feedback loop, which transforms nature of the receiver from spectator to participant.
Writing in 2001, the author and scholar, Marie-Laure Ryan, applies the two modes previously discussed ( diegetic and mimetic) to the genre of cybertext storytelling and identifies three distinct, related genres: hypertext, VR-type environments, and computer games. Ryan describes hypertext narrative in terms of the diagetic mode, just like print novels or short stories [which is why Aarseth doesn’t view hypertext as cybertext at all]. While, Virtual Reality (VR), Ryan observes, is a “standard case of mimetic, or dramatic narrativity” very much like cinema; albeit with a “fusion of the actor and spectator functions”. The latter observation arising from the fact that the same person participates in, and reads as story from, the actions within the virtual world.
It must be said, that since the time when Ryan was writing in 2001, consumer VR has rapidly evolved, especially over the last few years, to include, for example, interactive multi-player experiences. But, overall, these are mainly virtual social spaces without any literary narratology. For the most part, my own experience with VR as of 2021 is still very much aligned with Ryan’s initial concept of VR as mimetic narrativity; usually whereby a narrative is played out by touching the appropriate hotspots in the virtual world to open the next drama scene. A couple of examples of VR apps, I have personally tried, which might challenge this assumption; include titles like Elite Dangerous and Star Trek: Bridge Crew both of which require the participant to forge out their own narrative through interactive processes which involves player agency and therefore goes beyond spectatorship. However, I would strongly argue that such titles are in fact computer games and the VR hardware part of the gamer’s controls. As such, for the purpose of cybertextual narrativity, VR can be thought of in terms of 360 degree immersive storytelling in the same mimetic mode as film, but it can also be used in the same mode as computer games.
For Ryan, computer games are more problematic to position in respect to cybertextual genres; but the concept of computer game narrative should not be ignored, either:
“The inability of literary narratology to account for the experience of games does not mean that we should throw away the concept of narrative in ludology;”
Ryan, 2001
With respect to computer games (including the VR games) the receiver is neither pure spectator nor listener, the medium requires more than a viewer/listener; it requires player agency. Or, as Douglas Brown, notes in his 2015 thesis on suspension of disbelief in computer games, the receiver must actively and wilfully participate or else they are not engaging with the medium at all: “Since generally refusing to use agency is tantamount to not reading the text at all, this fundamental interactivity of gaming makes the gamer something other than an audience.” (Brown, p.12). Brown identifies further important aspects to games such as player-authorship and skill, which are unique to the medium and make it distinct from other types, such as books, picture or film. Ryan concluded that “Games thus embody a virtualized, or potential dramatic narrativity, which itself hinges on the virtual diegetic narrativity of a retelling that may never take place.”; proposing that “we need to expand the catalog of narrative modalities beyond the diegetic and the dramatic, by adding a phenomenological category tailor-made for games.”
So, it is the participatory nature of cybertextual computer-game: the skill, interaction, co-authorship and agency, combined with the virtualized nature of what may or may not take place during the participants decision making and physical interaction that calls for this experience to be thought of in terms of a distinct medium. Whereas the performance of a linear-narrative-reader takes place entirely in their head; the cybertextual process requires additional extranoematic effort. It is also clear from Aarseth’s account that while cybertextuality should not be considered only in terms of computer-driven textuality (for that would surely have future limitations); the concept should focus on the “mechanical organization of the text” (p1).
Mobile Computing Cybertext
“The development of mobile technology, global positioning systems (GPS), and augmented reality counters the tendency of computers to lure sedentary users into virtual worlds by replacing simulated environments with real-world settings and by sending users on a treasure hunt in the physical space”
Ryan, Foot & Azaryahu, 2016, pp102
The arrival of smartphones, which are effectively pocket-sized computers, has introduced new ways to experience a crossover between different types of spaces, places and narrative. In my present research driven project, Noirscape, for example, the narrative is anchored in real space through mobile technology superposed with alternative representations of the associated place using immersive photography and special effects while augmented reality (AR) presents the participant with clues to search for in the storyworld. The smartphone’s user interface and cloud-based features ranging from realtime notifications, GPS, and cameras to onboard machine learning provides the platform for a fusion of cybertext and real-world; where the medium embodies an infinite level of virtualized dramatic narrativity with more intimate participatory authorship -the medium begins to entwine fiction with the participant’s own life, living spaces and geographical locality.
Conclusion
It would be interesting to carry out further research to examine the nature of computer game narrative in an augmented-reality mobile setting whereby geo-temporal real-world environment is added to the existing fusion of spectator and actor. To understand where mobile computing and augmented reality fit within the existing ludological mode of computer game narrative, or to what extent, if any, mobile computing narrative could be considered an emerging genre of cybertext or even a distinct mode of literary narrative.
References & Resources
Berger, Karol. “Diegesis and Mimesis: The Poetic Modes and the Matter of Artistic Presentation.” The Journal of Musicology, vol. 12, no. 4, 1994, pp. 407–433.
Plato. Plato in Twelve Volumes, Vols. 5 & 6 translated by Paul Shorey. Cambridge, MA, Harvard University Press; London, William Heinemann Ltd. 1969.
Ryan, M. L. (2001). “Beyond myth and metaphor: the case of narrative in digital media.” Game Studies Vol. 1, Issue 1. http://gamestudies.org/0101/ryan/ [accessed April 2021]
AARSETH, E. J. (1997). Cybertext: perspectives on ergodic literature. Baltimore, Md, Johns Hopkins University Press.
Juho Hamari, Aqdas Malik, Johannes Koski & Aditya Johri (2018):Uses and Gratifications of Pokémon Go: Why do People Play Mobile Location-BasedAugmented Reality Games?, International Journal of Human-Computer Interaction, DOI:10.1080/10447318.2018.1497115
Ryan, Marie-Laure, et al. Narrating Space/spatializing Narrative : Where Narrative Theory and Geography Meet . The Ohio State University Press, 2016.
Nam, Yanghee. (2014). Designing interactive narratives for mobile augmented reality. Cluster Computing. 18. 309–320. 10.1007/s10586–014–0354–3.
RYAN, Marie-Laure, Kenneth E. FOOTE and Maoz AZARYAHU. 2016. Narrating Space/spatializing Narrative : Where Narrative Theory and Geography Meet Columbus: The Ohio State University Press.
Brown, Douglas. (2016). The suspension of Disbelief in Videogames. 10.13140/RG.2.1.3175.8968.
I’m a full-stack developer & designer based in France. I build apps for a variety of platforms; including web, mobile & XR
Don’t forget to give us your 👏 !
