Spatial Cognition — Making Virtual Spaces
As promised in the previous article, here’s the first practical writing about one of the most powerful features of cognition, namely “Spatial Cognition” and how it can leverage VR contents. As a bonus I will throw some concrete real life examples too. (IRL Alert!)
What is Spatial Cognition ?!
it’s concerned with the acquisition, organization, and utilization of knowledge about spatial objects and environments; be it real, virtual, or abstract, human or machine.
Put aside its the definition, in practice it covers many aspects of your understanding of your surrounding world. Any disruption in this form of cognition can lead to feeling lost, losing your balance , maybe not even being able to stand still or reach your coffee on the desk and many other dark consequences which I can’t even imagine.
To be honest, you can’t comprehend and inhabit any space without having the perception toolkit that helps you measure it and actively define it. It’s not an accident that you can’t find any single 3D software without it having implemented a coordinate system (Cartesian, spherical, etc). If you ask me, it should be the first thing any programmer of a 3D software codes.
To make it easier to analyze, cognition can be categorized into two fundamentally different modes: online and offline. However having a dualist approach has always been criticized but reduction is still a helpful entry point for understanding complex concepts.
Online Spatial Cognition
Concerned with “immediate input” from our local environment and deals with “here-and-now” tasks that require fast moment-by-moment processing. It’s always active and you can’t shut this down my friend!
Spatial mapping constantly tries to make sense of space it perceives through your senses and bodily experience in the world as a result of moving in space. This means how/what you receive from your sensory system and other characteristics of your body deeply affects your perception of space.
So of course, there’s a huge difference between how you, your dog and the annoying mosquito perceives your room.
Offline Spatial Cognition
Involves the long-term encoding, storage, and retrieval of spatial information and enable complex behaviors such as navigation, route planning, and direction giving. It’s a slower, high-level form of cognition that we switch to make more careful considerations, like when we make a mental check on something odd or plan future behavior.
Now things gets exciting when you realize that Spatial Cognition not only helps us to navigate and find ourselves in space, it even uses space as a bedrock to offload and pin information/memories for later use! It’s one of my beloved features of the mind.
🏩
^^ Did you know there’s a spatial-love-emoji already called Love Hotel ?!
Real Life Example of Spatial Cognition
Once we get the hang of space through our online spatial updating, then it becomes more comfortable to store/offload our thoughts into the perceived spaces.
There’s a huge difference between these two experiences and the mental process that occurs accordingly:
1- When you visit your friend’s house for the first time 🏡 your spatial cognitive load mostly consists of making sense of the new space and mapping the new house in your mind. Where the toilet is ? which areas I’m allowed to navigate in the house ? And many other basic info that you just automatically collect.
2- When you visit your friend’s house for the second time 🏡 you almost know a lot about it that if it was possible to document your thoughts about that house it could be a thick book that you’d never dare to read. And you’d confront and get reminded of the thoughts and memories of your last time visiting there this time. As you might subconsciously pinned/offloaded some of your thoughts there in some corners before.
It’s almost surprising how much presumably useless information we collect and at the same time offload and pin to spaces around us.
To push the example a bit more further, when you pass from your friend’s house while driving in the city, 🏡 again all those info you collected somewhat kicks in for fraction of seconds and feelings arise off of that. Where as this mental state couldn’t be triggered if you were driving from another path and didn’t pass that familiar house.
Did you just Say memory palace ?! 🏰
So if you haven’t heard of this concept, memory palace is a technique to create memories through consciously imagining a familiar place and starting to pin the chunked information around the imagined space (for example your house). The process of remembering is simply re-imagining a visit to that space where you saved memories.
This technique is fun and pretty straightforward, but I brought this up to recall the power and quality of spatial cognition and the fact that, it’s already happening subconsciously by your mind. So If you ask me, I would say your perceived world is already your memory palace! ⛫
How do we perceive Virtual Spaces?!
In terms of spatial cognition, dealing with virtual realities is somewhat confusing! Because in essence they disrupt our established logic of space mapping which we actively apply to the real life. It works well in real life because it’s a consistent reality. Space around you is merely static and seamless. The park near your house remains there, it won’t move, or gets replaced by a parking lot in a second! In contrast, virtual spaces are fluid and dynamic in a sense that anything can potentially happen to them. And they can be so disruptive that your mind wouldn’t be able to map them.
In my VR project False Mirror, one of the spaces I made is a minimal but huge hallway with one big chain of metal that is like a toy that you can touch and play with it’s satisfying physics in VR.
When I made the space this way (vertically tall), It felt great being in it but after some months in the development I decided to test iterations on this space. And what I did was super simple, I just rotated the whole space 90 degrees and made it horizontal. It bent my mind! Because the space that I was used to playing inside it for months now is different although the change was subtle but it almost gives a radically different vibe and feeling compared to what it used to be.
In the GIFs below, It’s obvious how this change in the space overhauls the feelings, and encourages different behaviors.
Creating virtual spaces, in terms of design is not too far from architecture as we know it in the real life. You’d need to consider lots of aspects that might seem unrelated but they actually are relevant! For instance, it really matters if the space that you make is supposed to be experienced by two persons simultaneously or maximum of ten or just a single user.
Talking about virtual architecture is interesting and still being explored. I’ll get back to this once I grab better matching examples.
PS. Don’t be surprised if in the near future you see a job posting of a tech company titled:
🔥 Virtual Architect vacancy — Apply now 🔥
The Problem of Spatial Confusion in VR
In fact, spaces in virtual reality are not connected to each other, they might be, in a specific VR experience but not in general. You try the experience A and then try to experience B and there’s no absolute connection between the two both spatially and temporally. That’s because firstly we’re not in the state of the Matrix-like Metaverse yet. And secondly we’re also in the middle of platform fights where every major VR hardware company tries to nail their own platform and the ecosystem of virtual contents remain shattered and disconnected from one another at least for a long time I believe..
So in this mess of virtual spaces, what happens to our good old system of spatial mapping… How confused is it already ?
Once you put your headset-on to experience a wild alienated surreal planet you get tired and then “exit” from the menus to load another experience which is just sitting in a theater and watching youtube! This inconsistency between spaces is distractive and sometimes overwhelming because it counters our natural perception of moving between spaces.
But before talking about ways to workaround this issue, It’s best to discuss user movement in VR first:
Moving in Space (Locomotion)
Space is being understood through movement. You wouldn’t remain in the hospital that you’re born in. From that moment until now you’ve been constantly moving in space.
This becomes really important when we talk about spatial cognition in VR, simply because you decide the type of locomotion your users are going to have in VR and also the amount of agency and control you give them to have mobility. I found an amazing paper breaking down VR locomotion and proposed a nice typology.
So according to the figure above, I’m going to discuss the effect of VR Motion type on spatial mental mapping process of the user.
1- Continuous: In case of going to a new place in VR, having smooth locomotion to navigate the space is definitely natural and helps your online spatial mapping to work as intended. As there’s no interruptions in between movements your mind doesn’t have to interpolate and take extra effort to connect gaps. Because you’re having a consistent stream of spatial information.
But the downside of having this type of locomotion is motion sickness! It can easily put you off balance if you move a bit fast or in a weird orientation.While there are many tricks to lower the amount of sickness, it’s still a tough challenge to effectively mitigate it.
Another cons of these locomotion is there’s a wide variety of types in this ground and you remain confused until you learn how to use the navigation method. So the learning curve can be steep and confusing for newbie users.
In Echo Arena you have another kind of continuous locomotion which is super fun, but it took me an hour or two to just get used to the method of surfing in space. Which many people can give up going through learning it!
2- Non-Continuous: the most famous and widely used type of locomotion is Teleportation. It’s easy to use and pretty straight forward. Wouldn’t make you sick at all. But instead has a problem with spatial mapping. As there’s no smooth transition between your location and your destination, your mind needs to interpolate information of the place you’ve been to and figure out its relations with your destination. This means extra cognitive effort and sometimes you can’t figure out and maintain a good sense of space because there has been lots of differences between point A and B so your mind gets confused and you’d need to hysterically look around (especially your back) to find out that your still in the same place.
Of course this is not natural but again there are some tricks to tone it down.
I use teleportation as the main locomotion solution for False Mirror, however I tried the continuous method too, where you can embody a tiny spaceship and fly seamlessly in space. Check out this breakdown video.
Workarounds to avoid spatial confusion
- Design
Do our best to design and optimize how we introduce virtual spaces inside our own contents and mitigate confusions to provide virtual environments that gets perceived better and more naturally by the mind and body of our users.
Being sensitive to the virtual architecture design and locomotion system are the key elements in this extent. Having known that, when someone is experiencing our reality, they naturally as human beings want to actively map and get sense of the new space. If we constantly disrupt this mental process of mind without being aware of, it leads to feeling lost, anxiety might kick in, and eventually it disconnects the user from the virtual space and breaks immersion.
- User adaptation
In another long-term perspective, it also worth mentioning that as humans we’re great at adapting to new situations. We evolve and get used to confusions too. So who knows our mind and bodies might figure out how to survive the spatial confusion of virtualities just over time . As my own body has got really adapted to motion sickness after these years of actively working on/in VR. I hardly get nauseous by the wildest VR trips now. So this is what we have to wait for and see!
- Hardware
A portion of these confusions relies on limitations of the hardwares too. The limited field of view (110 degrees for high-end VRs), lack of high-accuracy spatial audio or full body haptics and many other factors reduce our stream of spatial information. But still content can overshadow hardware limitations in great extent.
Provoking High-level Cognition through space (Daydreaming)
I know the title is weird! But let’s clear this out. How many times in VR have you ended up daydreaming and giving up your conscious attention from the story, activity or whatever that’s happening ? I bet it hasn’t happened yet or if so, it’s so rare right ?
Well, the main reason for this lack-of-room-to-daydream in VR, in my opinion comes from the thirst of grabbing the user’s attention among all of us VR creators! Can we even consider letting people go with their minds ?
I think this is what needs to be embraced more and as a practice to provide such spaces and moments in VR, I’d give another IRL example:
The city I live in (Tehran) is full of highways and I really enjoy driving aimlessly through them late at nights. The main reason is on the movement I so intensely daydream and somewhat get myself dived into stream of thoughts I couldn’t get exposed to at home. So I asked myself why does that happen in highways and not on the streets ? Is it something to do with the speed of my movement or the space that I’m moving through ?
Of course Highways are very different than streets and the main difference in my opinion is in the repetitiveness of them. Highways are simple overall having less contrast in shape, details and mostly repetitive. While streets are striving for adventures, stories, and full of details.
This means moving through highways takes less of your online spatial mapping and opens more room for high-level offloaded thinking.
My point is, if you ever wanted to provoke or encourage your users in a virtual space to daydream, like I do in the highways, first believe that you can! And second look for spaces and situations in real life that provokes the same feeling and try to replicate by your own design/aesthetic. (Lessons learnt from IRL)
Among all the various spaces I made in False Mirror. One of them has this quality (at least worked on me) and it was surprising because I remember hanging out in this space for many hours doing nothing but daydreaming in VR!
Phew, this was long and I only talked about a tiny portion of spatial cognition. I apologize if some topics that I discussed didn’t morph and blend nicely together. 🙈
These were the main points that caught my mind while writing this article. I will share more examples and thoughts on this topic so consider this as “part one”.
Oh and for the last real life tip, if you fear the anxiety from moving into a new place, try spending more time on random walks around your neighborhood!
Follow-up suggestions
If you’re interested to know about this topic from neuroscience and psychology standpoint, this video series by Paul Dudchenko covers it nicely.
In case you don’t know about the technique it’s a good breakdown.
Edited by Niloufar Goudarzi
