TURBULENCE: JAMAIS VU, an IDFA DocLab 2023 selected experience, is the first part of a series of mixed reality works about vestibular migraine, a chronic condition the creator Ben Joseph Andrews lives with, and is an attempt to look at the parallels between his condition and how we make sense of the inputs that create our sense of perception and reality. Interview with directors Ben Joseph Andrews and Emma Roberts (GONDWANA).
Ben: We are a creative duo based in Australia. We’re both interlopers from film originally.
Emma: We started working in XR in 2016 kind of by accident. Ben was experimenting on something and he asked me to come and help, but we were really immediately taken by how “un-film” XR was.
Coming from a film background where often you’re creating in a bubble, the directness of VR was really attractive. And then there are also the spatial considerations in XR, the ways that you can bring in space and the body into VR is something that is both like a screen and a blindfold was something that we immediately jumped into.
Ben: It felt organic. Before I moved into VR, the kind of film I was interested in was more immersive, experimental, non-narrative, exploring sensorial immediacy of what film can do. When we started working in VR, it just felt natural and what also felt very obvious to us was that it was less about the linear playback of something and more about the capacity to shape the multi-sensorial atmosphere around an audience and to be able to deliver immediacy that is both, direct but also in some respects, because of the blindfolded nature of VR, unknowable and that liminal in-between experience. I think for us this is VR’s richest potential.
What is Vestibular Migraine?
Ben: I’m diagnosed with a condition called vestibular migraine and while we were finishing our last VR piece, GONDWANA, I was having for months difficulty standing up, let alone putting on a headset because of the intensity and velocity of dizziness and vertigo and misbalance. It became evident to me that VR was a very hostile medium for me and Gondwana would be the last VR piece I created.
“With a 24-hour running time, GONDWANA is by nature a very organic experience” – Ben Joseph Andrews, Emma Roberts (GONDWANA)
Ben: So, a typical migraine that you probably might be most aware of is the pain dominant and episodic migraine, the really bad headache version––to put it simply. But migraine is a condition that affects the processing of information from multiple systems of the brain and it upsets them in various ways. Pain is just one manifestation of that in classical migraines.
There’s a whole continuum of migraine classes where pain is a secondary symptom or not a symptom at all. Vestibular migraine is essentially upsetting the vestibular system, which is responsible for balanced orientation, your sense of movement and sense of equilibrium. The symptoms that get expressed through vestibular migraine are balance, dizziness, the feeling of moving in ways that you are not actually moving. It’s a conflict between your sense of status and this dynamic sense of movement that is happening within you. Vestibular migraine is always 24/7 which is one of the big differences. It happens continuously but at different intensities and sometimes flares up.
Ben: Essentially, how I describe vestibular migraine is it’s kind of like synesthesia for motion. Where synesthesia is normally you hear colors and see sounds. With vestibular migraine, everything that I hear, see, smell, touch, my body interprets as movement. So I will see a tree bending in the wind and I will feel like I’m bending with its branches or rippling like its leaves. I can hear the murmurations of a crowd and there will be a phantom version of that movement that my body experiences and absorbs.
It’s something that is incredibly debilitating, but there is also a beauty in it. There’s something poetic in being able to become the rhythms of the world.
The Beginning of TURBULENCE
Ben: Since completing Gondwana, we started to explore this condition, vestibular migraine, creatively around this notion of an emergent aesthetic of motion misperception. The capacity for us to misperceive motion, which is often called vertigo or dizziness, where it is an illusion of motion that isn’t happening, but we think it is happening.
We started to develop the creative research behind all of this, and started to realize the eerie parallels between the VR headset as this device that is designed to mimic and mediate perceptions.
Emma: And I don’t think we talk about that often. With VR, we usually talk about the visuals and sounds and things like that. But it’s a vestibular device. It is a device that creates its sense of reality by knowing exactly where it is and you are in space. That became really interesting to dive into with this project looking at the headset and the technologies that surround it as both immersion in VR and also our own immersion in the physical world.
Ben: This is the first project that I’ve developed myself and that was really important in the creation of the work because it exposed me to the uncanny similarities between the headset and its attempt at perceiving movement, to know where we are in space, even to refresh the images. It was eye opening to see almost inside my own neurology where the headset became this strange mirror to how movement was happening in real time inside my own brain.
There is a parallel between this flow of data from the headset and our inner ear sending millions of signals to know where we’ve moved and how we’ve moved. This experience to look at this data in the game engine allowed us to look at this data as plastic, that can move and behave in different ways than just its usual one-to-one relationship. This became a powerful metaphor for how my migraine and the activity of migraine disrupts the regular flow of information. My whole fascination with VR has become renewed in a very different way.
TURBULENCE, the experience
Emma: TURBULENCE: JAMAIS VU is based on an experience that Ben has when a big flare up is about to happen. It’s kind of like this first sign that something is awry. And that is a sense of depersonalization or derealization, which jamais vu itself is the opposite of déja vu. When déja vu is having a sense of familiarity where you shouldn’t, jamais vu is feeling like you’ve lost familiarity with things that should have that sensation.
Emma: So in TURBULENCE: JAMAIS VU, it’s using a pass-through on a camera mounted on the front of the headset to emulate this feeling of losing familiarity. You’re sitting at a physical desk and the desk is covered with the paraphernalia that one might expect to be on a desk. Then you put on the headset. There is a small moment of grounding where you have your eyes closed and are just listening. Then the pass-through camera turns on and you are able to explore the desk as if it is your own.
Emma: The catch is that the feed from the camera steadily becomes more and more distorted as you progress through the experience. We were really interested in that particular feeling of pass-through where you know you’re seeing the world around you, but it doesn’t quite feel right. It has this feeling as if you’re performing in someone else’s home movie.
Emma: And that’s the parallel that we were really interested in between Ben’s experience of depersonalization and derealization and pass-through mixed reality could offer––knowing that this is your own body, not an avatar. You’re sitting at a desk in the real world, but it is made strange with the technology that we are using. The experience plays out over a series of simple tasks or prompts that we give you and become deliberately very difficult to achieve.
Emma: We didn’t want to create just a simulation with this piece. We wanted it to be a reflection on the technology that is used in the work and make you think about the role of the headset and the constructed nature of our own perception in creating reality.
Emma: TURBULENCE: JAMAIS VU is the first of three pieces we are creating, each of which deconstruct how the headset makes sense of movement and space and how our own bodies make sense of movement and space through using vestibular migraine as a jumping off point.
TURBULENCE: JAMAIS VU won the IDFA DocLab Award for Immersive Non-Fiction 2023, and the IDFA DocLab Forum Award for Best Project 2023.
Leave a Reply
You must be logged in to post a comment.