Rarely has the political and social context justified the presence of a documentary at the heart of festivals and, hopefully, soon available to all. The Polish-Canadian immersive experience THE CHOICE testifies to the difficulty of abortions in the United States. A look at a real social issue that did not wait for the latest Supreme Court twists to speak about it; the XR for impact project is 1000x more relevant today than ever, hoping to contribute to the current debate in a dynamic way. Interview with its creators and founders at Infinite Frame Media.
Infinite Frame Media just won a prize at the Sunny Side of the Doc 2022 festival, Best Immersive Experiences Pitch, for their upcoming project JACOB’S JOURNEY!
How to start as a VR creator (when you work on 3D cinematography)
Tom C. Hall – Our background was in cinema, and 3D cinematography first. I worked for a company in Toronto designing 3D camera rigs. At that time in the late 2000’, we used to work for high quality content for Imax or just basic not-that-good 3D films. We were exploring! A lot of ideas for the 3D cinema were not really practical… In 2012, I met Joanne at some 3D workshops in Dublin. She then came to Canada, and we worked together for a while in a 3D company. As 3D collapsed, Joanne found work at a post-production company in Toronto. They were looking for people to work on a multi-camera rig for a dome/planetarium project dedicated to the 150th anniversary of the Canadian Confederation (SESQUI: HORIZON). We worked on it for 2 years, developing new technologies etc. With Joanne, we started exploring VR at that point, as we realised how big narrative potential it has. We saw the power of XR, with some ideas on how to improve the possibilities of 360 video with more interactive experiences.
Joanne Popińska – When the wave of 3D projects happened, I was still living in Poland. Everyone was fascinated. Everyone tried it. But 3D soon died. I basically arrived in Canada to work for a company that closed its doors 2 weeks after. I worked for a 3D film festival in Poland, and I was hired to scout for new projects in North-America, especially at SIGGRAPH (Vancouver) where I discovered VR with Birdly and a few other experiences. I was blown away! I talked about it with Tom, and he started to work on a VR camera rig, we wanted to do some tests. Our first test film was inside our house with the camera on Tom’s head running everywhere as a cat would do – as he was chasing our cats. And strangely, people responded to it. And they wanted to interact with it!
J. P. – That’s when I understood VR is not just a new format. It’s a way of experiencing things differently for the viewer. In 2016, I was also doing my PhD at Polish National Film School in Łódź. The subject was supposed to be “directing 3D stereoscopic films”, but I changed it to discuss the emergence of the virtual reality medium (“New forms of narration in Virtual Reality”). And right at the same time, the Polish government started attacking women’s rights, abortion debate etc. and it was another reason why VR was important to address the audience in a more direct way. XR is great for impact campaigns obviously, as it speaks directly to the viewer. This is the origin story for us, and maybe the first step to THE CHOICE.
THE CHOICE, a “no-money” production
J. P. – THE CHOICE was made without any outside funding.. We define this experience by who we are. I am Polish (now Polish-Canadian), Tom is Canadian, so it’s a Polish-Canadian production. Our team is international: we have a Swiss VR artist, Canadian composer, American sound designers and Unity developers. When we started to work on it, there was no real system to support it financially. Today is different, in Canada you have many ways to get support for interactive narrative pieces – outside for games. The challenge was, at that time, to produce a bigger narrative project. We explained our vision, the difference between interactive and gaming, etc.
T. C. H. – We approached broadcasting companies who were starting to give some money to VR productions, but regarding our topic they advised us to start with a flat documentary – and to do VR on the side for fun. No one understood our will to produce a 6DOF project on abortion and testimonies from women, but going straight to VR and not television or cinema. That wasn’t our idea. And no doors opened for us, so we decided to do everything on our own. Shooting, script, post-production… And with our own money.
J. P. – Then we decided to launch a Kickstarter campaign (link) and, being really naive, to ask for 10-15K for the project. It wasn’t enough, obviously! It was an international campaign for a kick and easy project. But finally, we spent 2 years just developing the tech to shoot it. The end goal was to go full 6DOF, no 360 video. We needed the immersive aspect of it. We played with volumetric capture to feel the presence of these women, in the same room. I needed physical distance and/or closeness. But it was very expensive to go and produce it with a big studio in the US – and more importantly, it felt more like holograms and ghostly characters. Not really what we were looking for in terms of live interactions. The topic is so difficult, you can’t miss it because the experience is not completely immersive. We decided to combine stereoscopic and volume, and it was a good fit for us.
T. C. H. – When you watch 3D films such as PINA (Wim Wenders, 2011) where dancers are looking at you, you feel it in your soul. For THE CHOICE, we needed our characters to be present, an intensity of feelings also. Eye rivalry (when one eye sees something different from the other) is an important cue for us, to understand how to set up everything. When you combine both eyes, you complete the picture – but you need to work on both, especially on a 3D shooting. And that’s even more difficult when you work around human faces, as they have a lot of details (skin, hair…). In a 6DOF environment, the depth of the background, disparity, parallax… can work perfectly fine. But if you add a human face, it’s flat, it becomes difficult to display it in such an environment. So we worked on this for a very long time.
T. C. H. – Take a character crying. On a flat screen, it’s ok. In a more sophisticated space, it’s just a white dot. A pixel. It doesn’t communicate a real emotion. You need to pick up this, and to create it in a volumetric stereoscopic space.
Making a branching narrative piece: THE CHOICE
J. P. – We wanted to engage the user on a controversial subject, to say the least. For that kind of story, you need to think how to get the user to listen – not just being pushed from one sequence to the other. Ok, it’s about giving a “choice” to the user in THE CHOICE – this is the irony! I see it as a conversation between us and our audience. They can choose where to go, what to do. In Poland, the debate about reproductive rights is heavy. Some of my friends are anti-choice! I had to think about this, take a step back to build our story the way we can discuss things, open doors, let people make their own conclusions, without us explaining. If you don’t do that, you end up yelling at each other with no results.
J. P. – This is our real priority. Anti-choice people may have ethical, religious reasons to believe the way they do. It’s our mission to invite people to explore the issue. Especially with an immersive experience, we needed to share ideas without pushing them to the other side. That way we can change our minds. I had that kind of debate with my friends – and even my family. The interactive story is built for this, but we were careful not to lose anyone. We tested a lot of things for this, including A.I. But branching narrative doesn’t need to be complex to work fine.
J. P. – To understand our main protagonist’s testimony, you need to build emotional engagement. It’s the same thing for movies: you need narrative arcs. To build the story. We had to work around interactivity to avoid the interactions becoming too self-important. And, at the same time, I need to organise everything to take the user where I want him to be! It was a journey between the narrative, the interactive and the point of view.
T. C. H. – For us, storytelling is about the storyteller. There is an agenda around the experience, and he conducts everything. THE CHOICE has a powerful story, and going for a branching narrative story we had to keep it in mind. So it’s as simple as it can be, the interactions serving the story. Not the opposite. We stayed focused! It’s really important to give more power to the user, and it gives immersion more vivacity.
Drawing an immersive environment
J. P. – We had to think about the 360 environment, and how it can support the story. And not the opposite, again. It’s sensitive, because it can’t be overwhelming. This goes back to a study by Stanford University. They were testing how people react to a 360 environment and what they can see, how to retain information. In a 90-degree story, they remember all the facts and information. But they don’t get emotionally engaged, connected. This emotional connection occurs on a deeper level in a 360-degree environment. But there is another challenge. In a 360-degree story, it was too noisy, too much info. We’re used to a cinema screen, but with VR your brain is too busy. It’s a concept from neuroscience called the “Detective mode” where you look at everything but you don’t keep a clue.
J. P. – So, how to combine and use these two phenomenons: that in a 90-degree story people are focused on the story, but they need a 360-sphere to understand it on an emotional level, and immerse themselves subconsciously? In a 360-volume (6DOF), people started to get the story on this emotional level. They are involved. They felt for the main character, etc. For THE CHOICE, we decided it was essential to be focused on the storyteller, to know her story, but to also connect with her on this deep, emotional level. We came up with the idea of combining the volumetric capture of a person with a minimalist illustration to immerse you in the world of her thoughts and dreams. And we used audio as well to build this space! But watching a lot of VR content during production, we knew the visual and the audio needed to work together – and not the visual to take over the narration.
T. C. H. – In fact Zoe, our illustrator, did everything on Quill! She told us her work is supposed to be the background orchestra of the story, and not the soloist. Except for key moments of course, it backs up the story.
J. P. – A lot of people think that VR is a big empathy machine, walking in somebody else’s shoes, etc. But we didn’t do THE CHOICE for that reason. We did it to give the protagonist a voice, and the subject a reality. It may be difficult or intense depending on the sequence. We want you to see this and understand. It’s all about the testimony, meeting someone who went through all this. It’s realistic enough to get it. And then, the most important aspect probably, is that the audience enters this world with they own identity, self-awareness. They take part in this conversation. And when they take the headset off, it stays with them, for a long time.