Did you think we would stop at two? Think again! Because the interview with Ferryman Collective at Venice Immersive 2024 deserved a third (but final) part!
Together, let’s delve into the last three main challenges faced by those working in live immersive performances: intimacy in the context of scaling, the love/hate relationship with artificial intelligence, and overcoming the physical and mental limits imposed by today’s available technological tools.
NB: … brief recap for those who missed it: the interview involved theatrical director Stephen Butchko and artistic director Rick Treweek (MetaRick); Deirdre V. Lyons, producer and assistant director with Whitton Frank, producer, and Brendan Bradley (themselves directors of extraordinary pieces of immersive reality who also perform in the show), and Christopher Lane Davis (AKA Screaming Color), who creates visually stunning animations that complement the world MetaRick created (the show’s lead designer and visual artist) as well as the music and sound manager.
Challenge 6: Creating intimacy in a scalability-driven industry
“Our shows have quite a limited audience in a sense. But then, for example, we made a little promo world for UNCANNY ALLEY, and it has now been viewed 27,000 times. So there’s a huge audience out there. There are millions of headsets used by people that know our world, know our characters and want to find more about these things, whereas at festivals we are still limited to three, four people in a show. How do you bring this together? How do we merge the existing audience with the professional outlook of festivals like the Venice one?”
One of the most debated issues among creators of immersive virtual shows and performances over the years has been scalability. What does scalability mean for such a work? How far can I expand my audience while still preserving the characteristics that make these types of shows so unique?
Looking at successful productions like TONIGHT WITH THE IMPRESSIONNISTS. Paris, 1874, which can host up to 100 users simultaneously, one might think the possibilities are endless. However, the reality is much more complex, as Stephen Butchko explains:
“After years, now VR Chat allows us to go a certain distance before problems start. We did several test audiences for UNCANNY ALLEY: A NEW DAY, the biggest one made up of nine audience members and the two actors. Nothing broke. Rick and Christopher are optimization wizards and can build a world and optimize it to the point where we maintain frame rates and have everything working perfectly. So we can get a lot of audience members and they don’t all have to be invisible. …But the truth is that… we don’t want to. It goes back to what Brendan was saying about the intimacy of this immersive theater.
Like Deirdre likes to say, there’s something so strangely powerful and mystical even in an avatar standing in front of you and holding your digital hands with digital hands and touching your heart and asking you a question about your life. People aren’t prepared for that and our audiences, when they discovered they were interacting with real people, kept saying ‘This is what this field was missing’.
I think there’s something magical that we are tapping into with visual storytelling and with the intimacy that we’re creating with these works and it wouldn’t be the same with a larger audience”.
The intimacy that Stephen refers to is significantly at risk in larger settings, not merely due to technical issues – although that is certainly a factor to consider, as we will see shortly – but because of the challenge in maintaining a certain level of control over audience behavior.
“In the nine-people show,” Deirdre Lyons explains, “some people were really rowdy, while others were trying to listen to the story and couldn’t hear it. When there are so many people, it is much harder for the actor to corral all the people together and still make a good show. We learned that six people was a good balance.”
“As a user,” Screaming Colors adds, “I hate large groups in VRChat. I just can’t handle them. And I think one of the basic reasons is that our auditory apparatus is so sophisticated! Millions of years of evolution have taught us how to interpret the sounds of a room, and in VRChat, as much as there is spatialized audio, it’s not perfect obviously and the whole experience can be overwhelming”.
“In gatherings in VRChat, so many people often participate. And everyone always say that running these groups is like herding cats”, Rick Treweek tells us. “As soon as you have more than five people, it just turns to chaos. It doesn’t matter if you’re doing a presentation or a world hop or a show. It’s just human nature. You ask people to listen, they do it for one second and then they forget and start talking again. I don’t think it’s specifically a problem with acting shows. It’s a general problem that happens when a bunch of humans are together in a space where they can do anything they want. It’s very hard to control and definitely a thing to be explored”.
Challenge 7: Navigating artificial intelligence to preserve human connection
“No matter how advanced AI becomes, it will undoubtedly improve and mimic human experiences more accurately than it does today. But it will never be able to replicate what we, as carbon-based lifeforms, bring to shared experiences. And I freaking love that”
Stephen Butchko, theatrical director of UNCANNY ALLEY: A NEW DAY
Another aspect to explore is purely technological, specifically regarding the use of artificial intelligence and the current ubiquity of the term in the immersive sector.
It was well known that Stephen, Deirdre, Rick, Whitton, Christopher, and Brendan were, as they like to call themselves,“unicorns proudly wearing the Anti-AI badge”. I followed their reels and videos with great interest during the Hollywood Writers Guild strike, with Brendan Bradley emerging as one of its most visible figures. Named a strike captain by the WGA, he became “an accidental “influencer” on Instagram and TikTok, recognized as “the pineapple hat guy”.
What I didn’t know, and found immensely ironic, was the role of AI in UNCANNY ALLEY: A NEW DAY. In fact, it was artificial intelligence that generated the logo of the evil corporation our heroes are fighting against – almost the only visual element of the story generated by artificial intelligence. The choice to avoid replacing human interaction and intimacy with AI elements has a very specific meaning, as Whitton Frank explains:
“What we seek as human beings is connection. A hug in VR is extraordinary. You’re sitting there at home, arms open, headset on, and it feels so silly! And yet… it’s such a profoundly physical experience that our bodies remember even in a virtual space, recognizing the emotional meaning and connection it brings. In THE UNDER PRESENTS, that was the first gesture we, as performers and the audience, shared. But the moment you realize it’s not a human hugging you but an AI, that same gesture just becomes unsettling. It breaks the connection. Stripping it of its human essence is a huge disservice and a fundamental misunderstanding of what people want”
A concept also strongly reiterated by Brendan Bradley himself and Stephen Butcko.
“What we’ve discovered through our performances is that VR’s “killer app” is connection. Scaling human connection means going deeper into intimacy, fostering personal involvement, and creating more immersion. It’s not about how many hundreds of people we can pack into Fortnite—that’s a corporate take on metrics, which has absolutely nothing to do with how we’re using this tool” (Brendan Bradley)
“When GiiÖii first licensed WELCOME TO RESPITE from us, Screaming Color helped adapt the entire show into a full Korean-language version. When it premiered at BIFAN, they invited us. I don’t speak Korean, but during the dress rehearsal – performed entirely in Korean with their actors – I realized that didn’t matter. These actors, so sublime and grounded, made you feel the connection, and that was all the proof needed that the experience was working perfectly” (Stephen Butchko)
Challenge 8: Overcoming the physical and mental limits imposed by technology
One of the most significant issues I’ve personally faced when participating in immersive theater experiences in virtual reality is motion sickness. So far, only teleportation (when available as a design choice) and tunneling options have allowed me to join these shows and survive it; in other cases, I’ve had to exit quite quickly to avoid severe physical… problems.
Since WELCOME TO RESPITE, the Ferryman Collective team has carefully considered this challenge, always searching for new ways to make participation easier for audience members sensitive to motion. Whitton Frank shared some insights on their approach to this issue.
“I believe that UNCANNY ALLEY: A NEW DAY is our most successful show in this respect, as we’ve learned over the years what triggers audiences. In this show, you’ll notice that when you do need to move, it’s not for very long, and you quickly return to a still position. You don’t even have to run the length of the tunnel because a guard makes you stop. We also removed elements that could provoke discomfort. In this world, we’ve crafted an experience that allows most people to participate without feeling nauseous. We’ve learned a lot about how to mitigate those effects for some audiences”.
“Sometimes” adds Rick Treweek, who participated in the entire rehearsal process with Screaming Color to better understand the needs of worldbuilding and infrastructure, “I did say no to some of the things they wanted me to create (laughs). When you build a lot, you start to recognize those triggers. Especially when it comes to rotating the camera. If you give an actor complete freedom in those scenarios, they’re going to want to do wild things, which is natural. It ends up feeling like watching someone play a first-person shooter game. Even on a 2D screen, watching someone play can make you feel motion sick, while the player doesn’t because they know where they’re going to turn. It’s similar to driving a car.”
“Meta did a real disservice to the Quest 2 and 3 by shipping them with uncomfortable straps that can cause headaches”, reflects Deirdre Lyons. “These physical reactions, like motion-sensitivity, headaches, and all the other issues, are things that will improve with advancements in frame rates and technology, of course. However, a lot also depends on people becoming more accustomed to this world and figuring out what works for them with the headsets. I believe the audience is reaching a better understanding. As that happens, technology will continue to improve and start providing the tools that enhance experiences like ours. This medium is simply too powerful to fade away.”
Screaming Color highlights a different aspect that equally impacts the physical perception of both the user and the performers of the experience: sound.
“Among the factors that could make the entire system more manageable and popular is a complex approach to the volume and distance of different voices. Right now, the volume is too loud across the board. Ideally, our voices should be at 75%, while everything else at 100%. The actors’ voices should be audible from a hundred feet away, while audience members should only hear each other if they’re very close. This adjustment would increase accessibility to the experience without compromising the audience’s understanding of the story or the performers’ work. It’s something I’m currently working on; it’s time-consuming since the settings need to change depending on the scene, but it’s essential. Actors need to hear their cues and each other, and the audience needs to clearly understand what’s happening.”
To infinity and beyond…
“What’s been so exciting about Ferryman Collective’s journey over the past several years is how they began with a theater-first approach, adapting an immersive theater piece set in physical space into virtual reality. Then, with their second production, Gumball Dreams, they discovered this rich, beautiful existing VR universe and aimed to celebrate what could only be achieved in VR while bringing theatrical narrative to life. As an actor, what I find so successful in UNCANNY ALLEY is the blending of both these elements. It combines the theatricality of RESPITE with the dynamic interactivity and vibrant universe from Gumball, merging these aspects into a cohesive celebration of both mediums. It’s not just theater or just VR. It’s a genuine hybrid of the two, which I think is incredibly beautiful”
Brendan Bradley, actor on the show and himself director of immersive pieces
Years have passed since the first show by Ferryman Collective. The themes and content have gradually diversified, the worlds have expanded and been defined, and there have been changes in the onboarding process as well as the experiences offered to the audience.
Now, groups of friends can partake in the same performance, embarking on an adventure that blends elements of science fiction with romance, which they might later discuss over drinks at the pub, perhaps reminiscing about the characters they’ve grown fond of.
All these elements have made their last work, UNCANNY ALLEY: A NEW DAY, a captivating piece featuring a talented cast and an exceptional artistic and technical team. It entertains, engages, and even sparks a bit of imagination. It undoubtedly speaks to that same audience that is so cherished by the entire team, who eagerly awaits new ways to immerse themselves in these stories and enrich them with their presence.
The fan art created by artist Maryluis, featuring the determined main character of UNCANNY ALLEY: A NEW DAY, Gh0st, alongside the sweet, sentient service bot Atom sharing the scene with Nightmara, Beetlejuice, and the two Lone Wolfs Brad Pitt and George Clooney (all winners of the prizes awarded with the fanheart3 awards), beautifully illustrates how the world of immersive performances – much like cinema and theater – doesn’t end when the lights go out or the headset is removed.
The audience you’ve dedicated your performance to continues to exist… and it continues to remember. In this way, the magic of immersive storytelling and immersive performances lingers, inviting everyone to remain a part of the journey long after the experience itself has ended.
“It’s crucial for us”, concludes Stephen Butchko, “to ensure that our performances reflect the beauty of the world Rick has created, the stunning score, sound, particle effects, and animations that ScreamingColor did, the desire of the people who join us. This is what makes UNCANNY ALLEY: A NEW DAY so special. When our performances match that artistry, everything falls into place, and we become truly unstoppable”.
For more info about Ferryman’s next shows, click the link.
Leave a Reply
You must be logged in to post a comment.