“With the XR medium we want to put people on scene” – Nonny de la Peña (Emblematic Group)

She is known as the “goddess of VR”, but far from the skies Nonny de la Peña is mostly interested in the real world. Through a relentless work as a creator and producer of immersive experiences with her company Emblematic Group, most often documentaries for the last 12 years, she explores this interactive narrative whose limits are still unknown.

We met with Nonny de la Peña just after SXSW… and before Tribeca Immersive 2021 where her last project was selected as part of the Virtual Arcade section.

Since 2008 – What an amazing XR community!

Nonny de la Peña – I love SXSW people, and this year had a lot of interesting topics to discuss – even in these strange circumstances. Of course there is a specific flow to live events, the group of people you meet and a schedule you try to follow. It’s way harder to feel this recently, to be “in” the event. But I’m lucky enough to have a very good relationship with everyone in the XR industry: most of the time I see content out of the festivals, I discuss with creators and influencers every day… It’s amazing to see how the industry grew up! It’s a very robust community now, with social platforms etc. When the pandemic started, we were there for each other. We had online meetings every Friday night, and we shared job opportunities.

N. de la P. – There is also a value in how our immersive stories can have an impact on the world. My first goal was to show the invisible to everyone, the hungry, the social issues.. Since then, I’m glad I had a chance to inspire some of the XR people to go on their own way and to gather around common topics of interest. With Covid-19, people understood that the digital world is both useful and complex. VR or AR experiences, social platforms… We are today in a very positive space regarding the virtual creative community.

Feeling the impact – Our presence in XR

N. de la P. – It’s about putting people on the scene, to become a witness to a story. This is something unique to our medium. When we go to the movies, our bodies are along for the ride. When something scary happens, you jump! Virtual Reality goes further; to be able to interact, move through the story is something else. It has a much deeper connection to the idea of presence. When you bring a full body into an experience, you get a sensation of duality of presence: You feel like you are here and there at the same time. There is no other medium that can provide this! For Art Basel Hong Kong I created a piece called THE LIFE OF A WALL ON LIN HE ROAD (iteration of “Safely Manoeuvring Across Lin He Road” (1995) by Lin Yilin – link), where participants take on Lin’s role as roadside artist. While it was CGI, it tracks the building and the street just as it was when Lin performed. The idea of the work was to immerse people into how virtual reality would recreate performance art.

N. de la P. – And we will go further. Everyday there are new tools to help us on this – just take the example of the cameras of our smartphones. Journalistic practices are evolving fast, you can now scan entire scenes live. We will add motion effects to volumetric videos. With Engage, they can capture volumetric content and “rewind” that content so you can have a playback that emulates a video recording of a spatial experience that has actually transpired on their platform. Ultimately, I really want to inspire young generations to use this new media to tell stories, every day. It’s working great for educational purposes – and not for pre-recorded contents, but for live lectures and events. Students can interact instead of trying to connect through a flat screen.

Producing immersive content at Emblematic

N. de la P. – I remember telling Chris Milk that I wouldn’t do 360 videos because it didn’t allow for a fully embodied experience. Nearly five years later, our research on AFTER SOLITARY and GREENLAND MELTING – where we took the story from flat screens to 360 videos, then to volumetric walkarounds – confirmed what I was feelling. With ACROSS THE LINE, we worked with the Virtual Human Interaction Lab at Stanford University to study what subjects were experiencing. It was added to the work already done by Mel Slater and Maria Sanches Vives to dispel the notion of the validity of the “bystander effect.” (link) In the headset, it was very powerful. And people wanted to intervene all the time! It was a great demonstration on how we can trigger people’s sense of presence on scene at an event.

N. de la P. – Back when I worked on my piece GONE GITMO in Second Life, I knew we had to put people on scene – and everything since then has confirmed this idea. Our next projects at Emblematic (in Brazil with Lenovo) and a major project on Lyme Disease are also intended to bring us closer to the important topics we want to explore. We are also working on the platform REACH (link) to create volumetric-ready contents pretty easily and through WebVR. We presented it first at Sundance New Frontier two years ago. You can add, organize, and publish your VR experience quickly.

Tribeca 2021 with A LIFE IN PIECES: THE DIARY AND LETTERS OF STANLEY HAYAMI

N. de la P. – We worked the last three years on this project as a labour of love really. This is a story of a young man, 15 years-old, who comes of age in one of the concentration camps in the US for Japanese-American citizens during World War 2. He was really a deep-thinker, and despite his age, he had a unique voice for expressing the unfairness of his circumstances. Ultimately he was pulled from the camp and drafted — even though their parents were still locked up they would pull kids like Stanley, barely 18 now, and put them on the front line of WW2. (Stanley’s cousin refused to go and was put in federal prison.) Our hero died in Italy 10 days before the war was declared over. We took his beautiful drawings, and animated them in Quill. Our animator, Haley Tomaszewski, had their own grandfather in Poston concentration camp, as was the family of my co-writer Sharon Yamato.

N. de la P. – When Haley started, they didn’t have any experiences with Quill but I chose to give tis atypical demographic applicant an opportunity to work on A LIFE IN PIECES, to give them a chance – and they did great! Their drawings are really beautiful. At the same time we did a lot of photogrammetry inside the camps, the barracks themself. It’s a strange feeling to stand in these small rooms. Then we used stand-in green screen videos, due to our budget’s restrictions. In the experience, you have a reflection on a hologram to help offer a deeper connection for the user. I am so glad that at this moment with so much Asian-American hate found across America we are able to offer something, a piece of art, a rebuke. But most of all, it’s Stanley’s story and how you can find beauty and peace in a troubled time. If we can help to tell his story through a technology that “beams” you to another place, since he dreamed of being “radio’ed” around the world, I feel that we are being true to who he really was.