NOIRE (COLORED) is an experiment that deals with racism and segregation in the USA in the 1950s – with an uncommon universality, reinforced by the choice of mixed reality. Between real-life settings, multimedia narration and a subject that’s still (unfortunately) topical, we take a look back at a work that travelled from the Centre Pompidou (France) to XR festivals in 2023, and will be in Milan, Montreal and Luxembourg in the upcoming weeks.
Next shows:
Novaya, a commitment to immersive creation
Pierre-Alain Giraud – My arrival in immersive creation is in line with my early work in cinema, animation and documentary – often between France and Iceland. I’ve also created for the theater, notably alongside Arthur Nauzyciel and Stéphane Foenkinos, making films that are projected on stage. This year I directed a video clip for Björk with Gabríela Friðriksdóttir, “Victimhood“, whose visuals were then adapted for the stage for her “cornucopia” tour.
P.-A. G. – We set up Novaya, an immersive production company, after making a first work, SOLASTALGIA, which I co-directed with Antoine Viviani (and with Emanuela Righi on production) and which was lucky enough to go to Sundance in 2020. This experiment used the first generation of HoloLens, which wasn’t the easiest thing in the world. But the seed had been planted, and we wanted to continue along that path. For SOLASTALGIA (produced by audiovisual production company Providences), the storyline called for mixed reality, with a real setting and very physical interaction around the appearance of ghosts. At the time, the technological solution was limited, and the visual result unsatisfactory, but we were beginning to see the narrative potential of these technologies.
P.-A. G. – Novaya was then formed around these issues, but we absolutely had to succeed in doing better visually. With the arrival of Pierre-Luc and Mathieu Denuit to work on the MR software platform, we were able to launch other projects such as COLORED. Today, we have much more technical and artistic freedom, working on Unity with the second-generation HoloLens. Thanks to a Wifi streaming technology we’ve developed, we’re able to bypass the computational limitations of headsets, and show visually what we want, on several HoloLens simultaneously. It really changes everything.
Adapting COLORED for mixed reality
P.-A. G. – COLORED is our first MR project to come out of this consolidation phase, created from Tania de Montaigne’s book released in 2015. Stéphane Foenkinos had done a first theatrical adaptation, in which he asked me to make videos projected on stage. I was working on SOLASTALGIA and the play at the same time, and creating COLORED as an immersive installation quickly became an obvious choice. Mixed reality allows us to bring the ghosts of the past back into the present, so it has real documentary potential.
P.-A. G. – The Centre Pompidou came on board very early in the project, right from the development phase – starting with the communications and digital department with Agnès Benayer at the time, then Paul Mourey and Antoine Immarigeon who followed the project. The challenge came from the hybrid nature of the project, between digital, exhibition and theater, so we had to bring together departments at the Centre Pompidou that don’t necessarily work together very often. Communication and Digital, Spectacles Vivants, but also Production with set construction (Laurence Fontaine, architect-scenographer and her team), carpentry, electricity, the video department… and they’re still working alongside us on the cast! COLORED was in residence at the Centre Pompidou from February to April 2023, where it was actually built. The project is now touring France (Valence, Toulon in 2023 on a model of around 150 people/day, and other dates in 2024) and Europe (MEET in Milan, VR Pavilion at the Luxembourg City Film Festival), a second set is being built for the PHI Center, and a third for the National Theater in Taiwan.
P.-A. G. – Producing a work in mixed reality is quite different from creating a film, although the processes are similar. First, there’s shooting in a 4D Views studio, which allows us to film our actors in 3 dimensions. We went off to shoot at the IP Lab Studio in Taiwan (COLORED is co-produced by Taiwanese company Flash Forward Entertainment). The characters you see in the experience are really actors filmed, one by one, with 48 cameras all around them. We then receive the videos of each character in 3 dimensions, and carry out an initial editing process similar to that used in the cinema. On the computer, we build narrative sequences by placing images and sounds end to end. But then we look at our editing in space, and begin to work on “holographic” staging, which is closer to theater. You have to be able to project yourself into several dimensions.
P.-A. G. – As with SOLASTALGIA, we worked with Valgeir Sigurðsson (founder of Greenhouse Studios in Iceland and a close collaborator of Björk) on music and Nicolas Becker (Oscar winner in 2021 for SOUND OF METAL) on sound design. Given that the shooting studio didn’t offer much in the way of sound recording possibilities – everything had to be recreated afterwards, noised, spatialized and so on. Almost a theatrical approach, with speakers positioned in space. What’s more, we finished mixing the sound on location, to adapt to the acoustics of the Pompidou venue.
P.-A. G. – With COLORED, we had to develop technologies that could keep pace with our artistic needs. Now we have something stable that will enable us to produce new projects. For narrative works, there’s a real public expectation, and we hope to be able to help other creators work with these tools.
Envisioning the physicality of immersive creation
P.-A. G. – The first phase in creating COLORED was sketching. Seeing as we have a real set, to which we add virtual objects and characters… we had to imagine how the two worlds would fit together… A real mental puzzle to anticipate the placement of the holograms, the attitude of the spectators in the scenes, etc. We arrived at the shoot after working for several days with the actors. All the scenes had to be rehearsed very precisely, down to the second, because you can only film one actor at a time. When you see Claudette in front of the judge or on the bus, refusing to get out of bed, she had to do it all by herself, with very precise indications of look and timingThen we recomposed the sequences in post-production.
P.-A. G. – The spatial layout of the images then evolved considerably from what we had imagined: we had to confront the physical setting, the presence of the audience. We therefore carried out tests with groups of spectators, at 104 in Paris (Novaya is incubated at the 104factory) and then at the Centre Pompidou, which enabled us to improve the rhythm of the narrative, the positioning of the virtual characters, and to refine the links between the virtual sequences and the spectators… During the tests, many people were lost, didn’t know where to look and had the feeling they were “missing” important things in the narrative. That’s when we came up with the idea of adding streams of suspended particles to guide viewers, to direct their gaze more naturally towards the main action.
P.-A. G. – In this type of creative process, you have to improvise a lot and let yourself be surprised. There’s also a screen and a 2D projection in the installation, showing the narrator, Tania, and some archive footage. It wasn’t easy to find a place for this screen. r It was only at the very end that we managed to give the period images their rightful place, by letting ourselves be surprised to see Claudette’s hologram dancing in front of the screen. Our set is deliberately minimalist, so that it can be transformed into several different places, signified by a few objects, a few visual details, and also by sound. . Valgeir’s sound and music are powerful links to unify the experience, without ever totally taking over. The use of rain and wind can do much more for the spectator than a strong visual presence. Making the invisible, the shadows, speak, arousing emotion with a minimum of visuals, that’s what really interested us.
P.-A. G. – The real aim of COLORED was to produce a collective experience that would be striking in its form, without overshadowing its purpose. In the end, we live the story together, in a real physical dynamic and a form of collective consciousness that makes each session a unique and different moment. When you observe the ballet of spectators in the experience, you see the respect and freedom that each one grants to the other. It’s as if you’re physically watching people “socialize” as best they can, in the face of Claudette’s story.
What’s next for Novaya?
P.-A. G. – We’re currently producing JEUX, a project we hope to open for the Paris 2024 Olympic Games. It’s a documentary project around the place of sport in art, where we propose to look at each athlete as a work of art in itself, to renew the dialogue between art and sport.
Leave a Reply
You must be logged in to post a comment.