159 Pioneer Street
Brooklyn, New York
Through April 16
High in the rafters of Pioneer Works in Brooklyn, a digital display too kinetic and dynamic to exist in the real world swims through space. As Medusa ebbs and flows, seafoam-green square slats shoot from the 40-foot-tall ceiling all the way to ground level with seemingly no regard for the humans below. Too large to be seen in its entirety, the oscillating portions of Medusa that can be viewed ripple back and forth like the flow of the aurora borealis, or eddies and gyres in the ocean. All of it swirls above a player piano performing a thalassic soundtrack written by Kelly Moran, who will be at Pioneer Works accompanying the display in person on April 8 and 15.
This installation at Pioneer Works is the first time Medusa has been staged in North America. It made its debut at the 2021 London Design Festival, held at the Victoria & Albert Museum in London. The multimedia nature of an interactive installation that spans both the real and digital worlds required an equally interdisciplinary team to realize it. That includes Tin Drum, a mixed reality art collective founded in 2016; scientist and project director Yoyo Munk; paleo-futuristic Japanese architect Sou Fujimoto, who handled the actual architectural and design aspects; and pianist and composer Moran, whose original composition for the Brooklyn show riffs on the score written by Japanese composer Ryuichi Sakamoto for Medusa’s inaugural showing in 2021. (Sakamoto, who lived in New York since 1990, died last week at the age of 71 due to cancer.)
Navigating Medusa is easy and intuitive. It’s also something that would have been impossible even just a few years ago. After borrowing a Magic Leap 2 from the check-in counter—a mixed reality headset tethered to a pouch-sized PC—and completing a quick calibration, the entire loop opens up to the audience. It is presented as one instance for everyone, meaning every guest sees the same object from the correct vantage point as they explore it, as if Medusa was really there. As visitors walk between and through the waves of vertical columns to the layered oceanic sounds, the installation reacts and shoots stakes to the ground, changing with every body that passes through.
“A core theme within Medusa is the blurring between individual and collective behaviors,” Munk told AN. The topic is “reflective of the ways in which real-world architecture influences our individual and collective experiences of the world.”
Because Medusa can’t actually “see” the people underneath it—it perceives the locations of the headsets—Munk likens the interactivity to that of a motion-capture suit: “Each member of Medusa’s audience effectively serves as a single marker on the amorphous moving body of the collective. The installation has a disposition of its own (a cycling pattern of greater and lesser activity, representative of diurnal cycles), but its mood is influenced by what it observes about how its audience is connecting with it.”
Of course, even as the piece itself evolves, the technology that displays it has to keep up accordingly. The biggest hurdle to overcome is that mixed reality (MR), up until recently, has been pretty unpleasant to use. Older standalone headsets often felt cheap, with dim, curved lenses that offered narrow fields of view at a low resolution. The original staging of Medusa used the much cheaper Nreal Light glasses, but this iteration uses the aforementioned Magic Leap 2, which was only released in 2022 at a cost of $3,299 per headset. The upgraded hardware allowed a significant improvement to the work’s fidelity and immersion.
My time under Medusa was hands down the best MR experience I’ve ever had. The field of view is natural and wide enough to take in nearly the floor to the ceiling, and the lenses are bright enough where it feels more like wearing a pair of sunglasses than view-obstructing goggles. It bodes well for future interactive exhibitions that depend on creating a singular object—or even a set of them—that needs to live as the same piece for everyone viewing regardless of their position.
That synthesis of the digital and biological world extends to Medusa’s subject matter—surfacing deep sea life and planet-scale natural phenomena at a human size—making it impossible to disentangle from the medium.
“A playful blurring of the relationship between architecture and nature is deeply ingrained within Sou Fujimoto’s architectural practice,” explained Munk, “and this dovetailed rather conveniently with my own background in biology and procedural graphics. Working with Barbara Stallone and Francisco Silva (now practicing independently as Ex Figura), we were especially interested in the idea of using light as the basis of a three-dimensional structure, which is both inherent to the way that true mixed reality works.”
Munk continued: “We chose to build Medusa as an amalgamation of inspirations from non-human structures, filtered through a humanist architectural lens: our inspirations included atmospheric auroras as well as the body plans of deep sea bioluminescent siphonophores (close relatives to jellyfish), stylistically represented through the repeating overlay of thousands of square-sided vertical columns.
The goal, Munk said, “was to remain abstract and to eschew any sense of pure representationalism, […] to create a piece of abstract architecture that evoked the sense of human wonder that comes from observing these rare structures.”
The end result is seamlessly immersive, but also creates a piece that’s highly dependent on being installed in a building that can handle the strict technical requirements; a tall ceiling for full effect, full light control to get the most out of the AR headset, and a way of streamlining the equipment hand-off between venue staff and guests.
It also results in a totally unphotographable art piece. The only way to view Medusa is to see it in person on view at Pioneer Works through April 16. However, if you miss it, Tin Drum will present Kagami, a mixed-reality concert with music by Sakamoto scheduled for later this year at The Shed. It promises to be just as interactive and elusive.
Jonathan Hilburg is an electronics editor at Reviewed, part of USA Today, who focuses on gaming. Previously, he was The Architect’s Newspaper’s web editor. He lives in Manhattan and is keenly interested in the intersection of art, architecture, and context.