Echoes from the Future of Opera

In a sound studio at the Massachusetts Institute of Technology (MIT), Elena Jessop slipped on a long fabric glove into which was sewn a circuit board here, a pressure sensor there — all entwined with multi-colored cables and wires. A unique marriage of textiles and electronics, the glove allows its wearer to manipulate her signing voice using simple gestures.

“You can grab a note you’re singing,” Jessop explained, plucking a note from her mouth with her gloved hand. She shut her lips, but the note continued to reverberate. “Squeezing introduces a harmony note,” she said, pressing her fingers together. The note changed pitch. “You can also add a little bit of vibrato by shaking your hand.”

With the guidance of innovative composer Tod Machover, Elena (Elly) Jessop and colleagues in MIT’s Opera of the Future research group explore “what technology can do to enrich our interaction with music,” she said. Their largest ongoing project is Death and the Powers, a new opera that makes unprecedented use of technology to modify live musical performance. The production will feature special instruments — like Jessop’s glove — that interpret a performer’s desires, a giant electromagnetic chandelier played like a harp, and a fleet of opera robots. All this technology might seem at odds with traditional notions of opera, but the technical wizardry won’t steal the show. According to the MIT team, the idea is to incorporate technology only when it legitimately develops the story and themes.

Scheduled to premiere in Monaco in September, 2010, Death and the Power’s creative team includes former U.S. Poet Laureate Robert Pinsky as librettist, Director Diane Paulus of the American Repertory Theater, and Hollywood production designer Alex McDowell, who has worked on Fight Club, Minority Report and Watchmen.

The opera’s plot centers on powerful businessman Simon Powers, who uploads his consciousness onto a massive computer system in order to live beyond his natural life. Although Powers dies in the second act, his essence persists throughout the performance. His daughter Miranda, his wife Evvy, and his assistant Nicholas struggle to cope with his ambiguous death, as Powers learns to manipulate the external world through his new computerized existence.

“In the script it’s given that Nicholas has a prosthetic arm that somehow makes him specially able,” Jessop explained. “So we said, ‘What if he had a prosthetic arm that actually helped him be more expressive with his singing?’”

Inspired by that goal, Jessop began designing her glove, officially called the vocal augmentation and manipulation prosthesis (VAMP). Sensors embedded in its fabric interpret the wearer’s gestures.

Pressure sensors recognize when fingers press together, a gesture that extends a note and controls pitch. Bend sensors detect extension of the arm, which alters volume. And accelerometers measure acceleration and vibration, warbling a note with a shake of the hand. All these sensors wirelessly transmit data to a computer, where a program captures the sound coming through the actor’s stage microphone, modifies it according to his gestures, and outputs it through speakers.

Jessop’s colleague and doctoral candidate Peter Torpey turned to similar sensor technology in developing “disembodied performance” — a technological means of allowing an absent performer to remain actively involved in a live performance. Death and the Powers will use disembodied performance to maintain the theatrical presence of central character Simon Powers, even after his physical death. “My work shows that you can separate the human body from the performance and still preserve the salient forms of that performance,” Torpey said. “The stage itself comes alive as the main character.”

Offstage, the actor playing Simon Powers wears pressure sensors, bend sensors and accelerometers — which measure acceleration, orientation and vibration — on his elbows, shoes and torso. As with VAMP, these sensors wirelessly transmit data to a central computer that is programmed to modify the onstage performance using input from the actor’s singing and movements. For example, the walls of the Powers household ripple with color to reflect Simon’s mood, “bristling when someone walks by whom Simon doesn’t like,” Torpey said. Advanced surround sound technology — which can reverberate in any and every direction — will cocoon the entire audience inside the voice of Simon Powers, rather than simply projecting his singing in one or a few directions.

Death and the Powers also features a group of semi-autonomous robots, ranging in height from four to seven feet, with large triangular lights for heads. The robots serve not only as sculptural set pieces and stage lights — they are legitimate characters in the opera. Although these Operabots will be partly pupeteered, they are capable of sensing and avoiding one another and even performing semi-autonomous movement: “We tell them that at this point in the score you need to be right here, but along the way we can give them a bit of autonomy,” said Noah Feehan, who has worked extensively with the robots at MIT. The Operabots also function as a kind of Greek chorus — guardians of the opera’s narrative — continually retelling the story of Simon, Miranda and Evvy through a choreographed pageant. “You could read it that the robots are what become of the characters or that they are a form or memory — it’s very ambiguous,” Feehan said.

Many-legged furniture — inspired by the giant mobile sculptures of Dutch artist Theo Jansen — will come alive on stage. Suspended above the actors will hang a giant stringed instrument called the Chandelier, which can be stimulated by both a human player and by electromagnets, allowing for a much greater range of notes than a standard string instrument. The Chandelier provides the opportunity for a powerful duet between Powers and his wife, after his physical death — another form of disembodied communication.

Infusing an opera with so much technology presents unique challenges for researchers, performers and audience members alike. “The phrase ‘robot opera’ is so distasteful to some people. That is very much on our minds here,” Feehan said. “There’s a major danger that happens when people let technology become the primary focus of a performance,” Jessop added. “Unless the technology is supporting a story, an idea or experience, you probably won’t be interested.”

But as Torpey pointed out, “The real question is: What can the technology give you that you can’t get any way else?”

Peter Swendsen asks himself and his students exactly the same question. A professor of computer music and digital arts at the Oberlin Conservatory of Music, Swendsen believes technology “extends our ability to execute in performance any parameter of music — parameters like pitch, density, volume. In the last decade we’ve been able to see this more during live performance, in real time.”

According to Miller Puckette, who directs the Center for Research in Computing and the Arts in San Diego, computers give musicians limitless opportunity to experiment. “Once you get the sound in the computer, the possibilities are so vast. The only limitation is the imagination of the performer,” he said.

Jessop echoed his sentiments. “We can almost forget about what’s ‘technically possible’ — we try to make happen whatever we imagine.”