Get pumped?! Interactive production

In our previous LED feature, we introduced an LED Christmas tree that changes its lighting in response to visitors' movements. In this fourth column, we delve deeper into "interactivity," featuring an interview with Toshiyuki Inoko, founder of the ultra-technologist collective teamLab.
"Interactive" experiences—those that respond to natural human actions like voice or movement—blur the traditional lines between performer and audience found in conventional "stages" or "displays." They draw visitors into the artwork, making it feel personally relevant.
We will explore the effects of being "interactive" by categorizing them into the following three points and examining examples.

① Enhancing Participation

Mobile-based participatory installations have become essential content for events.
Water screens, which project images onto a sprayed water membrane, have traditionally been passive installations for viewing large-scale displays. At the "Kagawa Waterfront Festival," teamLab hosted the "teamLab☆Spin the Reel! Spin it round and round to reel in everything from seafood to monster fish! Tournament." Within the vast virtual ocean unfolding at the end of the virtual fishing line, various fish swam. By working together to shake their smartphones—used as reels—visitors could catch the fish projected onto the water screen. This high level of "participation" and "experience" was vastly different from the traditional act of merely "watching."

②Expanding Physicality

Raise your hand and its trajectory becomes visible; blow air and its movement shimmers as color.
Advancements in motion sensor technology now allow interactive responses to your movements, visualizing them and expanding your physical awareness.
Games using Microsoft's "Kinect," where your entire body becomes the controller, are a prime example. In the Xbox game "Fruit Ninja Kinect," the Kinect sensor tracks your arm's trajectory as if it were a blade, allowing you to slice through fruit flying through the air. The smooth responsiveness and satisfying effects create a sense of expanded physicality, making it feel as though your own arm truly wields the blade and you've entered the game world.

③ Enhance production effects

By changing colors and shapes in response to performers' movements or displaying scrolling text, interactivity elevates its functionality as stage equipment.
Nike Basketball's "House of Mamba" campaign in Shanghai featured a full-size, state-of-the-art LED basketball court. It incorporates motion tracking that reacts to players' movements, along with interactive LED visuals, graphics, and video. This allows spectators to enjoy the players' movements more visually. It goes beyond mere observation; it visualizes the players' movements to focus on and the plays to watch, enabling spectators to engage more deeply with the game.

●Advances in digital technology now allow audiences to be incorporated into works in limitless ways

The interactive presentation techniques introduced so far have been made possible by the evolution of digital technology. We spoke with Toshiyuki Inoko, CEO of teamLab, which has released numerous interactive works.

──I recently participated in "The light orchestra with Pepper" at the "Pepper Tech Festival 2014" (hosted by SoftBank). It was fascinating. My phone became part of the performance, randomly lighting up and making sounds. I was surprised (laughs).
(Watch the light orchestra video here↓)
http://www.team-lab.net/all/other/the_light_orchestra_with_pepper.html

Inoko: That hacked the app to light up and make sounds by tracking the audience's positions with sound (laugh).
Actually, the audience isn't doing anything actively. So, to take it to the extreme, it's conceivable that people walking down the street could unknowingly become part of a human character formation.
For example, on Omotesando Street, one day, at some point, suddenly everyone's phones get hijacked, and they start flashing from top to bottom of the slope. You'd be like, "Why are they flashing?!" and then your own phone starts flashing too. That'd get your adrenaline pumping, right?
"What the heck is going on?!" People might even start complaining (laugh).

──Yeah, complaints would come in (laughs).
This was also the case, but teamLab often creates interactive, participatory works using digital technology, right?

Inoko: With the materialistic values of the 20th century, viewers tended to be passive, just observing. There was very little room for the audience to participate.
With the evolution of digital technology, visitors can now engage with the work as much as they want.
I think there have always been things in physical spaces where people could participate—like building things together with cardboard or everyone drawing a picture. But then, while it's fun while you're doing it, it doesn't hold up as something to be viewed.
With digital, you can design the act of participation itself to some extent behind the scenes, so you can guarantee the state. You can add follow-ups, so to speak... turning what the user wanted to express into something we can handle... though saying "handle" sounds a bit harsh (laughs).
We became able to design and express change itself, premised on visitor participation.
In other words, by making the work "interactive" for the viewer, we've made it easy for them to "participate" in the work.

 

　

The evolution of interactive presentation techniques that keep raising customer excitement will continue! That concludes our fourth installment, featuring "Interactive" techniques.

---

Dentsu Inc. Experience Technology is a technology group that combines digital and analog technologies to create unprecedented, moving experiences.
Even ideas that seem impossible to realize may find a path to implementation through technology.
Feel free to contact us at et-info@dentsu.co.jp (Attn: Yoneyama/Murakami) with any opinions, inquiries, or requests.