Engineering Sensation: "Experience" Can Be Expanded! Taro Maeda (Part 1)

"Go Meet the People You Want to Meet!" Episode 14: Fumi Hito of Dentsu Inc. Event & Space Design Bureau met with Professor Taro Maeda, a leading authority in cybernetics research and Professor at Osaka University's Graduate School of Information Studies. We delve into his thought process and experimentation, tracing how he has fused his interest in humanity with his fascination with AI, crossing various scientific and humanities fields since his student days at the University of Tokyo.

Interview & Editing: Aki Kanahara, Dentsu Inc. Event & Space Design Bureau

 

We want to create "people" as a dynamic system

Hito: At Dentsu Inc., I belong to the Experience Technology Department within the Event & Space Design Bureau, where we research expanding experiences at events and in spaces through technology.

Personally, I have a strong interest in this field and am fascinated by the relationship between technology and human senses. I had read several of your articles and papers and wanted to meet you. I was particularly struck by Professor Maeda's video demonstrating how "vestibular electrical stimulation" (Note 1) can control human balance.

(Note 1) Galvanic Vestibular Stimulation (GVS): A technique that influences the sense of balance by placing small electrodes on the mastoid process behind the ear and applying a weak electrical current. Samsung's VR device "Entrim 4D," exhibited at SXSW this March, also utilizes this method.

 

We'll delve into this later (in Part 2), but first, could you tell us in a straightforward way about the content of your specialized research?

Maeda: To put it broadly, my primary research field is cybernetics. When people hear "cybernetics," they might think, "Are you building cyborgs?" But it's more about the concept of "human-machine theory" – the idea that if we view humans as organic machines, then human research can be approached within the frameworks of science and engineering. That's essentially what I'm doing.

The root of my primary research motivation is wanting to create humans. Why do I want to create humans? Well, as a child, I wanted to be a doctor who builds robots. While pondering what it means to build robots, I started thinking about what the most difficult elements are in robot construction. I realized it's difficult precisely because we can't create humans. So, I decided to study things that would help me understand humans, in order to be able to create them.

At Tokyo University, I realized I could teach myself mechanics and electrical engineering. I thought control systems seemed the most challenging, so I joined the department doing measurement and control. As a systems engineer, I think that was, in a way, the right decision. Ultimately, only systems engineers can create things that actually work. I originally pursued that field because I was interested in "humans as moving systems." And since I was fundamentally interested in humans, I wanted to study artificial intelligence, but it wasn't really taught at the undergraduate level. So, I joined a club and decided to create something for the campus festival.

While introducing AI using our seniors' programs, we built what you'd now call a head-mounted display. Initially, I made it because I wanted to enter the computer world. Though I couldn't write software as well as my seniors, I was confident in hardware. So, I decided to move into a different world, away from programming. I built a tank with a camera mounted on it, controlled it via radio, and created something that made you feel like you were piloting inside the tank by bringing that viewpoint to you.

While demonstrating a device where the camera moved in sync with facial movements, a senior who seemed like an alumnus handed me a business card from the National Institute of Advanced Industrial Science and Technology (AIST) and told me this was called telexistence (Note 2). That's how I ended up getting a job there.

(Note 2) Telexistence: Technology enabling humans to experience and act within a space distinct from their physical location with high presence. Research extending human presence itself into another space.

Maeda: My undergraduate thesis research was on neural networks, which recently gained significant attention through deep learning. While I was working on my thesis, backpropagation (Note 3)—the precursor to deep learning—was just starting to gain traction. I thought learning this might help explain why humans feel like they've entered a space. Why do people perceive a space as curved when they enter it?

(Note 3) Backpropagation: An error backpropagation method. A learning method for multi-layer feedforward neural networks, an algorithm for supervised learning. Published in 1986 by Professor Rumelhart and others at Stanford University.

 

Nitta: Huh? It feels curved?

Maeda: Yes. It's about studying illusions. Human spatial perception is curved relative to physical space—this is universal. We knew this curvature existed, so when using head-mounted displays for telepresence, we measured whether the world's curvature matched everyday perception. This was a major study in binocular depth perception, predating telepresence research. Then, while doing telexistence, I became interested in human perception itself. That's where psychophysics comes in. From there, we moved into research exploring illusions within virtual reality...

Nitta: That's quite interdisciplinary (laughs).

Maeda: For a while, people were wondering if he'd ever come back to engineering (laughs).

 

"Being able to create" means "understanding"

Maeda: On the other hand, did I stay in psychology forever? Ultimately, no matter how much I analyzed humans and wrote papers, I couldn't be satisfied unless I could use what I understood to create something. That's why I ended up returning to the field of engineering. I have this real sense that being able to create something means you understand it. My roots are in being a tinkering kid.

That mindset led to Parasite Human. I thought it could be realized by effectively applying the human illusion phenomena I was researching at the time to humans themselves. It's similar to virtual reality, but the difference lies in the perspective of information entering and leaving the human body. If you pursue this to its logical conclusion, it's less about putting humans into a computer world and more about extracting what humans are doing outwards. Ultimately, I want to know exactly what humans are doing and how they're moving, just as it is. Similar research includes lifelogging.

Nitta: Life logging simply accumulates your own sensor data and information.

Maeda: Just accumulating it, right? That's a waste. I thought, since there's a copy of myself experiencing the same things, I should just have that copy assist me. That's the concept behind Parasite Human.

Hitō: So the term "Parasite Human" itself was proposed by Professor Maeda?

Maeda: Yes. What kind of research is Parasite Human? It's similar to wearable sensory measurement devices. But it's not just about measurement; it's a mechanism that converts the measured information and can show it back to the human. You attach wearables: a display and camera near the eyes, a microphone and headphones near the ears, and a device near the hands that creates illusions about what they're touching. The core concept was to create a device that could intervene while simultaneously monitoring human senses, without hindering behavior. By enabling the capture of any information from bodily movements, when separated from the human body, it creates something like a "human spread" (akin to a "spread mackerel"). It results in a humanoid robot-like form without a skeleton or muscles—essentially, a motor and frame-less human. Only the cognitive model of the nervous and sensory systems remains.

Nitta: So wearing it makes the person wearing it become robot-like themselves, right?

Maeda: Exactly. Think of it as a "non-powered powered suit." When people imagine wearable robots, they usually picture powered suits, but this is a powered suit with absolutely no power source.

 

Parasitism is symbiosis; like a horse and rider as one, the robot can benefit the person.

Nitta: What I found interesting in your paper was the description of feeding various sensor information back to the human themselves, rather than sending it to the robot or an external device.

Maeda: Yes, exactly.

Hito: So, "parasite" – is it like the feeling of having a different self inside you? I understand "parasite" means to live off something else, but I'd like to hear the meaning you intended with the term.

Maeda: Back then, even within parasitology, perspectives were emerging that parasites, while living inside the body, are symbiotic and don't just cause harm. The robot we've been discussing doesn't move on its own, but by being worn and moved by a person, it can function just like a humanoid robot. In fact, it moves more intelligently than any existing robot. It doesn't move itself; it has the person move it.

Humans possess the highest intelligence currently existing. In that sense, by constantly absorbing information about movements optimized for a human form under that high intelligence, it truly parasitizes human movement. It's a symbiotic relationship, so it can benefit humans.

It's like horse and rider as one. When a person rides a horse, they move as if they were a single living being. But is the rider merely a weight to the horse? No. The rider is another set of eyes, another head, a partner moving as one. You might think the human is being controlled, but actually, the robot sends information that makes the human naturally feel it's the right thing to do. That's the design philosophy of the Parasite Human.

I thought, if we create a copy of a human, that would be the ultimate interface. If it's not an exact duplicate of yourself, you can't achieve true telepathic understanding. But a robot that has experienced everything exactly as you have becomes your replica. That's the ultimate interface, and the question is how to create it.

What this device does is experience the same things as you. It acts the same way you do. At the very least, once it's worn, it does everything together with you. This allows the robot to predict what the person is about to do. The idea is that if it learns all of this through deep learning, it could create a copy of you. Once it's confident the accuracy is high enough, it can correct you when you deviate from that. Of course, if you want to do something with a different intent, it should be able to influence you in a way that allows you to ignore it. By making it wearable, it's a method where the robot feeds sensory feedback back to the human.

※Continued in Part 2