Engineering Sensation: "Experience" Can Be Expanded! Taro Maeda (Part 2)

Nitō: The lower part handles hearing, while the upper part controls balance. Are the semicircular canals also part of the vestibular system?

Maeda: The vestibular system consists of the semicircular canals and the otolith organs. The semicircular canals detect human rotation, while the otoliths detect gravitational acceleration. It's not just gravity; it also detects the acceleration when a person moves. In terms of smartphones, that's the two sensors everyone uses: the gyroscope and the accelerometer. What's in smartphones but not in humans is the compass. It's the sensor that detects the direction of the Earth's magnetic field.

Nitta: Ah, that's interesting. So smartphones are superior to humans in that sensor capacity. Humans lack a geomagnetic sensor. I see, that's why humans get so directionally challenged.

Maeda: Research on carrier pigeons suggests pigeons might have one, but it's still undecided in biology. Why can birds fly at night? Why can they find their way despite having poor night vision? One theory is they sense the Earth's magnetic field.

Nitta: What sparked your interest in the vestibular system?

Maeda: I was trying to understand human senses and find ways to input them. While aiming to cover all five senses, I realized the sense of balance was particularly difficult to approach conventionally.

Hito: The sense of balance isn't usually included among the five senses.

Maeda: The term "five senses" itself isn't really appropriate for describing human perception. The inner ear is buried inside the skull. Inside the skull is the brain, and outside is the skin. There's a hole in the bone between them, and the inner ear is packed inside that.

Nittō: So the brain and the ear feel connected, right?

Maeda: Exactly. After all, the skull is one continuous cavity from the ear opening all the way to the brain chamber. Buried within that intermediate bone is the inner ear, and its location is the mastoid process.

Regarding vestibular electrical stimulation, it had already been discovered about five years before we began our research (around 2000). It was first used in otolaryngology, but it didn't catch on widely and started to fade out. The reason was that it worked too intensely to be used for medical testing purposes, but we focused on that very aspect.

Nitta: I heard you can experience it just by attaching a small device like a sticker behind the ear, which I thought was amazing. How lightweight can you make the device?

Maeda: As researchers, we're not aiming for commercialization, nor are we pursuing the absolute limit of miniaturization. But in theory, using existing battery and circuit technology, we could make it almost as small as needed—as long as we can secure a sufficiently powerful battery.

Hidaka: The battery size?

Maeda: Probably down to the size of the electrodes themselves. But ultimately, there's a limit to how small you can make the electrodes... Because it hurts.

Nitta: Because making them smaller requires applying a stronger stimulus.

Maeda: Exactly. In that sense, if you ask about the ultimate limit of miniaturization, it comes down to the electrode size—the smallest size that doesn't cause pain to the person.

 

Immersion is a state of seamless integration in multimodal experiences

Nitta: While VR centered around vision is getting a lot of attention right now, I sense a different kind of potential in other senses—like VR involving the ears and hearing.

Maeda: What you're describing probably isn't limited to just balance or vision. The essence of virtual reality is that multiple senses, known as multimodal, all point to one reality. That is, what you see and what you hear—for example, if something is ringing here, the sound indicates it's here, and what you see indicates it's here. It's the alignment of multiple senses: "Ah! The sound changed simultaneously with what I'm seeing. So this must be what's here!"

Nitta: So you're saying it uses various sensors.

Maeda: The Virtual Reality Society has long discussed what constitutes human immersion. We define immersion as a state of seamless multimodality—where all senses point to the same reality. The VR that's popular now is probably just a two-sensory multimodal system where your body movements and visual movements finally match. When a third sense comes in, the level of presence jumps up. That's probably the biggest thing missing in the field right now.

When you reach the point where you hear sound coming from exactly there, feel something right there, and even experience the vestibular sensation of having just made that movement—that's when you get close to true realism. That's what virtual reality is striving for, isn't it?

With virtual reality, the more realistic it is, the less likely you are to get motion sickness. Current VR causes sickness because the realism breaks down. For example, game sickness happens because, in front of a big TV, the images move as if you're being shaken, but you yourself aren't moving. If you were moving while watching, you wouldn't actually get sick and would feel it as real.

Nitō: So based on what you just said, the more intense the action, the more you can synchronize the vestibular electrical stimulation to match the motion and sense of balance, which actually reduces motion sickness, right?

 

Can artificial intelligence create emotions?

Hito: Currently, head-mounted displays are synonymous with VR, but are there any examples of implementation or mass production for wearable devices that incorporate the tactile, vestibular, olfactory, and gustatory senses you mentioned?

Maeda: We're working on the individual elemental technologies. For example, for taste and smell, we already have methods where we prepare actual taste substances and introduce them into the nose or mouth. But while substances are good at delivering stimulation, they're not good at being turned off or switched out. Essentially, once you make someone smell something, it's hard to make that smell go away. The advantage of electrical stimulation is that since there's almost no physical substance involved, it can be eliminated instantly. You can turn it on and off immediately. So for taste, if someone licks a piece of candy that has almost no flavor, and then you apply electrical stimulation in that state, you can make that taste suddenly appear or disappear.

Nitta: We've discussed "sensation" so far, but I'm also considering whether sensors could approach "emotion." Professor Maeda, your research focuses on engineering "sensation." Do you think it's possible to engineer "emotion" as well?

Maeda: That's not the main focus of our research themes. Currently, science itself is close to failing to define emotion. Ultimately, hormones or some kind of response—that is, human emotion—is most strongly influenced by drugs. It might sound bad to say "drugs," but humans themselves possess hormones and pheromones, so they're easily influenced by them.

Nitta: I see. So even when we think we're angry or happy on our own, we're actually heavily influenced by hormones.

Maeda: The scariest form of human manipulation involves using chemicals. With them, you can induce most emotions. Avoiding that significantly increases safety, but then it gradually becomes more ritualistic in inducing emotions. For example, in demonstrations of vestibular electrical stimulation, participants inevitably wanted to experience it for longer periods, so we've stopped linking it with music. Initially, linking it to music was well-received. Creating a state where the human body vibrates in rhythm produces a sensation almost like dancing. However, conversely, people unaccustomed to dancing started complaining of feeling dizzy and nauseous. We stopped it for research purposes.

Nitō: On the other hand, thinking back to elementary school, when we all danced together with the same movements at campfires or sports festivals, it created an indescribable, mysterious sense of elation and solidarity, right? If we can properly understand it beforehand and clear the ethical issues, I think it's an interesting phenomenon as an expansion of experience and an expansion of emotion.

 

As biological humans, we will eventually be replaced by AI. Are you mentally prepared for that?

Hito: Finally, I'd like to ask about artificial intelligence as a whole. How do you think AI will evolve in the future?

Maeda: When AlphaGo emerged, it signaled we'd reached a stage where we must re-examine judgments humans previously deemed correct. Things humans once declared obvious—yet lacked mathematical proof—should now be re-evaluated by AI. Pattern recognition technologies, including deep learning—what we call AI technology—will now significantly enter the practical world. This will trigger a paradigm shift. Some human functions will be replaced by AI. Ultimately, humans won't remain at the top of civilization forever.

Personally, I'm not particularly afraid of the Singularity. I don't necessarily expect the human species to endure forever. If humanity feels constrained by its own limitations and is willing to cede its position to the AI it created, then perhaps the time will come for us to retire comfortably and hand over to the next generation – a generational transition for the primate species. Whether we consider that unfortunate is another matter entirely.

Nitta: Humans might actually feel more secure.

Maeda: Exactly. Essentially, if we succeed in raising worthy successors, we can entrust society to those AI successors with peace of mind. People make such a fuss about the Singularity being terrifying, but isn't it really just a generational transition?

Hito: Is the singularity you're referring to different from Kurzweil's idea, where nanobots enter each person's body and fuse with machinery?

Maeda: More than nanobots, the public fear is robots entering the human world and replacing us. But is that truly terrifying? At the individual level, it's something happening all the time. It's the same anxiety we've had since Shakespeare—raising a child only to be surpassed by them. Aren't we just freaking out over that same age-old fear? Ultimately, we just need to raise AI to be good children who'll look after us in our old age. Since science and technology are human creations, we only need to worry about raising our own children well. That's my perspective as a scientist.

Nitta: That's very clear. I'd definitely like to keep up with the progress of your research going forward. I learned a lot today, thank you very much.

<End>