Interview with Matsukoroid Developer Professor Hiroshi Ishiguro Part 2: What is the Truly Desired UI According to a Robot Development Pioneer?

In the first part of our discussion with Professor Hiroshi Ishiguro, a leading expert in android research, we explored the potential for growth in the robotics business from his perspective. Following his insightful observations—such as "humans and technology are inseparable" and "robots will increasingly reflect human qualities"—the second part delves into concrete initiatives, including development paths in the service sector and proof-of-concept experiments.

Interfaces that make you feel "I've become smarter"

Nishijima: From our discussion so far (see Part 1), while there are many possibilities, I sense that for companies, the immediate focus will likely be on building systems and going through trial and error for some time. The Ministry of Economy, Trade and Industry's projections for the robotics business market suggest the service sector will surpass manufacturing in 10 years, but it seems we're still in an exploratory phase.

Ishiguro: It's true we can't leap forward overnight, but the service sector surpassing manufacturing is undoubtedly correct. Industrial robots are already quite widespread, and manufacturing has essentially maximized efficiency. While new innovations will emerge, the market expansion in that sector has likely plateaued. Therefore, the only path forward is toward the service industry.

Nishijima: What kind of development directions do you see for the service sector?

Ishiguro: This ties back to what we discussed earlier (in Part 1). For brand strategy, I believe interfaces that make users feel smarter are crucial.
　When people think of robots, they tend to focus solely on functionality. But if we consider them as services used by people with varying levels of technical literacy, the pursuit should be not about specs, but about "how to make people feel comfortable." I think this is especially necessary for robots.
　However, if that comfort devolves into mere hedonism, it becomes problematic. It would be great to have a good example, but if it's just a superficial tweak based on shallow human knowledge or ability, thinking "this will make it easier," without any underlying philosophy, it just spoils people and lacks depth. What is an interface that instinctively draws people in? What does it bring to people? I think we need interfaces that are deeply considered with a profound understanding of human nature.

Nishijima: True, if it's just about being easy, people will get bored or copy it quickly.

Ishiguro: With services, it's not about whether the person actually becomes smarter or more capable. It's all about whether you can make them feel that way and satisfy them. Of course, practical benefits are necessary, but making them feel good is crucial.

Telenoids Prove Effective in Welfare Facilities

Nishijima: Thinking a bit more about robot applications in the service sector, establishing business models in areas like caregiving is highly anticipated. Professor Ishiguro, you've conducted proof-of-concept experiments using the "Telenoid" android—which is the complete opposite of lifelike series—in multiple elderly welfare facilities in Denmark, right?
　Telenoids feature a plush, huggable body paired with a stark white head that minimizes human-like features. Actual conversations are conducted by a person in a separate room via a connected microphone. I understand this approach has proven highly effective.

Ishiguro: Yes. In elderly care facilities, even when caregivers or family members try to converse with residents, many feel "talking to them might be a bother" or "I can't express myself well," making them reluctant to engage and often withdrawn. But when talking to a Telenoid, they can sometimes converse for up to an hour.

The most positive outcome was calming residents who were prone to agitation. Having even one such resident is very challenging in these facilities, and the root cause is often frustration from inability to communicate. So, when they started conversing with the Telenoid, they felt no need to be timid. They became increasingly proactive and their moods stabilized.
　Denmark is a leader in welfare, and we conducted this series of experiments as part of a national project. However, the resistance to robots in the media and public opinion, or concerns like caregivers feeling "robots might take our jobs," aren't so different from Japan. Still, once they saw Telenoid's effects, the caregivers started welcoming it. We've now entered the final experimental phase of determining "how to build the service model."

Nishijima: That's a major step forward. By the way, what stage are the field trials in Japan at?

Ishiguro: We were selected for JST's (Japan Science and Technology Agency) team-based research support program "CREST" and are conducting trials at about 10 welfare facilities. User reactions show high communication-enhancing effects regardless of country. The challenge is that Japan doesn't have the same capacity in care and welfare settings as Denmark. It's simply too busy to continuously introduce new technologies. The service itself is well-received, so we want to explore this further.

Toward Realizing Autonomous Dialogue

Nishijima: Will Telenoid eventually be able to speak autonomously?

Ishiguro: Yes. Currently, an operator speaks through a microphone, but we plan to make it autonomous. At present, no robot can converse autonomously. What we can do now is facial recognition and, like Telenoids, moving the neck and hands in sync with conversation, or moving the mouth in sync with speech. We're staying at that level, so over the next five years, we aim to create a robot that can understand human intent and requests and converse using all manner of gestures.
　This project is part of the "ERATO" program, supported by JST like the previous CREST initiative, which backs highly original research. It's called the " Ishiguro Human-Robot Interaction Project."

Nishijima: That would be incredible if realized. It means the government is finally supporting research into autonomous dialogue for robots.

Ishiguro: We have to get there. Right now, robots are like paper tigers. Expectations keep growing, but the actual progress inside is minimal. For example, there's nothing truly new in actuators.

※Mechanisms that enable forward/backward movement and rotation using hydraulic or electric motors. Here, it refers to the mechanisms that move robots.

For humanoid robots, I think we've made significant progress in hardware, especially sensor technology. But when it comes to what's truly innovative inside? If I had to point to something, it would be improvements in stabilizing speech recognition, gesture control technology, and recognizing human actions. Even though they can talk, they still only understand individual words, and only in quiet environments. As for gestures, you can add all sorts of theatrical ones, but making them naturally match the conversation? That's still impossible.

Nishijima: In that regard, wouldn't you say Professor Ishiguro's remotely operated Geminoid is a success?

Ishiguro: Indeed, it works well except for the autonomy part. That's precisely why, for remotely operated robots, I believe the crucial question is "how and where to implement autonomy" when the operator is distant. There is an auto mode, but currently, it's obvious that it's "saying the same thing every time." That's what I want to change.

Nishijima: Regarding remote operation, for example, the Ministry of Land, Infrastructure, Transport and Tourism is conducting experiments like placing humanoid remote-controlled robots on heavy machinery to operate in dangerous locations. Could that kind of development be possible?

Ishiguro: Absolutely. They're meant to replace humans. However, when considering whether remote-operated robots will expand into ordinary locations beyond hazardous areas, it boils down to a simple cost comparison with human labor. Unless the working population shrinks significantly, I don't think we'll see that level of adoption.

A Future Society Where Human Connections Grow Closer

Nishijima: We've heard a very broad range of topics, from business potential and the importance of interfaces to actual field tests in service sectors. Other projects, like the theater production featuring androids developed with playwright Oriza Hirata (*), are also fascinating. In which direction do you think you'd most like to focus your efforts going forward?

※Seinen-dan International Theater Exchange Project 2014 "Seinen-dan + Osaka University Robot Theater Project: Android Version of 'Metamorphosis'"

Ishiguro: Fundamentally, my interest is quite simple: "How far can we understand 'what it means to be human'?" Even regarding interface principles, it boils down to what principles are necessary for humans to interact naturally.
　For example, there are still many hypotheses that traditional neuroscience and cognitive science cannot prove. Questions like "What is the mind?" or what consciousness is, what emotion is. We need to clarify these things... It's too meta-level to be clear-cut, I think, but we need to get to a point where we can at least "think we understand" a bit more. Otherwise, we can't build robots that can interact properly with people. As a researcher, if it's grounded in these questions, I'll do anything and I think anything goes.

Nishijima: Thinking that way, your wide-ranging projects start to feel cohesive.

Ishiguro: What concerns me a bit about future robot development is the economic aspect. Unlike how society matures or technology advances by addressing problems that need solving, it feels like we're forcing growth.
　Changing the subject slightly, in terms of growth and development, forcing robot development won't lead to a happy outcome either. Even if companies enter the field going forward, we should elevate the stage, keeping this point in mind. For example, Japan should host more serious robot competitions. And when holding such competitions, they must be taken seriously – requiring genuine investment of both people and funds. Not just for fun or spectacle. Something akin to Formula 1 in the automotive industry.

Nishijima: If the government encourages serious competition among companies, technology could advance significantly. Finally, could you share your expectations for a society where robots continue to develop?

Ishiguro: Technological advancement extends human capabilities, naturally leading us to question "What defines humanity?" I envision a future society where new media and robots emerge, yet the time and depth of human interaction actually increase. In that context, a society incorporating robots doesn't seem so bad. I hope my work helps shift perceptions about robots.

Nishijima: I agree that as technology advances, people become more interested in each other. Thank you for your valuable insights.