A social experiment in Shinagawa's office district: "A world where humans and robots coexist"!

Dentsu Inc. International Information Services (ISID) conducted a proof-of-concept experiment for goods delivery between two locations—the Shinagawa Intercity Area in Tokyo and the Keio Shinagawa Building—using an autonomous delivery robot and a remote communication device from March 20 to 23.

This social experiment utilizing robots employed the remote communication device "TiCA (Chika)" currently under joint development by the company's Open Innovation Lab (InnoLab) and the University of Tokyo's Reikimoto Laboratory.

It was an open innovation initiative with technical cooperation from ZMP and Microsoft Japan, and support from Nippon Steel & Sumikin Real Estate, Shinagawa Grand Commons, and Keio Corporation, which operate the facilities and buildings along the travel route.

■Experiment Objective

• Verification of the latest autonomous driving technology toward a society where humans and robots coexist
• Verification of technology to enhance human-robot communication

In the experiment, ZMP's autonomous delivery robot "CarriRo Delivery" stored coffee in a delivery box at a cafe within Shinagawa Intercity, its starting point. Accompanied by technicians, it then autonomously traveled several hundred meters to the Keio Shinagawa Building and delivered the coffee to the Inolab studio.

The route was programmed using 360-degree camera footage of the pre-planned course. Additionally, the robot was equipped with multiple built-in sensors to ensure safe and accurate navigation along the correct path.

Mounted on the upper part of CarriRo Delivery's body is "TiCA," a wearable remote communication device currently under development by Inolab and the University of Tokyo's Rekimoto Laboratory.

TiCA was developed based on the concept of IoA (Internet of Abilities), not IoT. IoA represents an attempt to "extend" the "abilities" of humans or robots in distant locations via a network. TiCA was originally developed as a device worn on a human shoulder to provide support. This experiment involved TiCA adding capabilities that CarriRo Delivery inherently lacks, such as "human-to-human communication via voice and LED" and "visual information via a 360-degree spherical camera."

TiCA, equipped with a 360-degree camera, connects its field of view to a remote operator. When complex communication is required, the remote operator (human) intervenes, enabling TiCA to engage in voice communication with local people. During this interaction, TiCA displays LED expressions corresponding to the operator's gaze information to the person conversing directly in front of it. This design creates "a sense of conversing while making eye contact."

Image of the autonomous CarriRo Delivery and TiCA communicating with pedestrians

During CarriRo Delivery's autonomous operation, various "edge cases" (extreme situations) can occur, such as pedestrians or obstacles blocking its path, or situations where it wants to use an elevator but cannot press the button. In this experiment, remote operators intervened via TiCA in such situations, supporting CarriRo Delivery by communicating with local pedestrians using voice and LED displays.

 

A cafe employee loads coffee onto CarriRo Delivery. The glowing sphere in the upper right is TiCA, responsible for communication.

Once the experiment began, the CarriRo Delivery unit loaded with coffee and its TiCA companion started their autonomous journey toward the INNOLAB Studio inside the ISID building, several hundred meters away. Under the watchful eyes of engineers and media, they proceeded smoothly, navigating branch points without error.

At "potential edge cases" placed along the route, operators intervened with TiCA as needed. TiCA communicated with staff acting as pedestrians to navigate and resolve each situation.

Progressing toward the destination under the watchful eyes of engineers and media. Navigating narrow passages with precision.

One pre-set edge case: encountering a stroller. Operator intervention with TiCA enabled smooth communication with the staff member playing the mother.

Hiroshi Morita, Chief Producer at Inolab, explained the significance of this initiative: "While autonomous delivery robot projects are advancing worldwide, insisting on 100% automation makes commercialization difficult. By having humans supplement the robot's capabilities, as with TiCA, the potential for service implementation expands dramatically. In the future, we aim to equip it with AI to enable simple voice communication automatically based on the situation."

He also emphasized that for CarriRo Delivery and TiCA to perform as envisioned, advancements in 5G networks and GPS are essential. "Once 5G becomes practical, it will enable real-time transmission of TiCA's 360-degree video to remote operators," he noted. "As cities advance toward universal design by 2020, we'll see environments that are not only 'barrier-free' but also 'robot-free'."