Demonstration Experiment of Remote Robot Operation Conducted Using Local 5G and Communication Quality Control Technology at Avatar Robot Cafe DAWN ver.β

Realizing natural communication that supports the social advancement of people with physical disabilities

Tokyo - January 17, 2023 - NTT East Corporation (President and CEO: Naoki Shibutani, "NTT East"), NTT Corporation (President and CEO: Akira Shimada, "NTT"), and OryLab Inc. (Co-founder CEO: Kentaro Yoshifuji, "Ory Laboratory") are cooperating on the permanent experiment shop "Avatar Robot Cafe DAWN ver.β¹" operated by Ory Laboratory. In the demonstration experiment, the avatar robot "OriHime-D²" was used in the cafe and operates using Local 5G³, which is a high-quality (large-capacity, low-latency) wireless access environment provided by NTT East, and a communication quality control technology developed by NTT. By combining these technologies that control the communication quality between terminals, NTT East and NTT have realized natural remote robot operation that does not cause time lag due to communication latency for remote operators.

The results of this demonstration experiment are expected to expand the application areas of avatar robots such as OriHime-D and enable natural communication through low-latency applications. These developments will also broaden employment and other opportunities for people who are unable to leave their homes due to disabilities, illness, or other reasons and promote new work styles that are not confined to location.

Background

NTT has sponsored for Avatar Robot Cafe DAWN ver.β, a permanent experiment launched by Ory Laboratory on June 21, 2021 and cooperating to conduct joint demonstration experiments.⁴ The cafe allows people who have difficulty going out due to disabilities or illness to provide customer services through an avatar robot.

Human-operated avatar robots such as "OriHime-D" are expected to be able to perform various actions depending on the location and situation, from communicating by conversation and facial expressions through audio and visuals to producing precise movements while avoiding people and obstacles. Currently, the avatar robots are wirelessly connected via the Wi-Fi installed in the cafe. However, quality degradation due to radio disturbance has caused time lags and communication interruptions which brought increase of operator stress, reduction of operational accuracy and interference with cafe services. Therefore, realizing natural robot operation in a cafe demands the development of a wireless access environment that is not easily affected by other radio waves and a reduction of communication latency between the avatar robot and the operator, including the wireless communication section.

To solve these problems, we built a Local 5G, a high-quality wireless access environment in the cafe and conducted a demonstration experiment on remote robot operation by combining with NTT's communication quality control technology (Fig. 1).

Fig. 1: Demonstration experiment configuration diagram

Initiative Overview

Building Local 5G, a high-quality (large-capacity, low-latency) wireless access environment

Local 5G is a 5G system that local companies and local governments can flexibly build and own. Local 5G makes it possible to build a stable wireless access environment, as it is resistant to deterioration of communication quality due to interference with other radio waves (disruption of images due to disconnection and decreased throughput, etc.). In addition, it features high-speed, large-capacity, low-latency communication, and customization such as high-speed uplink and priority control of specific terminal communication.

Communication quality control technology that realizes natural remote control

For wireless access provided by technologies such as LTE, 5G, Wi-Fi, etc., wireless communication quality control technology according to each wireless communication standard is adopted, and it is necessary to make settings according to the adopted standard. In recent years, a variety of applications have emerged for different purposes, and the requirements for network communication quality, such as communication speed and latency amount, are diversifying. Therefore, assurance of communication quality has become a pressing issue. To solve these issues, NTT has developed a new communication quality control technology that centrally controls the communication priority of communication packets exchanged between applications.

This communication quality control technology consists of a communication packet volume estimation function and a communication packet transmission timing control function. The communication packet volume estimation function predicts the volume of communication packets that will flow through the network in the future based on the communication status of the wireless network and the past communication status of each application when each communication packet was input into the wireless network. In addition, the transmission timing control function controls the transmission timing of communication packets so that the required amount of latency is achieved according to the priority of communication packets for each application. By combining these two functions, this technology achieves communication quality control between applications in communication layer 2 (the data link layer) and communication layer 3 (the network layer). In addition, these functions can be implemented in various devices and wireless access points, and are characterized by their ability to control communication quality without depending on the wireless network communication standard or wireless communication equipment used.

The Demonstration Experiment

In the demonstration experiment, NTT Musashino R&D (Musashino City, Tokyo), NTT Central Training Institute (Chofu City, Tokyo), and the Avatar Robot Cafe DAWN ver.β (Chuo-ku, Tokyo) were connected by optical fiber with a total length of 100 km, via the Local 5G installed in the cafe. We evaluated the robot operability and the network performance between applications when an operator with disabilities performed service staff work at a cafe from the Musashino R&D Center.

By utilizing Local 5G, we eliminated issues in remote control due to deterioration in video quality and confirmed the effect of improving the operability of the avatar robot. In addition, we confirmed that the communication quality control technology developed by NTT can maintain low-latency performance without being affected even under conditions where large-capacity dummy traffic that usually causes an increase in latency of several tens of milliseconds or more is added to the Local 5G. With the cooperation of NTT Claruty Corporation⁵, we also interviewed the operators of the avatar robots. They confirmed that we could reduce the stress related to operating the equipment and that they could operate the robot and interact with customers smoothly without feeling any network latency.

Fig. 2: Demonstration experiment at the Avatar Robot Cafe (left), Operator at the NTT Musashino R&D Center (right)

The Target and Role of Each Company

Future Developments

The results of this demonstration experiments are not limited in application to the avatar robots that were tested this time but are also expected to be applied to other remote operation use cases that require low-latency network performance, such as remote-controlled industrial robots that are active in manufacturing factories and construction sites.

In the future, we will further expand the application areas of avatar robots such as OriHime-D, equip various devices with the communication control technology utilized in this achievement, and combine them with Local 5G to apply them to factories, construction sites, and farms. Through these technologies, we will realize a future where everyone can participate in society regardless of disabilities and embody new ways of working.

Notes:

^{1 2}About the Avatar Robot Cafe DAWN ver.β and OriHime-D:
An experimental cafe presided over and operated by Ory Laboratory, where people who have difficulty going out due to chronic illnesses such as ALS or severe disabilities work as service staff by remotely controlling the avatar robots "OriHime" and "OriHime-D". "OriHime-D" is an avatar robot with a total length of about 120 cm, which can be remotely controlled to perform tasks that involve physical labor, such as customer service and delivery. "Avatar Robot Cafe" and "OriHime" are registered trademarks of Ory Laboratory Co., Ltd.

³About Local 5G (NTT EAST's activities regarding Local 5G):
https://business.ntt-east.co.jp/solution/local5g/(in Japanese)

⁴About the joint experiment:
"Cooperation on Avater Robot Cafe DAWN ver.β: Conducting a demonstration experiment of remote robot operation toward the realization of the IOWN concept" https://group.ntt/jp/newsrelease/2021/06/17/210617a.html

⁵About NTT Claruty:
A special subsidiary of the NTT Group where employees with disabilities work together
https://www.ntt-claruty.co.jp/