Initiatives to Enrich Communication for ALS Symbionts
To create a world where people with ALS can freely move, express themselves and interact with others

Tokyo –June 14th, 2023 – NTT Corporation (NTT), in cooperation with WITH ALS and Dentsu Lab Tokyo, through research and development, aims to realize a world where people living with amyotrophic lateral sclerosis (ALS)^[1] can be freed from physical limitations caused by ALS progression, and communicate with others freely.

1. Background and Purpose

"Communication is the most important thing." These are the words of people living with ALS. As ALS progresses, cognition remains normal, and muscle strength throughout the body gradually loses function. It is said that by putting on a respirator, you can live your life to the full term, but on the other hand, by tracheostomy surgery to put on a respirator, you lose your voice. The choices made to keep living can also lead to loss of speech.
　Because of the loss of the means of communication through spoken language and physical expression, many people fear disconnection from society and lose hope of living. Globally, more than 90% of ALS patients refuse to wear a ventilator.
　NTT is promoting Project Humanity, which aims to solve problems by focusing on people, not only those with diseases and disabilities, but also those who support them. This initiative, as part of Project Humanity, aims to change the current situation surrounding ALS for people living with ALS through the implementation of our communication technology.

2. Past Initiatives

Advances in speech synthesis technology have made it possible to record the voice of a person and realize synthetic speech with a voice that is unique to the person. However, as ALS symptoms develops, people often have difficulty speaking when they decide to record their voice. In addition, current speech synthesis technology requires the voices to be recorded in a proper recording environment, and it is a significant burden to obtain voice recordings. For this reason, there is currently a huge barrier for people living with ALS to use their own synthetic voice.
　A year ago, we succeeded in reproducing a person's voice using speech synthesis technology from the audio of a few minutes of recorded video. This technology enables communication in multiple languages with the voice of the individual. Last year, we partnered with WITH ALS and Dentsu Lab Tokyo to use this technology on the Cannes Lions stage. A Japanese ALS symbiont who cannot speak has experienced dialogue and musical performance in English with his own tone of voice.
　And this year, we succeeded in creating synthetic voices of multiple ALS symbionts from even fewer recordings and recordings of a few seconds. Even if you are not able to speak now, you can still create a synthetic voice if you have the audio recording of your voice before you lost your voice. The synthesized audio created with this technology will be used in a live music performance at the ALS Awareness Music Festival "MOVE FES." hosted by WITH ALS on June 18, in conjunction with World ALS Day on June 21.

[MOVE FES. 2023 Supported by AIRU]

Date and Time: Sunday, June 18, 2023, 17:00 ~ 21:00 (Open 15: 30)
　Venue: EX THEATER ROPPONGI
　(1 -2-9 Nishiazabu, Minato-ku, Tokyo 106-0031)
　Ticket Information: https://withals.zaiko.io/e/movefes2023
　Hosted by: WITH ALS

3. Future Initiatives

In the future, we plan to expand nonverbal expressions so that people living with ALS can communicate more freely. First, we will further develop the motor-skill-transfer technology based on NTT's biometric information and work on the following.
1. Free manipulation of avatars by ALS symbionts in metaverse space
2. Reproduce ALS symbiont's motion in real space
　In the operation of the avatar (Figure 1), the body is equipped with a myoelectric sensor that acquires biological information, and the biological information obtained by the minute muscular activity of the body is converted into operational input, thereby realizing to operate the avatar freely. Dentsu Lab Tokyo is responsible for creating rich expressions. In the reproduction exercise (Figure 2), the muscles are controlled by presenting electrical muscle stimulation to the body to achieve the reproduction of intended motion.

Figure 1 Avatar Operation Using a Myoelectric Sensor

Figure 2 Nonverbal Communication Using Muscle Electrical Stimulation

Regarding avatar representation, we will use body motion generation technology to realize the nonverbal representation of the ALS symbiont as well as the speech of synthetic speech. Body motion generation technology automatically generates the corresponding motion of a person's speech by using a motion generation model constructed from the characteristics of speech and motion data collected at that time.
　We will also consider cooperation with technologies available for ALS symbionts. OriHime eye+Switch, a communication device of Ory Lab Inc. ^[2], which supports sight input, enabling communication by operating the OriHime robot. In this way, the range of communication is further expanded through robots.

4. Future Roadmap

This fiscal year, we aim to realize the technology that can freely operate avatars from the slight body movement of ALS symbionts, as well as to realize communication using muscle electrical stimulation to add body movement expression. In FY2024, we will further develop communication expressions so that people can experience interaction without feeling the limitation of ALS in a metaverse space or in a real space via robots. We will ultimately contribute to a future in which all people can interact with each other without barriers through our innovative communication.

<Reference>

^[1]ALS (amyotrophic lateral sclerosis): A disease in which the motor nerve is damaged and the command from the brain to the muscle is not transmitted, and the muscles of the whole body gradually become difficult to move. Even if you can't move your body freely, your brain still functions, and your consciousness and thoughts are still clear.

^[2]Ory Lab Inc.: A company whose mission is to realize a new form of social participation through communication technology, eliminate loneliness among people, and expand the potential of society itself. In 2020, Ory Lab Inc. entered into a capital and business alliance with NTT.

About WITH ALS

Founded in 2016 by Masatane Muto, who is facing amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig's disease. The mission of WITH ALS General Incorporated Association is to raise awareness of the disease and promote advocacy in search of new treatments and support systems.
It also develops action plans and provides information to build hope and enhance the quality of life of those with the disease and their families. https://withals.com/

About Dentsu Lab Tokyo

Dentsu Lab Tokyo is a creative R&D organization that combines research, planning, and development. Under the philosophy of "PLAYFUL SOLUTION" and "Be Irregular", we use digital technology and ideas to develop expressions that move people's minds and solve social issues that the world demands today. https://dentsulab.tokyo/.

About NTT

NTT contributes to a sustainable society through the power of innovation. We are a leading global technology company providing services to consumers and business as a mobile operator, infrastructure, networks, applications, and consulting provider. Our offerings include digital business consulting, managed application services, workplace and cloud solutions, data center and edge computing, all supported by our deep global industry expertise. We are over $100B in revenue and 330,000 employees, with $3.6B in annual R&D investments. Our operations span across 80+ countries and regions, allowing us to serve clients in over 190 of them. We serve over 75% of Fortune Global 100 companies, thousands of other enterprise and government clients and millions of consumers.