NTT announced that it will start offering its proprietary Japanese Large Language Model (LLM)¹ "tsuzumi²" to businesses aiming for operational efficiency and digital transformation (DX) promotion, starting from March 2024. This information was conveyed at a press conference held on November 1, 2023.

The model has already begun to be implemented in the medical and contact center sectors, contributing to operational efficiency and enhancing employee experience/customer experience (EX/CX).

At the press conference, NTT President and CEO Akira Shimada emphasized, "As we pursue sustainability, it's important to solve social issues such as reducing power consumption. 'tsuzumi' is positioned to solve these social challenges."

Shingo Kinoshita, Executive Officer and General Manager of the Research & Planning Division, noted, "As LLMs become larger in scale, they require a huge amount of energy for learning. 'tsuzumi' leverages its ultra-lightweight nature to pursue reduced learning and inference costs necessary for LLM operation."

¹Large-scale language model (LLM): A language model trained using a large amount of text data, which possesses superior capabilities in understanding language and generating text.

²tsuzumi: "tsuzumi" is currently in the process of trademark application. Focusing on the processing performance of the Japanese language, it represents the expectation for language model technology that drives industrial development, similar to how the tsuzumi drum initiates the start of a Gagaku (ancient Japanese court music and dance) ensemble.

1) What is tsuzumi?

For over 40 years, NTT has been engaged in research and development of technologies that allow computers to analyze and process natural language used by humans in everyday life. NTT Laboratories is a leader in natural language processing research in Japan, leveraging its high-precision language processing technology.

In recent years, ChatGPT and other LLMs have attracted much attention. The application range of these models is rapidly expanding, and they are being used for input support, speech recognition, transcribing conversations into text, and automated responses in a variety of fields, including medicine, finance, insurance, and call centers.

While LLMs hold the promise of improving the efficiency of specific business processes, current LLMs require a significant amount of electricity for training. It is said that a GPT-3-scale LLM requires the equivalent of one hour of power from one nuclear power plant3 per training session. 4 It also requires a large GPU cluster to operate, and the cost of tuning and inference for specific industries is enormous. Therefore, sustainability and reducing the costs of preparing the training environment for LLMs have become significant challenges.

³Approximately 1300 MWh for a GPT-3-scale model with 175 billion parameters ⁽¹⁾, which is the same amount of electricity as one nuclear power plant for one hour (approximately 1000 MWh).

⁽¹⁾ https://gizmodo.com/chatgpt-ai-openai-carbon-emissions-stanford-report-1850288635

NTT has been conducting research and development of LLM to solve these issues and announced that it will start providing "tsuzumi," a large language model with lightweight yet world-class Japanese processing capabilities, from March 2024. In preparation for commercial launch, we have started trials with Kyoto University Hospital in the medical field and Tokio Marine & Nichido Fire Insurance Co., Ltd. in the contact center section.

2) Features of tsuzumi

One of the key features of "tsuzumi" is its lightweight model. tsuzumi, developed by NTT, is available in two versions: an ultra-lightweight version with a parameter size of 600 million (0.6B) and a lightweight version with a parameter size of 7 billion (7B), which are 1/300 and 1/25th the size of Open AI's GPT-3's 175 billion (175B), respectively. The lightweight version is designed to perform high-speed inference on a single GPU, while the ultra-lightweight version can do so on a CPU. This design significantly reduces the costs required for training, inference, and tuning.

NTT Laboratories has more than 40 years of experience in natural language processing research, and its research capabilities in the field of AI are among the best in the world. NTT Laboratories also ranks first among Japanese companies in the number of papers accepted for publication at top conferences in the field of natural language processing, and first in the number of excellence awards received by the Association for Natural Language Processing (over the past 10 years).⁴

⁴Top 100 Global Companies Leading in AI Research in 2022
https://thundermark.medium.com/ai-research-rankings-2022-sputnik-moment-for-china-64b693386a4

When using an LLM, the cost of relearning all the huge number of parameters when trying to learn new knowledge can significantly increase the computational learning cost. To address this, "tsuzumi" introduces adapters tailored to the specific needs of users and scenarios. For example, it is possible to tune for linguistic expressions and knowledge specific to a particular industry with a small amount of additional learning. This approach allows for more efficient and targeted updates to the model's capabilities without the need for extensive retraining.

We also plan to support multimodality, which enables correct understanding of diagrams and graphs contained in presentation slides through modal expansion of language + vision. For instance, it will be possible to use this for tasks that require human-like cognition, such as searching and screening documents with images like invoices or manuals, or using AI to evaluate product descriptions and pricing plans posted on websites.

Furthermore, modal augmentation of language + vision + hearing makes it possible to generate responses based on the questioner's demeanor in addition to verbal questions. For example, if a child speaks in a low-spirited voice, the system can understand the "low-spirited situation" as it is and respond with a warm and gentle voice. This capability has potential applications in counseling and call centers, where it can provide automated responses tailored to various user situations. It can also be applied to concierge services such as car navigation systems and smartphone navigation systems, using the user's location information and area status.

3) Examples of "tsuzumi" and industry-specific areas

Medical Domain

Through the structuring of electronic medical records, it has become possible to utilize medical record information. Furthermore, structured electronic medical record data facilitates the analysis of medical data, such as the effects of medication and side effects. This is expected to contribute to reducing the duration and cost of pharmaceutical development and lead to more effective drug development.

Customer Support Domain

Our goal is to reduce the annual 800,000 hours of after-call work by more than 50% in response to non-life insurance accidents.

Regarding commercial service provision from March 2024, NTT Group companies will sell the service to corporate customers. We are also considering offering the service in the form of solutions including system integration, which will be customized and provided according to customer needs.

Even after the launch of commercial services, NTT Group will continue to enhance tuning functions and roll out multimodality, and promote the development of applications in the cyber security field and an AI constellation that can autonomously collaborate and discuss issues.

4) Sustainability and Social Contribution

At the core of "tsuzumi" is the goal of solving social issues. We will contribute to sustainability by being conscious of power saving and eco-friendliness.

NTT will further accelerate its efforts to create new value and enhance the customer experience.

Please look forward to the future development of "tsuzumi" services.

5) NTT R&D FORUM 2023 - IOWN ACCELERATION

tsuzumi" was exhibited at "NTT R&D FORUM 2023 - IOWN ACCELERATION," held from November 14 (Tuesday) to 17 (Friday), 2023.

Through lectures and exhibits, NTT Group introduced the specific services, systems, use cases, and elemental technologies of IOWN (Innovative Optical and Wireless Network), which started service in March this year, as well as the latest results of NTT Group R&D in an easy-to-understand manner.

https://www.rd.ntt/forum/2023/

■Large-scale language model (LLM) "tsuzumi" explanatory information

News Release
Commercial Service Using NTT's Unique Large-Scale Language Model "tsuzumi" to be Available in March 2024

NTT's version of the large-scale language model "tsuzumi" | NTT R&D Website
https://www.rd.ntt/research/LLM_tsuzumi.html

For inquiries from the media, please contact

Nippon Telegraph and Telephone Corporation
Public Relations Division
ntt-pr@ntt.com