The development of artificial intelligence (AI) has brought about large-scale, pre-trained "foundational" models that can be used for a number of applications, such as generative AI. Although vital to today's society, such models require periodic updates and tuning, which is both time-consuming and costly. Updating and tuning AI models involves preparing data, selecting a base model, configuring settings, and running the training process. It's a detailed exercise, requiring both huge computational resources and know-how, where the main challenge is the need to re-train models whenever there are updates to improve performance or address issues like copyright and privacy vulnerabilities. Until now, this frequent need for tuning has been a barrier to the more widespread adoption and operation of foundational AI.

NTT is working to make the process much easier and quicker, for less money.

The company's "learning transfer" technology has worked out how to use past learning trajectories to greatly reduce the cost and time associated with re-training AI models. Learning transfer takes advantage of the high symmetry in the parameter space of neural networks, specifically through the concept of permutation symmetry, where the order of neurons or layers in a neural network can be changed without altering performance or output. The parameter space of neural networks refers to all possible values for the parameters that can be set in a neural network. Permutation symmetry allows for the re-use of parameter sequences from previous learning trajectories across different models, making the tuning process far more efficient.

Deep learning works by optimizing the parameters of a neural network model using a training dataset. The changes in these parameters over time form a learning trajectory, which is influenced by variations in the initial values put in and randomness. Until now, learning trajectories have been unique to each model due to these variations. NTT's research, however, has shown that under permutation symmetry, the learning trajectories of different models can be nearly identical. By transforming past learning trajectories with appropriate permutation symmetries, NTT's learning transfer technology allows new models to achieve high accuracy with minimal additional learning.

Doing this enables faster convergence to the target accuracy, compared to standard learning methods, which reduces the tuning costs for updating or changing foundational models.

Potential benefits of learning transfer extend beyond just cost reduction. It also has the potential to reduce the environmental impact of AI operations by lowering the computational power required for re-training models. This is particularly important when it comes to sustainable AI development, where the goal is to balance technological advancement with environmental responsibility.

What's more, learning transfer may play a major role in the development of next-generation AI technologies, such as the AI Constellation concept. AI Constellation involves a collaborative network of AI models, including LLMs and rule-based models, which work together to produce diverse and robust solutions. By enabling easier updates and maintenance of these foundational models, learning transfer supports the dynamic and evolving nature of AI ecosystems.

Learning transfer means that companies can use generative AI solutions without being slowed down by the high costs of model tuning. It opens up new possibilities for industries where AI models need to be frequently updated to remain effective and secure, such as:

- Medical Diagnostics: AI models used for diagnosing diseases from medical images (like X-rays or MRIs) need frequent updates to include new medical knowledge. Learning transfer can reduce the cost and time of updating these diagnostic models, ensuring they stay current with the latest medical advancements.

- Fraud Detection: Financial institutions rely on AI models to detect fraudulent transactions. With learning transfer, these models can be rapidly updated to adapt to new fraud patterns, improving security and reducing losses.

- Chatbots and Virtual Assistants: AI-driven customer service bots can be updated to handle new types of customer queries or changes in company policies, without the need for complete retraining, ensuring consistent and accurate responses.

These are just three examples, but the possibilities are extremely diverse. Easier, quicker and cheaper AI retraining also has implications for personalized medical treatment plans, algorithmic trading, sentiment analysis, navigation systems, object detection, predictive maintenance and quality control.

NTT's learning transfer technology is a big step forward in the field of AI. By re-using past learning trajectories, it reduces the cost and time required for tuning AI models, promotes sustainability, and supports the development of collaborative AI networks like the AI Constellation. Not only enhancing the efficiency of AI operations, but also paving the way for broader and more impactful applications of AI across a wide range of sectors.

NTT—Innovating the Future of AI

For further information, please see this link:
https://group.ntt/en/newsrelease/2024/05/07/240507b.html

If you have any questions on the content of this article, please contact:
NTT Service Innovation Laboratory Group
Public Relations
nttrd-pr@ml.ntt.com

Daniel O'Connor

Daniel O'Connor joined the NTT Group in 1999 when he began work as the Public Relations Manager of NTT Europe. While in London, he liaised with the local press, created the company's intranet site, wrote technical copy for industry magazines and managed exhibition stands from initial design to finished displays.

Later seconded to the headquarters of NTT Communications in Tokyo, he contributed to the company's first-ever winning of global telecoms awards and the digitalisation of internal company information exchange.

Since 2015 Daniel has created content for the Group's Global Leadership Institute, the One NTT Network and is currently working with NTT R&D teams to grow public understanding of the cutting-edge research undertaken by the NTT Group.