Developed next-generation computing infrastructure that enables optimal use of diverse computing resources
- Four times power efficiency improvement confirmed in video camera analysis of smart cities -

Nippon Telegraph and Telephone Corporation (NTT, President & CEO: Akira Shimada) has developed a new computing infrastructure that flexibly connects various computing resources and optimally uses only the resources required for computing. As a use case, we have confirmed that power efficiency can be increased up to approximately four times ^*1 in video camera analysis of smart cities. For the next step, we will expand the application areas of the computing infrastructure we have developed, and contribute to the realization of carbon neutrality by further improving power efficiency through the implementation of photonics-electronics convergence technology for the interconnect among computing resources.
　This technology will be presented at the NTT R&D Forum -- Road to IOWN 2022 ^*2 to be held from November 16th, 2022 to 18th.

1. Background

As shown in the Society 5.0 by the Cabinet Office of Japan, it is expected that AI analysis and other methods will generate new value for industries and society from the vast amount of data generated in real space in the future. However, for example, real-time analysis of a large number of camera streams in a smart city requires a large system for transferring and analyzing large amounts of data, which consumes a lot of power. To solve this problem, various accelerators such as GPUs have been developed to improve the power efficiency of analysis, but with traditional computing infrastructure, there have been issues such as (1) CPU load becomes a bottleneck because the CPU is responsible for transferring data to the accelerator, so that the performance of the accelerator cannot be fully utilized, and (2) it is difficult to adaptively allocate the most appropriate accelerator at all times in response to changes in camera images, resulting in a situation where the assigned accelerator is not fully effective.

2. Overview of the computing infrastructure developed

In response to these challenges, NTT has developed a next-generation computing infrastructure that enables various computing resources to be linked each other flexibly and allocated to each server.
　An overview of the computing infrastructure developed is shown in Figure 1. This computing infrastructure mounts several different accelerators, such as GPUs, FPGAs, etc., configures resource pooling for each accelerator type ^*3 , and allocates as many optimal accelerators as needed, depending on the process executed for each application. In addition, this computing infrastructure provides data transfer between accelerators without going through the CPU, preventing CPU bottlenecks caused in massive data transfers.
　NTT has been studying the architecture of this next-generation computing infrastructure at the IOWN Global Forum ^*4 in collaboration with its members. As a result, an architecture document entitled "Data Centric Infrastructure (DCI)" was published by the organization in January 2022. ^*5.
　Using this computing infrastructure, we evaluated a video camera AI analysis system for smart cities ^*6. As shown in Figure 2, we have demonstrated that by allocating the necessary number of optimal accelerators according to scene changes, the power consumption per camera can be reduced from an average of about 320 W to about 120 - 80 W (up to about 2.5 - 4 times more power efficiency) compared to a conventional system ^*1. (Figure 3)

Figure 1: Overview of the computing infrastructure developed

Figure 2: Processing example of this system in use case of camera image inference processing

Figure 3: Power Monitor Screen Results (Measured Power Reduction Effect)

3. Future developments

Together with the members of the IOWN Global Forum, we will expand the application of this computing infrastructure and aim to introduce it in various fields. At the same time, the third-generation photonics-electronics convergence device ^*7 being developed by NTT will be applied to the data transfer path between the accelerators with the aim to increase the power efficiency 8 times by 2025, and commercialization in 2026. Furthermore, by implementing the fourth-generation photonics-electronics convergence device ^*8, we will complete the configuration with optical backplane inside the system, aiming for full-scale business development of systems capable of further power reduction.

^*1: Power savings compared to power consumption measured for typical CPU and GPU-based server configurations as of 2020, when the IOWN roadmap was formulated. Including hardware evolution. The effect also depends on the input video scene (e.g., the percentage of time a person is shown).

^*2: NTT R&D FORUM - Road to IOWN 2022
URL: https://www.rd.ntt/e/forum/

^*3: Managing multiple computational resources, such as accelerators, in a server together so that resources can be used as needed.

^*4: IOWN Global Forum: An international forum established in 2020 to realize the IOWN concept. It currently has 109 members.
URL: https://iowngf.org/

^*5: Data-Centric Infrastructure Functional Architecture, IOWN Global Forum, January 27, 2022.

^*6: This result utilizes technology from joint research and development with Fujitsu Limited.
URL: https://www.fujitsu.com/global/about/resources/news/press-releases/2021/0426-01.html

^*7: Photonics-electronics convergence devices for board connections. Mounted on a switchboard.

^*8: Photonics-electronics convergence devices for chip connections. Implemented in a package.