NTT's Image Recognition AI Accurately Detects Rust in Social Infrastructure Facilities
~ Reducing operational costs through single-run inspection of facilities ~

NTT Corporation (TYO: 9432) has developed an image recognition AI that successfully detects rust on a variety of social infrastructure facilities with high accuracy. The image recognition AI can distinguish multiple types of infrastructure facilities from images of roadside facilities taken by NTT's Mobile Mapping System (MMS)^*1 to detect rust that has formed on the facilities (roadside facilities and utility pole-mounted facilities). The system boasts a 97.5% detection accuracy.^*2

This technology will be exhibited at Tsukuba Forum 2022^*3 on May 18 and 19, 2022.

1. Background

Social infrastructure such as roads are facing issues including deterioration due to age, increasing inspection costs, and a shortage of inspection personnel. To address this situation, the Japanese government's Future Investment Strategy 2018^*4 seeks to increase the percentage of infrastructure managers who deploy new technologies such as sensing and AI to 100% by 2030. The NTT Group is working to reach this goal by utilizing 4D digital platform™^*5, which integrates, accumulates, and analyzes a variety of sensing data obtained from society in digital space. We are carrying out inspections (data acquisition and automated analysis) using MMS, drones, and other technologies.

NTT has developed image recognition AI technology that detects rust on facilities from captured images. This technology allows facility inspectors to capture images of many roadside infrastructure facilities in one run by using multiple high-resolution MMS cameras to capture images of roadside infrastructure facilities.

2. Testing and Results

Using MMS, Nippon Telegraph and Telephone West Corporation (NTT West) captured images of actual installed infrastructure facilities. The MMS used for acquiring images is shown in Figure 1. For this trial, MMS photographed roadside infrastructure facilities at regular time intervals. The system's side-facing digital cameras took images of roadside facilities (guardrails, signs, mirrors, etc.) and upward-facing cameras captured images of utility pole-mounted facilities (hardware, cables, etc.). The image recognition AI was applied to 1,000 images of roadside facilities (of which 587 images showed rust) and 1,000 images of utility pole-mounted facilities (of which 135 images showed rust). The AI was trained to recognize the type of facility and detect rust.

The results of the experiment are shown in Table 1. For all the images of roadside facilities and utility pole-mounted facilities, the AI technology correctly recognized the type of facility in 1,885 of the 2,000 total images (recognition rate of 94.3%^*6) and correctly detected rust in 704 of the 722 images of facilities with rust (detection rate of 97.5%^*2). Figure 2 shows an example of facility recognition and rust detection.

NTT's newly developed recognition AI makes it possible for separate field inspections currently performed by each infrastructure manager to be consolidated into one MMS run. It is expected to reduce the operational costs of inspections. Furthermore, uniform inspection quality can be achieved because the image recognition AI uses uniform standards to detect rust from acquired images with high accuracy.

Figure 1. MMS used to photograph infrastructure facilities

Figure 2. Results of infrastructure facility recognition and rust detection

3. Features of the Image Recognition AI

The image recognition AI used in this test has the following features to realize high-accuracy detection (Figure 3).

In addition, because attributes of roadside facilities and utility pole-mounted facilities are assigned on a pixelwise basis, it is possible to determine the type or part of the facility where rust has formed.

This image recognition AI is trained on several tens of thousands of actual images of infrastructure facilities in the field and images of rust that has formed on facilities. It thus can be used by infrastructure managers to inspect facilities without being limited to specific facilities.

Figure 3. Features of image recognition AI

4. Future Development

We will move forward with the practicalization of rust detection using our newly developed image recognition AI for images taken by MMS and drones. Furthermore, we are aiming to realize efficient maintenance of all social infrastructure by using digital information. Besides rust detection, our image recognition AI can be used to detect deformities and estimate with high accuracy longitudes and latitudes of installed facilities using GNSS information and images captured by MMS.

^*1:Car-mounted system with equipment such as LIDAR (Light Detection and Ranging), GNSS (Global Navigation Satellite System), and digital cameras to capture spatial information such as 3D coordinates and images of roads, buildings, and facilities as the car is driven through an environment.

^*2:Percentage calculated by using the number of images of rust correctly detected by the image recognition AI as the numerator and the total number of visually-confirmed images of rust as the denominator.

^*3: URL (https://www.tsukuba-forum.jp/e/). This technology will be exhibited under the title "Smart Operation and Maintenance of Social Infrastructure."

^*4:Cabinet Secretariat, "Future Investment Strategy 2018," URL (https://www.kantei.go.jp/jp/singi/keizaisaisei/pdf/miraitousi2018_zentai.pdf)

^*5:"4D digital platform™" is a registered trademark of NTT.

^*6:Percentage calculated by using the number of images of facilities (roadside facilities or utility-pole mounted facilities) correctly recognized by the AI as the numerator and the total number of facility images as the denominator.