1) Highlights of Open House 2024

NTT Group held "Open House 2024," an event to show the latest R&D results of the Communication Science Laboratories to the public, between June 24-26, 2024 at NTT West QUINTBRIDGE and PRISM (located in Osaka City, Osaka Prefecture). The event was hosted by NTT Communication Science Laboratories ("NTT CS Labs").

The theme of this year's Open House was "Let's talk about future communication," and the latest research and development technologies (22 items in total), such as 3D image presentation using optical illusions and real-time voice conversion, were exhibited and research lectures were given.

2) Introduction of Exhibits

1. Seeing Sound with Light

Using optical technology to visualize sound waves in high definition, to analyze in detail the nature of sound, such as how sound is generated and transmitted, using a high-speed camera, laser light, and AI processing.

Sound travels through the air as waves, like ripples on the surface of water, but it is not possible to directly observe the propagation of sound. In contrast, this technology uses optical technology and a high-speed camera to capture sound waves transmitted through the air during atmospheric pressure changes, as moving images of minute modulations of laser light. In addition, the sound field image noise reduction technology based on deep learning removes noise to achieve highly accurate visualization of sound.

Sound is a familiar part of our daily lives, but there remain many unsolved research issues, such as what is the ideal sound and how to evaluate it. By using this technology to visualize subtle differences in sound that cannot be detected simply by listening, we aim to create a comfortable sound environment by promoting higher quality and differentiation of acoustic products such as speakers and headphones.

2. Instantly change your voice and speaking style to your preference

Technology for real-time conversion and generation of voice color and voice attributes with high quality and low latency. Traditional voice conversion technologies required waiting for the end of a speech segment before converting it, making it impossible to change the voice mid-conversation. However, this technology makes it possible to convert a speaker's voice into another speaker's voice during live streaming and to make the voice more understandable in web conferences and call centers.

In addition to linguistic information, speech contains various complexly intertwined factors such as gender, voice pitch, and emotion. This technology uses a deep generative model that takes into account differences among speakers in voice inflection and voice quality to untangle these factors and achieve high-quality, comfortable voice expression.

This technology can be applied to a wide range of voice communication applications, both face-to-face and remotely, such as live broadcasts and enhancing speaker confidentiality in call centers.

3. How often do major infrastructure failures occur?

A technology that calculates the failure rate by scale, without approximations, due to frequent failures of infrastructure components like cables and wires. In network infrastructures such as telecommunications and electric power, highly reliable designs are required to minimize the failure rate for each scale of failure to avoid large-scale disruptions. However, conventional technologies require an enormous amount of time to examine all failure patterns and user disruption patterns.

This technology uses a data structure called a decision graph to quickly and precisely determine failure rates by scale, eliminating the redundancy of repeated calculations inherent in traditional methods. This allows for the calculation of scale-specific failure rates in under a second for networks the size of a prefecture, achieving speeds over 100,000 times faster than conventional methods.

This technology helps ensure that the designed infrastructure meets the high reliability standards required of modern network infrastructures. We are aiming for a future in which network infrastructure designs are more resistant to large-scale failures.

4. Feelings for friends expressed in pictures

A technique for measuring children's intimacy with others based on their drawings in a simple and natural way. Children aged 3 to 6 were asked to draw pictures of themselves and others, and the distance between the nearest points of the drawn figures correlated to the closeness that the children felt towards those people. This is the first study in the world to show that drawing is a simple method to measure intimacy in young children.

The closeness that young children feel toward others is the foundation of friendships and is an important theme in the field of developmental psychology. However, conventional measurement methods require time-consuming preparation, and the intent of the questions is difficult to understand. However, we were able to measure intimacy with others by drawing, which is familiar to young children and easy to implement.

Even in early childhood, various issues such as isolation and bullying can arise. Detecting these problems early and establishing supportive methods to build positive relationships while preventing them from becoming serious problems can improve early childhood wellbeing and have a positive impact on the child's future.

5. Giant 3D image presentation using a collection of hand-held monitors

The world's first technology that enables viewers to perceive a 3D image that appears to pop out, even when using multiple monitors of various sizes with gaps between them. Conventionally, when multiple monitors are arranged with gaps between them, it was thought that huge 3D images projected across the entire monitor could not be perceived well. However, this technology utilizes the human visual system's mechanism for filling in visual information, allowing the image to appear seamlessly connected and emerge as a 3D object, despite the gaps between the monitors. In addition, since multiple monitors are gathered together to form a single giant screen, restoration is easy even if one of the monitors has a problem.

By further advancing our understanding of the human visual system and evolving this technology, more immersive 3D images will be accessible on standard devices. We are also aiming for a future in which anyone can easily experience powerful images that are currently only available with specialized equipment or in special facilities, in everyday settings.

6. Visualize your exercise skills!

Technology to quantify motor skills and degree of handedness or footedness by simply performing simple repetitive exercises. Conventional hand and foot dexterity surveys are easily influenced by correction efforts and lack quantitative measurement based on actual motor skills. Conventional methods of measuring hand dexterity also require special equipment, making them difficult to evaluate easily.

This technology demonstrates that the less variation in the trajectory of repetitive movements performed with a common smartphone attached to the hand or foot, the higher the motor skill. By rotating a smartphone held in the hand or attached to the foot in sync with sounds, the trajectory of acceleration is measured, making it possible to evaluate motor skills and quantify the degree of dominant hand or foot. It also helps visualize improvements in motor skills with growth and declines due to aging.

By further developing a simple method for evaluating motor skills, we can propose training tailored to specific sports and easily visualize left-right balance and learning effects in physical rehabilitation. We also aim to develop it as a tool to explore the relationship between motor function and brain information processing.

3) Research Lectures

Digest of the lecture given by NTT Communication Science Laboratories on "Natural and Comfortable Video Display that Accommodates Human Perception".

Natural and Comfortable Video Display that Accommodates Human Perception
~Optimization of display images based on the human visual information processing model

NTT Communication Science Laboratories Human Information Research Department
Daiki Fukiage

With the development of information display technologies and display devices, any location in the real world may be used as a screen for information display in the near future. However, new display technologies using projectors and transparent displays cannot always provide the ideal display as in a movie theater, because the appearance of the displayed image varies greatly depending on the surrounding brightness and background patterns. In such cases, understanding how humans see things is the key to solving the problem.

The importance of image display technologies that align with human visual characteristics is expected to grow. As part of this approach, our research group is working on modeling the human visual information processing process and automatically optimizing the displayed images based on this model.

For example, augmented reality displays often semi-transparently display other image information in the real world. However, it is difficult to maintain the desired visibility of the semi-transparent image, because the visibility of the semi-transparent image is affected by the background. To address this issue, we investigated how the brain's contrast detection mechanism influences visibility and developed a visibility prediction model with a module that mimics this mechanism. In projection mapping technology, images and video are projected to blend into the real environment, but depending on the brightness of the environment and the light absorption rate of the real object being projected onto, the contrast of the projected image may not be sufficient to achieve the desired appearance. To counter this, we proposed a method to optimize the projected image so that it comes closest to the "perceptually" ideal image by using a computational model that reproduces the representation of visual information in the brain.

In the future, any location in the real world might be used as a screen for displaying information. However, given constantly changing display environments, manually creating optimal displays is impractical. Our research aims to provide a natural and comfortable experience in such situations by using models of human perception.

4) Related Information

NTT Communication Science Laboratories Open House 2024
https://www.kecl.ntt.co.jp/openhouse/2024/index_en.html

NTT Communication Science Laboratories "Open House 2023" [Digest Report] - Full of hints to decipher people, society and the Earth!
https://group.ntt/en/magazine/blog/openhouse2023_report/