NTT and IMAGICA GROUP Launch a Comprehensive Joint Study on Expression and Production Techniques for the Real-Cyber Fusion Space of the IOWN Era

Tokyo - December 8, 2022 ― NTT Corporation (President and CEO: Akira Shimada, "NTT") and IMAGICA GROUP Inc. (President and CEO： Nobuo Fuse, "IMAGICA GROUP") have embarked on a comprehensive joint study on expression and production techniques for the real-cyber fusion space of the IOWN¹ era. By combining NTT's advanced technology for processing audiovisual, emotional, and 3D with IMAGICA GROUP's creativity and technology, we will utilize the advanced digital twin computing of IOWN to develop new media experiences and communication UI/UX.

NTT is collaborating with various partners to advance research and development on innovative information networks and communication platforms, including IOWN. IMAGICA GROUP offers a wide range of high-quality services and products on a global, one-stop basis, from video planning to production, video editing, and distribution. In addition, we have established a horizontal research and development organization across the group and are actively working on the development of new technologies while promoting the accumulation and advancement of existing technologies.

Previously, NTT and IMAGICA GROUP have collaborated on various initiatives, including joint demonstrations of high-definition, high-presence video as member companies of the Next Generation Content Distribution Forum. The latest joint study to realize ultra-realistic digital twin computing of spaces and humans is not limited to the conventional field of video production and provision, but progresses towards a space where the real and cyber fuse and coexist. Particularly, we are planning to proceed with the examination of expression and production techniques.

Outline of the Joint Study

1) Creation of a new form of communication that integrates the real and cyber

In addition to existing regional issues, there are new issues in fields such as education and medical care due to changes in local communities. To solve this problem, human resources called the "affiliated population" outside the region will play a new role in regional development. At the same time, human activity is expanding and diversifying into the online space through means such as SNS and web conferences, and cyberspace as represented by the Metaverse.

Therefore, to expand the "affiliated population," in this joint study, we will construct a photorealistic cyberspace that allows people to immerse themselves in tourist spots from the comfort of their homes through the digital twin computing of spaces and people, with the aim to create a new kind of integrated communication. Specifically, this is achieved by combining various media processing and AI technologies such as 3D spatial media processing technology and Another Me®² related technologies currently being researched and developed by NTT with IMAGICA GROUP's high-definition and highly realistic images and content production know-how.

As the first step of such efforts, NTT has jointly launched the "TENGUN Ogijima Project"³ to promote regional co-creation, with NTT West Japan, Ogijima Life Research Institute, and Kenohi Co., Ltd. We plan to utilize the results of the study from both companies.

2) Adding value to entertainment content through ICT

Live entertainment, centered around music and theater, is important for enriching people's lives. NTT and IMAGICA GROUP are working to combine and synchronously transmit ultra-high-definition images of 12K pixels in the horizontal direction along with various media information, resulting in a future that will allow viewers to experience live entertainment remotely and demonstrating the value of live viewing⁴.

Recently, due to the proliferation of digitalization and remote culture, there are more opportunities to enjoy live events such as live music remotely. However, we believe that it is difficult to replicate the same emotional experience as on-site, such as a sense of unity and enthusiasm. NTT uses human digital twin computing to estimate and model the emotional state of the audience and stimulates the senses of sight and touch to create a virtual live space tailored to the desires of each user.

In this joint study, we aim to create an unprecedented and attractive media experience in the virtual live space by combining NTT's technology and IMAGICA GROUP's planning and production of new media experiences. In addition, we plan to proceed with the investigation of production using three-dimensional space display and expression, replication of traditional skills and performing arts, and expression and production in live entertainment as techniques for providing more attractive live entertainment content.

3) Examining methods of providing diverse content and media based on the umwelt

German biologist Dr. Jakob von Uexküll proposed the idea that "various organisms live in their own species-specific perceptual world, called the umwelt, and act as the subject of it." At NTT, based on how this concept applies to the global world, we will create a new UI/UX through IOWN that responds to the diversity of people, things, environments, and their respective values. We are considering information provision methods that can adapt the information to be conveyed and the method of processing it.

In this joint study, we will provide content and media experiences that meet the value of diverse audiences, such as multimodal content not limited to visual and auditory and content that changes interactively according to each user's senses, preferences, emotions, and behavior.

Specifically, NTT is researching and developing "personalized sound zone technology⁵ that accurately collects the sound information from one's surroundings, processes the total situation, and appropriately controls sound. Additionally, we are developing technologies such as hyperspectral compression imaging technology⁶ (which makes it possible to capture hyperspectral images with a normal-size digital camera that can distinguish the characteristics of subjects that are difficult to grasp with the human eye) and remote presentation technology (which realizes a more realistic experience by generating tactile sensations through vibration in addition to images and sounds). We will apply these technologies to content and media experiences and plan to study expression and production techniques to provide new value beyond the conventional provision of video and audio.

By promoting field demonstrations based on the results of this joint study, we aim to visualize and evaluate user value to create innovative technologies. In addition, we will consider fields and opportunities to allow customers to experience this value more widely, such as the Osaka Kansai Expo, and contribute to the realization of a prosperous society for the IOWN era.

¹IOWN (Innovative Optical and Wireless Network) is a network and information processing platform that includes terminals and provides high-speed, large-capacity communication and enormous computational resources using innovative technologies centered on light. IOWN mainly consists of three parts: an "all-photonics network" that also uses optics in terminal processing, "digital twin computing" that enables advanced and real-time interaction between things and people in cyberspace, the "cognitive foundation" that efficiently deploys ICT resources, and various other technologies. Through IOWN, we will contribute to the realization of a prosperous society by creating various services and new value in a wide range of fields.

²Another Me is a digital reproduction of a real person the transcends the constraints of the real world and acts autonomously in society, sharing the results as the individual's own experience. The technology aims to dramatically increase opportunities for people to play an active role and grow.

³"NTT Launches the 'TENGUN Ogijima Project' to Promote Regional Co-creation: Joint studies aimed at creating and expanding interconnected populations using the photorealistic 'Ogijima' metaverse of IOWN"

⁴"The world's first futuristic live viewing that synchronizes 12K wide video and lighting: State-of-the-art live spectacle 'VISIONS SUPER LIVE VIEWING supported by LDH'"

⁵"Media Processing for the Ultimate Private Sound Space," NTT Technical Journal. 2020.10, pp. 52-56.

⁶"Establishing a World-First Technology for Capturing Hyperspectral Images and Videos by Combining Meta-lens and AI with Ordinary Digital Cameras"