One of the consequences of the recent COVID-19 pandemic was the creation of restrictions on movement in the real world. People were unable to meet one another, and meaningful communication suffered.

There is a need for a virtual reality space closer to actual reality than ever, featuring the fusion of real and virtual with extended reality (XR) technology, where rich communication and diverse experiences can be enjoyed.

NTT is providing new aural and visual experiences that combine real and virtual using the latest technology of NTT Laboratories. To debut its pioneering XR, NTT chose to use the medium of traditional Japanese theater form kabuki, with its stylized performances that blend drama, dance and highly ornate costumes.

"Cho-Kabuki Powered by NTT," which NTT hosted along with two partners--telecommunications and media company Dwango and film and kabuki production company Shochiku--at Makuhari Messe in Chiba Prefecture in April this year, demonstrated a groundbreaking production technology that allowed the audience to enjoy an enhanced sense of presence on stage and a feeling of immersion in the story.

Theater sound production traditionally uses speakers to deliver a uniform sound to the audience. More recently, movie theaters have used multiple speakers to produce spatial sound, where sounds could be heard ahead of or behind the audience members or to the sides. However, traditional speakers nevertheless found it hard to reflect the movement and action of the movie being shown and were unable to give a genuinely intimate sound. Meanwhile, although home audio using earphones or headphones can reproduce spatial sound that can be heard from any direction and be immersive, it has not been used in theaters because it blocks out surrounding sounds.

NTT's innovation is to use open-ear earphones that allow wearers to hear surrounding sounds while delivering three-dimensional spatial sound that does not interfere with the natural ambiance of the venue. Earphone-wearers can listen to the live voices and music of the performers while simultaneously listening to the spatial acoustic effects of the Cho-Kabuki performance. It's the best of both worlds: an intimate, personal aural experience for every individual while simultaneously enjoying the sounds and ambiance of the theater with every other audience member.

Another entertainment breakthrough was how "Cho-Kabuki powered by NTT" allowed participants to experience virtual characters with the naked eye, without needing smartphones or augmented reality glasses, and see real-life characters on the stage. Its "XR shared conversation system" technology made this possible.

Viewing or even having a conversation with a stereoscopic hologram indistinguishable from the real thing is something we have all seen before in science fiction movies. However, to have an experience close to replicating it, it was necessary to wear unique, special eyewear to see the image; not everyone likes wearing eyewear. Moreover, since the pictures and photos appearing in an individual user's eyewear can't be seen by people around them, everyone must wear something on their eyes to participate. The experience becomes unnatural.

Over the decades, many attempts have been made to show 3D holograms without using eyewear. Still, until now, they have suffered from problems such as insufficient resolution and complicated and large-scale equipment. It's remained the domain of sci-fi.

Thanks to NTT, science fiction is now on the point of becoming a reality.

NTT has developed its "glasses-free XR shared conversation system" to enable everyone to enjoy XR communication. NTT's system features a type of mirror with a display that can adjust how it reflects and transmits, along with a multi-layered 3D layout design that shows both real and virtual objects seamlessly interacting. The technology allows the user to experience something close to what we know from science-fiction movies by watching Cho-Kabuki from their seat.

Even more exciting than that, however, the glasses-free XR shared conversation system means that by simply sitting on a chair in front of a display unit, anyone can communicate with a 3D hologram right before them. Since the conversation can be seen by everyone nearby, it is also possible to share with multiple people.

Visitors to the "Cho-Kabuki powered by NTT" trial were able to experience close-up communication with the virtual character Hatsune Miku, who takes the leading role in Cho-Kabuki. The tech even allowed visitors to take pictures with Hatsune Miku.

NTT began its trials with the traditional art of kabuki, but the potential uses of its technology are vast. And not just in the entertainment field--eventually, XR technology has the potential to expand into regular remote work, games and virtual events, and other industrial areas such as construction and manufacturing.

NTT--Innovating the Future