Ultra-High-Presence Video Live Viewing of "KABUKI LION SHI-SHI-O" Las Vegas Performance

On May 7, 2016, Nippon Telegraph and Telephone Corporation (NTT) and Shochiku Co., Ltd (Shochiku) conducted three experiments in which videos with a high sense of reality were transmitted using NTT's latest technologies, including "Kirari!", an immersive telepresence technology designed to enable viewers of a video to experience an event taking place at a remote site as if they were right at the event site. These experiments were the first joint experiment by NTT and Shochiku aimed at creating a new way of appreciating a Kabuki performance. Members of the press and invited reviewers gathered in Haneda Airport International Passenger Terminal to enjoy a live performance of "KABUKI LION SHI-SHI-O: The Adventures of the Mythical Lion", which was taking place at the MGM Grand Las Vegas, USA.

At the technology exhibition called "KABUKI LION Interactive Showcase," held in fornt of the theater in Las Vegas, we provided a demonstration exhibit, "Henshin Kabuki," which featured a virtual display of kumadori, a special makeup technique used in Kabuki. Many visitors experienced this virtual cosmetic treatment of their faces.

Remote ultra-high-presence live viewing at the Haneda Airport

The following three types of remote live viewing of the performances in Las Vegas were conducted using a video transmission network linking Japan and the USA, which consisted of GEMnet2 (a network for research and development owned by NTT Laboratories) and links provided by Internet2 and Pacific Wave in the USA.

(1) Remote address on a stage with a quasi-3D video produced using Kirari!

The image of Ichikawa Somegoro on the stage in Las Vegas was extracted using the object extraction technology of Kirari! and transmitted to the event site at Haneda Airport. The image provided such a high sense of reality that the audience at the airport felt as if Somegoro was addressing them right there. When reporters at the Haneda site asked him what he thought of the convergence of the traditional art and the latest technology, Somegoro answered, "I think there may be types of Kabuki performance that are only possible using video, that is, there may be types of Kabuki performance that can only be truly appreciated through video." He said this in an easy and natural manner, giving an impression that he was standing right in front of the questioners.

While in Las Vegas, Icihikawa Somegoro, participated in a virtual photo session at the event site at Haneda Airport.

(2) Space reproduction using multiple 4K screens (spherical room)

The video captured by nine 4K-definition cameras installed in the Las Vegas theater where the Kabuki performances were put on was compressed in size by a factor of about two hundred using H.265/HEVC technology, which features a high compression ratio. The video and audio were synchronized using MMT technology. Error correction technology was applied to the video signal before transmission across the Pacific to Tokyo, where the live viewing was taking place. This was the first attempt in the world to transmit multiple synchronized 4K screens across an international border. The viewers were able to experience powerful, seemingly unbridled performances of the Kabuki actors with a high sense of reality that made them feel as if they were watching from the front row of the theater.

Live viewing using multiple 4K screens

(3) Live high-quality delivery of a 4K spherical video to a mobile terminal

Wearers of a head-mounted display (HMD) at the Haneda event site were able to see, through the HMD, a video from any direction they chose, from the spherical video captured by a 360-degree spherical camera installed at the Las Vegas theater. Thanks to the use of the interactive streaming technology for spherical video developed by NTT, a high-quality video could be delivered efficiently to smartphones, which have only a limited communication bandwidth, enabling many attendees at the Haneda event site to experience the pleasure of watching a Kabuki performance.

Live viewing using a head-mounted display (HMD)

Henshin Kabuki

In Hensin Kabuki, NTT's angle-free object search technology was used to recognize automatically, with a high degree of accuracy, the kumadori-ed mask that an attendee chose and held in his/her hand. An augmented reality image of the kumadori was superimposed on his/her face (a process we called "Henshin Kabuki" or a makeover Kabuki). In addition, we implemented projection mapping whereby dynamic poses or facial expressions unique to Kabuki were projected onto a huge 3D facial object. Furthermore, NTT's light projection technology, called "deformation lamps," was used to give an impression of a variety of realistic movements, such as those expressing laughter or anger, to a still kumadori image on the wall.

This exhibit was so popular that people queued up to have this unexpected experience. Ichikawa Somegoro, Nakamura Karoku and other Kabuki actors also enjoyed the prospect of a new world in which Kabuki, Japan's most famous performing art, merges with advanced technology during the free time between their performances.

KABUKI LION Shi-Shi-O cast attending NTT's technical exhibit:
From left, Nakamura Hayato, Otani Hirotaro, Ichikawa Komazo, Ichikawa Somegoro,
Nakamura Karoku, Nakamura Kasho, and Nakamura Yonekichi

Future development

Using this event as a springboard, NTT and Shochiku will strengthen activities to develop a next-generation Kabuki through the use of ICT.