Instant reproduction

Enjoy a new experience that transcends distance and time!

The first ever live transmission of phorograph video.

Challenging to create a new sport viewing experience.

Until now, the reality of heated games at sports venues was something only for the spectators at the venue to enjoy. In general, spectators who are not there watch the games on TV, on public viewing displays and on other flat displays, but at the international sports event in 2021, a new communication technology emerged, overhauling this notion, named "Kirari!", an ultra-realistic communication technology that transmitted the whole space of the game at the venue to remote viewing venues. Starting off by the project team wanting to deliver to people who are unable to visit the venue an experience as if they are there, this project ended up embodying a more significant meaning, as COVID-19 spread. Due to COVID-19, concerts, sports events and other events got cancelled, quickly giving way to online streaming. Meanwhile, the sense of immersion and unity felt in physical space was gone. How do we get back these important sensations that are so integral for people watching sports, and how do we reproduce them? Faced with these new questions, the project team's challenge was to create a new sport viewing experience for the future.

The "now" of the competition space transmitted whole, bring together technologies.

To reproduce the whole real scenes at the competition venue in another location, accurate reproduction of video and sound on a spatial level will be vital, evaluating the best approach that incorporates numerous factors such as the features of each sport, the size of their competition space, and the environment of the destination venue. For sports especially, the there is a high hurdle to reproduce the scenes as it requires speed, three-dimensionality, and a sense of immersion. To solve these issues and create a new sport viewing experience, NTT needed to advance its various innovative technologies and understand how best to combine them. The key was the expertise in harmonizing all these factors.

Reproducing the sense of immersion and unity as if one is there.

Transmitting the heated badminton games at Musashino Forest Sport Plaza to Miraikan, which is 23 km away. It was a complex challenge that needed to reproduce not just the performance of the players that moved about randomly on the court, but the movement of tiny shuttlecocks that were rallied at high speeds. The key was "real-time subject extraction technology for any background," that cut out randomly moving objects at high speeds, "ultra-realistic media-synchronizing technology (advanced MMT)" that transmitted data such as video, sound and space simultaneously, and "multi-layered spatial bird-eye holography displaying technology" that reproduced a sense of depth that was not possible with conventional holography. These leading technologies were connected on advanced levels, transmitting the scenes of the games in real time, in steady and three-dimensional holography, offering a new-real sense of being there.

Traditionally, sailing regattas were watched through binoculars and on monitors. This time, NTT used 5G communication services and Kirari! to create a new way of viewing regatta that is dramatic beyond real with an incredible sense of immersion. With "super-wide video synthesizing technology," NTT put together dramatic footages taken by boats and drones with multiple 4K cameras, synthesizing them in real time and displaying as a super wide video with a horizontal resolution of 12K, putting three 4K videos side to side. The video was transmitted live to a 55-m-long wide vision set on the water's surface, installed right in front of the audience area. The dramatic video taken by the 4K cameras was also transmitted to the members of the press from around the world, gathered at the main press center in the Tokyo Big Sight. NTT also installed a virtual stand where families of the athletes who could not visit the venue due to COVID-19 and other reasons were able to cheer from remote locations, creating a new and immersive way of watching sports both at the physical venue and remotely.

At the marathon, which was held in Sapporo, "real-time remote cheering project*" went ahead. NTT used its ultra-low-latency communication technology to create a space that would look as if spectators in Tokyo were cheering on the roadside as the runners were running past. Delay in communication was the biggest hurdle in delivering to the marathon runners in Sapporo, travelling five meters per second, cheering from Tokyo in real time without a time gap. At the marathon venue of Sapporo and the remote venue of Tokyo, 50-m-wide and 2-m-high LED displays were set up, showing each other's life-size videos in real time. The delay that occurred when compressing 4K video data was minimized with "ultra-low-latency media processing technology." For transmission, "disaggregation configuration technology" was used to directly send out uncompressed 4K videos and sound to an optic long-distance channel in SMPTE ST2110 format, making ultra-low-latency transmission possible, minimizing the delay between the sender entering the video to the receiver outputting the video, to about 1 msec. These most advanced communication technologies recorded ultra-low-latency of 100 msec (milliseconds = one-thousandth of a second) for transmission between Tokyo and Sapporo, presenting a possibility for the future of watching sports in the remote era.

Connecting to the Future

Delivering real excitement to any location, and to anyone. Can we reproduce the heat of the games, and the perspiration of the athletes?

If Kirari! is to be used not for the sports mentioned earlier but for large-scale sports like baseball, football, as well as music festivals, further technological innovations will be necessary. Reproduction of larger spaces and more complex movements will be possible as current technologies get updated and telecommunication speeds and capacity improve. If Kirari! becomes widespread in society, anyone will be able to enjoy the real excitement of physical sports venues even if they are unable to visit the venue for reasons such as distance and disability. Furthermore, there will be moving experiences that only Kirari! will be able to offer. What if the invisible heat at the venue can also be reproduced? What if we could feel the athletes breathing and see them perspiring at a distance of just one meter away, which is closer than physically being at the venue? It is technology that can touch the heart of the spectator and create a moving experience directly all over the world. That is the future that Kirari! should look to. One day we might see new dramatic and exciting, beyond-real experiences emerging from this technology. Our challenge to give sparkle to the eyes and heart of spectators continues.

Related Links

お問い合わせ

NTT Service Innovation Laboratory Group
E-mail / randd-ml@hco.ntt.co.jp