Finally--only one year to go before Tokyo 2020!

With the aim of conveying sports' multifarious attractions to as many people as possible, NTT held the inaugural "SPORTS VIEWING RE-IMAGED" exhibition in order to enable as many people as possible to experience sports' multifarious attractions as well as lots of excitement by capturing how sports are conveyed from zero using the power of technology, art, and design.

As NTT touts the re-imaging of visual sports viewing, auditory sports viewing (listening), multisensory sports viewing, and sports viewing concepts, what kinds of exciting experiences will NTT's newest challenge bring us?

Re-imaging of Visual Sports Viewing: Full Transmission of Athletes What is this new, unprecedented "experiential value"?

Ever since its launch in 2015, Kirari! ultra-realistic communication technology has been gaining attention with each innovative demonstration. This technology aims to "completely" transmit sporting spaces or live performance spaces in real time, enabling anyone, anywhere in the world, to enjoy such events at anytime as if it were only natural.

For the recently held SPORTS VIEWING RE-IMAGED exhibition, a demonstration badminton match was presented using basic Kirari! technology, "Extraction of Objects with Arbitrary Backgrounds" and "Advanced MMT Technology". The parts of 4K footage filmed at the match venue showing the players were extracted and transmitted, then projected as aerial images onto a simulated court in a public viewing venue, thereby giving the appearance that the players were battling it out over the net, right in front of spectators' eyes.

Left: Venue when players' images are being projected. Right: Venue when the players' images are not being projected.
The images of the player in front and the player behind as well as the shuttlecock are each extracted and faithfully re-imaged on the simulated court.

Extraction of objects in real time using AI (Artificial Intelligence) and other technologies enables images of the players to be extracted during play, with sound superimposition timed so that there is no sense of strangeness when the match is re-imaged at the desitination venue. This is made possible by "Advanced MMT" technology, which enables synchronous transmission of images, sound, and spatial information.

This demonstration presented a badminton match (with two players), but for this technology to be applied to sports such as soccer, where multiple players frequently come into contact, there are various hurdles that must be overcome in order for further evolution to be achieved together with impact at the other end of the scale. However, it is anticipated that there issues will be resolved in the near future, realizing viewing of various sports that envelops spectators in a "sense of being present" as if the players were battling it out in front of their eyes, even though the venue where the match is actually being played is far away.

Re-imaging of Viewing Concepts: Dramatically Changing Forms of Sports Viewing New Sports Viewing Experiences Being Created by The "Swarm Arena" Project

"Kirari!" is not the only technology that is changing our sports viewing styles. In the age of IoT, mobile objects such as drones utilizing AI/robot technology and cars that drive automatically are also building societies while connected to the Internet.

Generating new user experience concepts through fusion of ICT and art and pursuing research, NTT has in collaboration with Ars Electronia Futurelab developed a new form of sports viewing whereby convenional image displays (which fit into square frames) are spread throughout entire spaces.

Using large groups "swarms" of "Grand Bots" (mobile robots equipped with hexagonal image displays) and communication control technology for controlling each bot (Swarm control), NTT is undertaking an experiment to re-image the very concept of sports viewing.

Detecting, analyzing, and transmitting images of athletes, drones fly in the air or along the ground, matching their movements to those of the athletes, enabling the production of three-dimensional images.

At the SPORTS VIEWING RE-IMAGED exhibition, a demonstration showing the movement of swimmers flimed by Grand Bots was conducted. With the appearance of honeycomb, the swarm of Grand Bots repeatedly links and separates as the bots express the beauthy and dynamism of the swimmers' movements. This may not be a simulated experience where spectators at a viewing venue are watching (with the naked eye) action (images) filmed at a remote location, but rather a completely new experience. The demonstration gave spectators this sense of premonition.

Re-imaging of Visual Sports Viewing: ICT Colors Windsurfing through Real Time Surrounding Image Synthesis Technology

Sailing is a sport carried out far away, on the ocean. In the past, it has been difficult to convey the real thrill of this sport--the powerfulness of yachts cutting through the ocean waves, accelerating more and more as the strength of the wind fills their sails. NTT's "Real Time Surrounding Image Synthesis Technology" has made it possible for spectators to have the unprecedented experience of feeling this previously unknown sense of powerfulness of yachts on the ocean while sitting in a venue on land as if they were actually there on the water.

This "Real Time Surrounding Image Synthesis Technology" is a kind of Kirari! ultra-realistic communication technology By seamlessly and in real time synthesizing images filmed with 4K cameras placed in multiple locations in a horizontal direction, this technology realizes ultra-wide images with greater-than-8K high-definition. Moreover, Kirari! is also used for adjusting color tone so that images appear naturally as one image and synchronizing screens that have been split from projection purposes, and it is anticipated that this technology with make new viewing styles for various sports even more powerful.

Re-imaging of Auditory (Listening) Viewing /Multi-sense Viewing: Inclusive Sport Experience Technology that Enables "Everyone" to Enjoy Sport using Sound and Sense of Touch

Japan's population of persons with physical disabilities is approx. 4.36 million. ^*1 In addition to the frustration of being unable to view sports, it is said that many of these people have concerns about inviting various misunderstandings due to their disability.

Accordingly, we will report on the re-imaging of sports viewing brought about by inclusive sport experience technology such as the creation of viewing environments that take into consideration the improvement of accessibility in a diversity of situations.

At sporting venues, sporting spaces where players with visual impairment roll a ball with a bell inside along the floor, relying on the sound of the bell as they attempt to shoot the ball into their opponents' goal have been re-imaged using Wave Field Synthesis technology that creates acoustic fields in arbitrary places. This experiment in sports viewing relying only on sound without sight enables any person to experience the sensations that actual Goalball players feel.

It can no doubt be said that an effective means of widely conveying the appeal of disabled sports would naturally be a new form of sports viewing that enables spectators to actually experience sensations that normally only athletes competing in the stadium can know.

Goalball
The game is played with three players on each team. Players rely on the sound of a bell inside the ball and the feeling of the line tape under their hands and knees as they attempt to shoot the ball into their opponents' goal.

^※1 Status of Persons with Disabilities, White Paper on Persons with Disabilities 2018
(*Japanese Version Only)

Speakers laid out so as to surround the experiential space

Another method is re-imaging of multi-sense viewing. This experiment aims to present a new form of sports viewing by adding tactile information so that everyone--regardless of whether or not they have a disability--can enjoy sports together.

"I was born without sight, and so up until now, 'sports viewing' for me consisted of the sounds of commentators' voices and balls hitting or bouncing off objects."

So said Mr./Ms. Ikematsu, who participated as a sight-impaired person in a demonstration of viewing a tennis match by feeling vibrations that was conducted at the SPORTS VIEWING RE-IMAGED exhibition venue.

Demonstration participants viewing a tennis match by feeling vibrations.
Vibrations are conveyed from the model of the court to the participants' hands, enabling them to feel the back-and-forth of rallies.

Describing the experience, Mr./Ms. Ikematsu said, "So up until now, 'sports viewing' for me consisted of the sounds of commentators' voices and balls hitting or bouncing off objects. The tactile information was therefore very novel for me. There was lots of information that I cannot get from just commentary, such as the players bouncing the ball and gathering their concentration before they serve, the strength of serves, and the tempo of rallies, so it was a very interesting experience. Incidentally, while I myself am strictly a supporter, I have many friends with the same disability who also play sports, and so I think it would be great if the attraction of disabled sports could be spread more widely through the power of technology so that eventually they are no longer something special but are accepted as popular sports just like any other sport.

Viewing Sports or Feeling Sports: From Sports Being Events to Watch to Becoming Events to Feel

Sports have a dynamism that cannot be captured in images.Moreover, there are sensations that only athletes competing in the stadium can know.

Currently, in Japanese we refer to "sports viewing" as kansen.
However, since the word kansen contains the character for "watch", it can be said that the term proves that the action it itself describes is inherently limited to "viewing".
Perhaps sports viewing in the future will become closer to an experience of "feeling" rather than "watching".

Then, if the diverse realities, many attractions, and excitement of sports could be conveyed to huge numbers of people through the use of technology, what kind of future awaits us?
It is NTT's intention to continue to update people's images of sports viewing as we utilize ideas and technologies born from communications.

Look forward to as-yet unknown forms of sports viewing and the discovery of new values within sports viewing.

Only one year to go until Tokyo 2020...