NTT showcases "a natural conversation among multiple persons and robots" and "interactive signage using image search technology" at SXSW 2017

1.Overview of the exhibits showcased by NTT

(1)Robot Showcase

The NTT Group's AI technology suite corevo™ (noise-tolerant sound pickup, speech recognition, speech synthesis, and discussion dialogue technologies) are incorporated into "Repliee S1," a robot with no skin, for hiding inside machinery, and a cute, female type android called "U" to enable them to conduct natural discussions with people. Through actually conversing with Repliee S1 or android "U" on the site, visitors will be able to think about the meanings of communication and synchronicity.

There will be a feature session involving Professor Hiroshi Ishiguro, Dr. Ryuichiro Higashinaka (senior research scientist, NTT Media Intelligence Laboratories), Repliee S1, and android "U" under the title "Humans and Robots in a Free-for-All Discussion" at JW Marriott Austin between 3.30 p.m. and 4.30 p.m. on March 12. This session will demonstrate both a daily discussion and an esoteric discussion on the relationships between humans and robots conducted by the two persons and the two AI robots, into which speech recognition, speech synthesis and discussion dialogue technologies ^(*3) have been incorporated.

(2)Interactive Signage Showcase (technical assistance provided by NTT DATA Corp.)

We have developed a new interactive advertising medium by combining digital signage with what a person wears or has in hand. This new medium is named "interactive signage." It will be showcased under the title "Linked Virtual Ad. ˜Talking Beer˜." When a person passes by a signage unit, the sign provides information to him/her in a personalized manner, without requiring him/her to take any conscious action, thereby offering a memorable new experience.
"Linked Virtual Ad. ˜Talking Beer˜" is an interactive digital signage service that automatically identifies the brand on the container of beer that a person is holding in his/her hand. When the person stands in front of the digital signage unit, motion capture technology locates the position of his/her hand. Then, angle-free object search technology, which is one of the corevo™ technologies, identifies the brand of beer he/she is holding. Once the beer brand is identified, AR technology is used to display a character associated with that brand. The character provides information about the beer. This service can simultaneously identify the brands on multiple beer cans. In such case, the characters associated with the respective beer cans greet each other and work together to present product information in an entertaining way.

Image of presentation in "Linked Virtual Ad. ˜Talking Beer˜

The features of the technologies showcased in these exhibits are described in the attachment.

^*1Overview of Japan Factory

• *1Overview of Japan Factory

^*3Dr. Higashinaka will be the featured speaker at SXSW 2017.
http://www.ntt.co.jp/topics/sxsw2017/index.html (Japanese Only)

(Attachment) Features of the technologies

(1) Noise-tolerant sound pickup technology

This technology uses intelligent microphone technology developed by NTT Laboratories. Noise-tolerant sound pickup is realized by combining a number of small general-purpose microphones and acoustic signal processing. The latter uses the directions from which sounds are coming, frequency properties, and temporal variations in these properties to reduce the magnitude of ambient noise while minimizing degradation in the target sound, thereby making it possible to pick up the target sound clearly. This technology allows highly accurate speech recognition even in a noisy environment.

(2) Speech recognition technology

This technology achieves high-speed, high-accuracy speech recognition by using acoustic modeling based on the latest deep learning and utilizing an optimal search network based on a weighted finite state transducer (WFST).

(3) Speech synthesis

This technology synthesizes speech with rich variations by combining two elements that characterize speech: speaker and tone.

(4) Discussion type conversation technology

This technology makes a deep discussion on a specific topic possible by using large structured discussion data as conversation knowledge. The discussion type conversation system argues and rebuts in a timely manner by understanding the speech and conditions of the speaker and appropriately managing the discussion state. This technology is useful for building consensus between people and systems or for assisting in decision-making.

(5) Angle-free object search technology

This technology makes it possible to search for a 3D object in a photo, even when the photo is not taken from the front of the object. Object search accuracy has been significantly enhanced through statistically estimating the degree of importance and the frequency of occurrences of image features. The number of reference images that formerly had to be pre-registered in a database has been reduced dramatically to 1/10 of those required under conventional methods by accurately identifying correspondences between the input image and reference images.
http://www.ntt.co.jp/news2015/1502/150216a.html (Japanese Only)