Microsoft ends support for Internet Explorer on June 16, 2022.
We recommend using one of the browsers listed below.

Microsoft Edge（Latest version）
Mozilla Firefox（Latest version）
Google Chrome（Latest version）
Apple Safari（Latest version）

Please contact your browser provider for download and installation instructions.

November 26, 2018

NTT Develops Non-Rigid Object Recognition Technology with a Small Number of Database Images - Not Only Rigid 3D Objects but Non-Rigid Objects Such as Soft Packaging or Cloth Products can be Recognized with Low Preparation Cost -

Nippon Telegraph and Telephone Co., Ltd. (hereafter "NTT") has developed "Angle-Free Rigid and Non-Rigid Object Information Retrieval" which recognizes and searches deformable objects with a small number of database images (Figure 1).

By this technology, in addition to rigid 3D objects, non-rigid objects such as soft packaging or cloth products can be recognized with high accuracy.

Conventionally, image recognition technologies including deep learning based ones require preparation of many database images capturing multiple deformation patterns in order to recognize deformed objects. If an unknown deformation is added to the object, many more database images are needed to recognize it.

Our new technology, "Rigid and Non-Rigid Object Image Matching," recognizes an object with an unknown deformation as the same object as the non-deformed object image in the database.

This technology can reduce the preparation cost of database images, which makes it possible to develop a "product management" system for a retail industry in which new products are released frequently. This is also applicable to "product recognition" which realizes power saving of the check-out business, and "product information presentation" which enables browsing product descriptions in multiple languages (Figure 2).

Introduction

In order to realize a digital transformation that improves our lives by connecting all kinds of people with ICT, it is essential to establish technologies that can recognize and search from images taken by smartphones and cameras in place of human eyes.
However, compared to rigid 3D objects, non-rigid objects such as soft packing products and cloth products have different patterns of deformation. Therefore, the appearance of the images is greatly changed, resulting in lower recognition accuracy. In order to maintain the recognition accuracy, it is necessary to register the image corresponding to the deformation of the target object as a reference image. However, if the method of deformation of the target object is arbitrary, it is necessary to prepare a large number of images, and there is a problem that the burden on the cost is large.

R&D achievements

In the "Angle-Free Rigid and Non-Rigid Object Information Retrieval" developed by NTT, by applying the geometric constraint for each of the multiple partial areas instead of applying the geometric constraint to the entire object, by the "Rigid and Non-Rigid Object Image Matching" which accurately specifies the correct correspondence from the correspondence relationship of the image features, the correct correspondence between the input image and the reference image can be specified even if the object is deformed. As a result, in addition to rigid 3D objects, it is possible to recognize and search with high accuracy even for non-rigid objects in which arbitrary deformations occur.

Technical points of "Angle-Free Rigid and Non-Rigid Object Information Retrieval"

"Angle-Free Rigid and Non-Rigid Object Information Retrieval" is one of NTT's AI technology corevo® *1, and it was realized based on "Angle-Free Object Information Retrieval" *2 which can recognize and search 3D objects only by registering images from a few directions (Figure 3). The points of this technology are as follows.

In the conventional "Angle-Free Object Information Retrieval", there has been a problem that the recognition accuracy deteriorates for deformed objects because the constraint condition on the same object is derived from the projection geometric examination based on the assumption that the object has a fixed shape. On the other hand, in the case of a learning based method in the market such as a deep learning based one, if a non-rigid object is used as a target, since there are countless deformation patterns, there is a problem that the recognition accuracy deteriorates unless a large number of reference images observing various deformations are prepared in advance.

Therefore, "Rigid and Non-Rigid Object Image Matching" has been developed which accurately specifies the correct correspondence from the correspondence of the image features by applying the geometric constraint to multiple partial regions instead of applying the geometric constraint to the entire object (Figure 4). According to this technology, it is possible to specify a correct correspondence even if the object is deformed by clustering using geometric features such as distances and rotation angles between multiple correspondence relationships based on the result of correspondence of image features between the input image and the reference image (Figure 5). With the "Angle-Free Rigid and Non-Rigid Object Information Retrieval" expanded by this technology, it is possible to realize the recognition and retrieval with high accuracy by only registering a small number of images even if the pattern of deformation is arbitrary.

^*1corevo® is a registered trademark of Nippon Telegraph and Telephone Corporation.

^*2"Angle-Free Object Information Retrieval"
http://www.ntt.co.jp/news2015/1502/150216a.html

Future Plans

NTT will continue to make practical use of this technology and will aim to realize digital transformation through collaboration with various partners.
Furthermore, the application for product recognition using this technology will be introduced in NTT R&D Forum 2018 Autumn held between November 29 and 30, 2018.
"NTT R&D Forum 2018" site https://labevent.ecl.ntt.co.jp/forum2018a/e/index.html Launch in New Window