HenGenTou (Deformation Lamps): A magical lighting system to produce illusory movements in static photos and paintings

Background

"What is happening? Trees in a landscape oil painting are fluttering in the wind, and a man in a portrait suddenly starts speaking. This cannot be real." What if there were some way to produce such fantastic experiences.
A promising method may be projection mapping, a cutting-edge technology that can effectively changes visual impressions of a real object, by way of projecting a well-designed pattern onto the object's surface as a screen. Conventional projection mapping techniques, however, cannot produce movements of static objects per se, since they "paint" a new color and texture on the object's surface without preserving the original surface's appearance. In addition, although the goal of conventional projection mapping is to produce a physically correct image on the object's surface, it seems theoretically impossible to make a static picture move in a physically correct way only by image projection.

Achievement

NTT Communication Science Laboratories has developed a novel type of projection mapping named 'HenGenTou' (Deformation Lamps). HenGenTou can add a variety of dynamic impressions ranging from natural liquid flows to facial expressions to a printed image or other static materials. This capability is a result of our long-term scientific research on human information processing, in particular visual processing for natural movements.
HenGenTou makes use of visual illusions to add a movie-like dynamic appearance to a two-dimensional (2D) static object, such as fluttering flames and blowing wind. It can apparently do this even for three-dimensional (3D) objects, although the viewing condition is restricted.

Future plan

Since HenGenTou can broaden the expressiveness and increase the salience of almost any static object, it will provide a new methodology for image expression in a wide range of fields.

(1)Advertising

HenGenTou can add motion impressions to static signage, such as a printed poster or a hand-written signboard, to emphasize the main message of an advertisement.

(2)Interior design

HenGenTou can induce illusory movements on the wall or floor and produce a variety of visual effects in rooms: e.g., an undulating water flow or rising heat waves.

(3)Art and entertainment

HenGenTou can animate printed characters to make them look much more resplendent or more bizzare than the original static ones. While HenGenTou is good at adding motion impressions, conventional projection mapping is good at changing the color and texture of 3D objects. Using HenGenTou in combination with the conventional technique, artists and creators can produce novel material expressions that have never been seen before.

Examples

URL of demo movie: https://www.youtube.com/watch?v=wIHzWJm5398

• *This URL is an external site.

(1)Animation of 2D static pictures

HenGenTou can add realistic motion impressions to photographs, paintings, and walls and produce visual experiences that no one has ever had before.

• Printed flames fluttering in the wind.
• A man in a portrait changing his expression.

Figure 1. Basic HenGenTou system

(2)Animation of 3D static objects

Our technique can be applied to 3D objects. For complex 3D objects to which applying projection is difficult, superimposing motion with a transmissive LCD is a promising alternative.

Figure 2. Deformation of a 3D object using transmissive display