Scientists have developed a new algorithm that could transfer acclaimed photographers' signature styles to your cellphone photos.
"Style transfer" is a thriving area of graphics research, but standard style-transfer techniques tend not to work well with close-ups of faces, said YiChang Shih, a Massachusetts Institute of Technology (MIT) graduate student.
Shih and colleagues at Adobe and the University of Virginia instead perform what they describe as a "local transfer."
Using off-the-shelf face recognition software, they first identify a portrait, in the desired style, that has characteristics similar to those of the photo to be modified.
"We then find a dense correspondence - like eyes to eyes, beard to beard, skin to skin - and do this local transfer," Shih said.
One consequence of local transfer, Shih said, is that the researchers' technique works much better with video than its predecessors, which used global parameters.
Suppose, for instance, that a character on-screen is wearing glasses, and when she turns her head, light reflects briefly off the lenses.
That flash of light can significantly alter the global statistics of the image, and a global modification could overcompensate in the opposite direction.
But with the researchers' new algorithm, the character's eyes are modified separately, so there's less variation in the rest of the image from frame to frame.
Even local transfer, however, still failed to make modified photos look fully natural, Shih said. So the researchers added another feature to their algorithm, which they call "multiscale matching."
"Human faces consist of textures of different scales," Shih said.
"You want the small scale - which corresponds to face pores and hairs - to be similar, but you also want the large scale to be similar - like nose, mouth, lighting," said Shih.
Shih said the technique works best when the source and target images are well matched - and when they're not, the results can be bizarre, like the superimposition of wrinkles on a child's face.
But in experiments involving 94 photos culled from the Flickr photo-sharing site, their algorithm yielded consistently good results.
