Google & UC Berkeley’s AI Realistically Removes Unsightly Shadows From Photos
Video screenshot via Cecilia Zhang et al.
Here’s a technology that could significantly shave off photographers’ editing time, or even transform smartphone photography should tech giants pick it up. Researchers at the University of California Berkeley and Google Research have developed an AI that effectively erases unwanted shadows from portraits, so you won’t have to wait until after 4pm, when the sun’s rays are less harsh, for an outdoor shoot.
Their paper, published here, details an algorithm designed to remove or soften two types of shadows: those created from external objects and those that appear due to natural features on the face.
The AI works in a “realistic and controllable way,” the researchers described, to recognize unsightly shadows and highlights, and then reduce them until none of those detract from the subject.
This means that, for the everyday person, they won’t have to deal with confusing lighting conditions again. Mom won’t snap a harsh photo of you and convince you that it looks “fine.”
For professional photographers, the technology could salvage outdoor photography work in settings where lighting is impossible to control, such as photos from wedding shoots conducted under the intense afternoon sun.
However, the technique can occasionally make portraits look soft and “flat,” and sometimes smooths out fine details—such as hair—that should be retained.
The team also shared that the algorithm can, at times, confuse between facial and foreign shadows, especially if the facial shadows are “sufficiently harsh.” Natural shadows on the face could be erased if the technology presumes they were cast from external objects.
“It is not clear how existing light stage scans or in-the-wild photographs could be used to construct a large, diverse, and photorealistic dataset in which both foreign and facial shadows are present and available as ground-truth,” the researchers also shared.
The effects presented by the team are pretty compelling, nevertheless. See the technology at work in the video below.
[via Digital Photography Review, video and cover image via Cecilia Zhang et al.]
Also check out these recent news
