This AI-assisted image editing tool from MIT can change the background in any image
For long, Photoshop has been the program that many of us go to make our photos look great with quick fixes and creative enhancements.
Now, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed an AI-assisted image editing tool that automates many parts of the editing process for photos. This will make the job of editing photos easier.
“Instead of needing an expert editor to spend several minutes tweaking an image frame-by-frame and pixel-by-pixel, we’d like to make the process simpler and faster so that image-editing can be more accessible to casual users,” says Yagiz Aksoy, a visiting researcher at MIT’s CSAIL.
“The vision is to get to a point where it just takes a single click for editors to combine images to create these full-blown, realistic fantasy worlds.”
Called as “Semantic Soft Segmentation” (SSS), this method uses AI (artificial intelligence) to automatically separate objects in an image that enables easy image editing. For instance, using this technique, you can change the look of the background or merge foreground and background images into an entirely new scene.
How do SSS works?
The tool studies the original image’s texture and color and combines it with information collected by a neural network about what the objects within the image actually are. The neural network processes the image features and detects “soft transitions” such as hair and grass.
“Once these soft segments are computed, the user doesn’t have to manually change transitions or make individual modifications to the appearance of a specific layer of an image,” says Aksoy. “Manual editing tasks like replacing backgrounds and adjusting colors would be made much easier.”
Pixels across the image are then associated with each other by color. This information is combined with the features detected by the neural network to estimate image layers. However, this task is not simple, as computers need to be taught how to do this.
“The tricky thing about these images is that not every pixel solely belongs to one object,” added Aksoy, who presented the paper at the annual SIGGRAPH computer-graphics conference in Vancouver this past week. “In many cases, it can be hard to determine which pixels are part of the background and which are part of a specific person.”
Where can SSS be used?
According to Aksoy, SSS in its current iteration could be used by social platforms like Instagram and Snapchat to make their filters more realistic, mainly for changing the backgrounds on selfies or simulating specific kinds of cameras.
Plans for future
In the future, researchers have the plan to work on lessening the time it takes to compute an image from minutes to seconds and to make images even more realistic by improving the system’s ability to match colors and handle things like illumination and shadows.
Source: CSAIL
