Content-aware image resizing (also called content-aware image retargeting, content-aware image scaling) is to enlarge or reduce the size of an image with minimum distortion of salient or important areas of the image. This is a very useful image processing feature for professional photography, online publishing and handheld-device display.
Adobe Photoshop CS4 has recently released with a new feature, content-aware scaling. A recent iPhone application, PicTricks, also provides a feature of content-aware image resize, Magic Resize.
This technology was originally published in ACM SIGGRAPH 2007 (S. Avidan and A. Shamir, Seam Carving for Content-Aware Image Retargeting, SIGGRAPH 2007), by a research scientist at MERL, Shari Avidan, who moved to Adobe Research Lab in Cambridge, MA in 2008. Adobe licenses MERL’s IP for this technology. We are not sure about which technology PicTricks uses. If they use the same technology as MERL’s, will there be an intellectual property issue?
The technology is called “Seam Carving“, i.e., based on the evaluation of high importance pixels (people, buildings, etc.) and other pixels, the algorithm(graph cut) detects seams (continuous lines of pixels joining opposite edges of the image) which have the lowest importance, and deletes those pixels or add interpolated pixels along the lowest importance seam. The image operations are done in gradient domain. The following video gives more details of the technology.
Later on, the authors extend the image retargeting work to videos, with another SIGGRAPH paper (M. Rubinstein, A. Shamir and S. Avidan, Improved Seam Carving for Video Retargeting, SIGGRAPH 2008) and the following demo video.
Will video retargeting go to iPhone or Adobe’s After Effects? From the computation point of view, it is unlikely this application will go to iPhone in the near future, due to iPhone’s low processing power, but it is possible that video retargeting will appear in the next release of Adobe’s After Effects.