It’s making a huge buzz on the internet right now and if you’re heading to SIGGRAPH ASIA then you’ll get a chance to hear alot more about it. What is it? PhotoSketch.
We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet. Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images. We also provide a novel image blending algorithm to allow seamless image composition.
Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful; we also evaluate our system using the results from two user studies.
Cliff notes version: Draw a basic sketch of what you want and label the components (as shown above), and the system automatically extracts the necessary components from a huge database and composites them together. The research consists of 2 main parts: A new blending algorithm to make the images fit in their new environments much better, and a new filtering and classification system for images.
You can read the full PDF of their work here (with many more examples than the sailboat/kiss floating around everywhere), and see a great demonstration video after the break.
Amazing 🙂