Mental Canvas visual

Courtesy Julie Dorsey, NSF

Mental Canvas is an NSF-funded digital sketching application that allows 3D exploration of 2D drawings. But in 2015, is drawing still relevant? “There’s nothing on the computer that can touch it,” says Julie Dorsey.

For Julie Dorsey, sketching is a way to solve uncertainties; it’s a tool for trying out possibilities. Dorsey is the creator of Mental Canvas, a sketching technology that occupies the intersection between 2D drawing and 3D modeling. Thanks to forward-looking investment from the US National Science Foundation (NSF), this new media form may transform the way we design.

Historically, scientists have used sketchbooks to tease ideas apart — they draw because they seek answers. While the digital revolution has enhanced fine arts like music and photography, it has added little value to drawing, Dorsey argues.

Dorsey, a professor of computer science at Yale University, realized that the languishing state of free-hand drawing was actually hurting productivity across several domains. Combining the computer science and design disciplines, she set out to find a more direct way to work with sketched shapes during the crucial ideation stage.

At its core, Mental Canvas is a graphic design system that fuses photographs and hand-drawn shapes to duplicate real world geometry and space. It quickly adapts these 2D drawings into 3D representations, and lets a viewer reposition a sketch to see what’s behind it or to the left or right.

Postcards from the edge and beyond. Mental Canvas technology returns sketching to its place in the creative process. “Mental Canvas grew out of an idea that drawing really is important,” says Julie Dorsey, inventor of the technology. “Unlike a CAD tools, sketching allows for ambiguity and exploration and doesn’t require a known set of dimensions or a well-defined geometric form. Sketching multiple takes and viewpoints allows coherence to arrive gradually.” Courtesy Julie Dorsey; NSF.

“It’s really applicable to anyone who draws or sketches as a regular part of their workflow,” Dorsey says. “This technology gives us the freedom to try something out and add something to it without having to impose a level of detail or resolution not yet achieved. Coherence is obtained gradually.” Rather than rush to a constraining Computer-Aided Design (CAD) model, Mental Canvas allows designs to evolve in a more continuous way.

Too often, an artist’s individuality is lost in digital illustration, but not so with Mental Canvas. Everything remains hand drawn and reflects the hand of the illustrator. A good example is The Other Side, a 3D expansion of Istvan Banyai’s 2005 book of the same name. Available as a downloadable app, The Other Side lets readers pan, zoom, and fly through each illustration – tilting the tablet even moves the picture in true 3D fashion.

The Other Side is the first commercial application of Mental Canvas, and is enough to indicate Dorsey has discovered a new media form. Along with it, she has issued a call to slow down, explore ideas, and not rush into constraints that short-circuit the creative process.

In a time of lightning computation and instant connection, Dorsey’s technology encourages reflection before a rush to production. This parallels the patient fostering afforded by the NSF as her idea came to fruition. Mental Canvas began as a loose idea she wasn’t even sure would go. However, a receptive NSF program director saw potential.

Dorsey initially received a Small Grants for Exploratory Research (SGER) award, allocated for high risk, yet promising research. After further development, her technology had progressed enough for an Early-concept Grants for Exploratory Research (EAGER) grant for research that shows signs of transforming its domain but remains untested.

In 2012, Mental Canvas had evolved to such a state that it received an Innovation Corps (I-CORPS) grant, a NSF funding mechanism designed to usher work previously funded by public grants into a commercialization of technology. From the I-CORPS program, Dorsey’s research was funneled into the Small Business Innovation Research (SBIR) program that moves scientific discoveries into the private sector.

“Being funded by SBIR means that we could explore what seem to be the most promising ways to commercialize this technology, as opposed to being locked into an investor’s priorities,” Dorsey notes. “There was much greater focus on more rigorous and challenging potential applications that will have a higher impact in the world.”

Back when she first started on the project, the technology required high-end graphics processing units and extensive CAD training to operate. Now, it is available to the ordinary consumer using an everyday tablet like an iPad. This also means researchers are no longer tethered to heavy, expensive computers and displays — they can take their work with them.

This lightweight, mobile approach makes Mental Canvas an appealing and timely technology.

Whether you’re an archaeologist envisioning a reconstruction, or an automotive designer studying a new form, Mental Canvas provides the freedom to explore many possibilities. Interactive storyboarding even provides a sense of what a shot will look like from multiple perspectives, so it’s clear how a video or presentation might flow.

Mental Canvas returns to the drawing board and comes back with an elevated and expanded role for the act of sketching. Dorsey’s technology reminds us of the creative process’s lasting value, and the rewards of patient investments.

Breakout sketch. Mental Canvas technology bridges 2D sketching and 3D modeling, and allows users to navigate drawings fluidly. Scientists now get a 3D look at their back-of-the-envelope molecular structures; architecture students gain a new perspective on their building designs; moviemakers can find a new way to visualize storyboards; and readers get new enjoyment from their illustrated books. Courtesy Julie Dorsey; NSF.

This article was written by Lance Farrell and published originally on ScienceNode.org. Read the original article.

Tags