This afternoon I took in a panel discussion entitled “Future Directions of Graphics Research”. I had expected a panel of experts going into blue-sky visions of research so mind-bending it would leave us all raving lunatics, but instead I found something much different.
The Computer Graphics industry is suffering from its own success. Recent smashes like Avatar, Toy Story 3, and others have several people thinking that computer graphics is ‘done’, there’s no more research to do. The technology has matured to the point where we can not only realistically create digital actors, but completely make up alien planets and worlds. What is possibly left to do?
Such thinking is beginning to impact researchers and academics financially, as government grants are becoming more and more scarce. Shrinking government budgets doesn’t help things, so the academic community has decided to come together and write up a lengthy report to the National Science Foundation (NSF) detailing areas that still need research.
While the paper is still in development, they decided to discuss some of the areas with us and then open the panel for a ‘townhall’ format to allow the audience to pitch in with new ideas and contributions.
They broke it into five areas.
The Virtual Human
Current digital human models, be it for medical training or entertainment uses, are very basic and rudimentary. Typically working on just inverse kinematics models and rigid skeletal frames, they lack any real physical form or anatomical details. A true “virtual human” would simulate all of the various internal surfaces and organs at work inside the human body, from the flexing muscles of the arms and legs to the internal organs and skeletal structures that make it all work. Each of these surfaces would move according to real-world physical simulations and create responses identical to the real-world.
The next step after creating a virtual-human would then be to create groups of virtual humans to model interaction scenarios, and then physically correct clothing models to work with them. Most modern cloth models aren’t really physically accurate, only good for sheets or basic surfaces, a true virtual human would need cloth capable of seams, elasticity, and all of the other trimmings of modern clothing. Along similar lines, we would need a far more capable hair simulation than anything currently available.
The resulting product would have a multitude of uses ranging from the Entertainment Industry (digital actors), Medical Industry (surgical training and prosthetics design), and the Fashion Industry (modeling clothing).
Very Large Datasets
This is an area I am personally very interested in. As we continue to deploy large telescopes and sensors around the world, we find ourselves generating Terabytes of data almost daily with no efficient way to control it. Websites like Flickr and YouTube collect digital media at a rate faster than any human on earth can possibly hope to experience, and none of these trends show signs of slowing down anytime soon.
One particularly interesting datapoint presented was the recent ability to high-resolution scan the brain. With new scanning technology and better slicing equipment (30 micron slices), it is actually possible to scan slices of brain tissue with such detail to actually ascertain the electrical wiring of individual neurons. Unfortunately, to scan a 1 mm-cubed area generate 1-Petabyte of Data. Scanning a 1 cubic centimeter brain (a Mouse’s brain, roughly) would create a 1 exabyte dataset.
When working with datasets this big, most of our current algorithms break down. How do you store an exabyte dataset efficiently? It would have to be done over distributed servers to used efficiently, but how do you compute most modern structures (Kd Trees, etc) on distributed servers? How do you efficiently use resources like Cloud Computing to make this fast and efficient?
Interaction
Put simply, interaction is just communication between the User and a Computer. At it’s most fundamental, you can break this down into 3 steps:
- The user conceptualizes their intent, and then expresses this intent to the computer .
- The computer receives this expression of intent, and attempts to understand what to do to achieve the desired goal. Then it works on it and presents the result back to the user.
- The user analyzes the result, and determines if it meets the desired intents.
Traditionally, the means for expressing your “intent” to the computer were just a Mouse and Keyboard. However, modern technology such as ubiquitous sensors, cameras, and multitouch displays all add additional ways to interact with a computer.
Most importantly, they believe that such “Interaction” should be used for what they term “Interactive Experimentation”, along the lines of SimCity and Spore. Building models with semantic attributes integrated for gravity, sound, weight, and more to create virtual physics playgrounds for users to create whatever is needed.
Yes, it takes a bit of a weird turn towards the end there, I’m still not sure how we got from “Interaction” to “Spore”.
Education and Knowledge Dissemenation
In the 1980’s, print publishing went through a bit of a revolution as we were first introduced to Desktop publishing. Suddenly anyone could become an author, democratizing the process and opening writing to a whole new world of people. That revolution is happening again as publishing moves to Kindles, Nooks, and iPads as the digital revolution takes hold. Combine that with efforts from groups like Google to scan and index all printed books, and claims of the internet becoming a repository for all human knowledge start to sound less far-fetched.
As such, a new mode of textbooks is now possible. Augment classical text-form textbooks with live video or information related to current-events like Twitter or live news streams, and textbooks become a dynamic and never out-of-date source of information.
Of course, the tools for creating such dynamic information need to be built. In the talk, the example given was that Wikipedia is a great example of how massive databases can grow organically with crowdsourced involvement. However, What we need is something just as easy to use and open to the masses, but capable of much more than just text information. It would need to be capable of holding videos, photos, biological information, and much more, without requiring programming knowledge to implement it.
Modeling Simulation To Action To Design
The final area is a somewhat all-encompassing one, covering the current state of computer aided design and simulation. Currently we have pretty good blast models, CFD models, rendering engines, and more. However, all of these are separate disconnected parts, and frankly rather fragile. A CFD model that works great in one situation, generally fails in any other. The goal is to create a complete ‘reality simulator’ that can simulate all aspects simultaneously accurately, in a single integrated tool.
At this point he uttered the phrase that has driven fear into many a researcher: Our Goal is not Pretty Pictures, but to Further Understanding.
The Q&A
At this point, about an hour had gone by. Personally, I was very disappointed by everything I had heard. Aside from the one mention of “Very Large Data”, almost none of what I’ld heard fell into the realm of computer graphics. Important yes, even fascinating and exciting. But not computer graphics.
My thoughts were best summed up by one questioner in particular who brought up that while everything they said was fine and dandy, but some very obvious stuff was missing:
- In the CG systems of tomorrow, what will replace Pixels?
- In the CG systems of tomorrow, what will replace Rasterization and RayTracing?
- How will be build the CG Systems of Tomorrow?
These are the core tenets of computer graphics, and completely left out of this “Future Directions” panel. While the idea of a “Virtual Human” is fascinating and useful, it sounds more like a biomechanics and human anatomy project, not a computer graphics project.
It’s also worth pointing out that this exact same paper was published by the IEEE back in 2006. I remember attending a VisWeek presentation on the report, and many of the same topics were discussed.
Thankfully, the paper is still under development and review, and hopefully some of these issues will be fixed before it’s finished. What do you think? Is this a ‘good’ list of future directions, or completely off the mark?
@
Brian Wyvill I agree that implicit modeling shows huge promise, things like Level Sets & PDE’s look interesting for the type of massive datasets I’m looking at these days. As simulations begin to compute in Level-Set space rather than discrete-space, it only makes sense that the visualizations follow suit.
I’ve noticed that many of what I would think would be perfect SIGGRAPH CG Research papers have begun to pop up at the IEEE VisWeek conferences, although even now that’s being overwhelmed by InfoVis. It seems that raw Scientific Visualization & Computer Graphics papers are rapidly becoming without a home.
Hi Randall,
thanks for your comments on the Future Directions of Graphics Research. I was interested in your “What Follows Pixels?” question. I was wondering the same thing but thought a more likely question would be “What follows polygons?” My own research is in the field of functional representations using implicit modelling. I have been advocating a no polygons world for a long term and was thought a freak at GDC a few years ago. SIGGRAPH has certainly not embraced papers in this area with various referees not seeing an immediate application in the graphics industry even though the implicit modelling community has solved many of the early problems. Just some thoughts! – Brian
The fact that displaying real things requires the understanding of the mechanics and biological facts of this things seems to me as a logical consequence. Computer graphics will be interwoven with many other scientific domains more and more.
But I miss two things:
– Computer Graphics for holographic devices: Holographic displays as projection technology as well as full displays are on the horizon. To mange the huge calculations, Computer Graphics can help to optimize
– Information transportation: there are a lot of info graphics, but it’s all craftsmen work or art. These have always a right of existence, but to create the efficient(with respect ot science of human recognition and aesthetic principles) information transportation graphics automatically has much space for research, I think
Holographic displays were mentioned by one of the questioners after the presentations, and included as a potential ‘turing test’ for photorealistic graphics moving forward. i think it moves back to more basic questions like “What Follows Pixels?” Maybe holographic voxels..