A new press release from NVidia discusses an interesting project from the University College London Hospitals (UCLH) Heart Hospital and UK Visual effects company Glassworks. Together they build a virtual heart simulator perfect for transesophageal echocardiography (Imaging the heart through the mouth & esophagus) training. Using the impressive power of NVidia Quadro GPU’s and 3D Vision technology, they’re able to maintain realistic stereoscopic images running at a full 30fps. In fact, it’s already an integral part of training at Duke University.
“Simulation technology has enabled us to take a quantum leap forward in our teaching,” said Dr. Madhav Swaminathan, MD, FASE, FAHA, of Duke University School of Medicine’s Division of Cardiothoracic Anesthesia. “This particular system essentially simulates the beating heart clearly. To explain how an ultrasound image is formed and how it correlates to anatomical features is extremely difficult. When you’re changing the image plane with a probe it’s hard to understand what parts of the heart you are seeing on the screen — because the heart is three dimensional, and you’re using 3D on a 180 degree plane. A simulator makes it possible to see side by side not only how an ultrasound image is generated, but what the cuts mean in a controlled, relaxed environment where you don’t have to worry about interfering with a patient’s clinical care or taking too much time. This virtual environment technology gives residents a jump start.”
An interesting note, this is done without CUDA. Currently, the application simply uses OpenGL and GLSL, meaning it should run on ATI cards as well. However, in the end they state they’re investigating improving performance by moving lots of the system to CUDA.
Get the full release after the break.
UCLH Heart Hospital Teams with Glassworks to Create HeartWorks
CHALLENGE:
In 2006, three clinicians at University College London Hospitals (UCLH) Heart Hospital, Drs. Sue Wright, Andrew Smith, and Bruce Martin, got together to share their frustration about the lack of a realistic model of the human heart that could be used to teach cardiac anatomy. They came up with the idea of creating a virtual heart, and quickly realized that if it were based on an anatomical 3D data set, such a model could be used to generate a simulated ultrasound image. The idea had significant implications for training, as it could also be used to simulate the experience of transesophageal echocardiography (TOE or TEE), an examination difficult to practice, as it relies on inserting an internal probe to capture images of a patient’s beating heart.
When these clinicians shared their idea with the team of creative artists at award-winning, UK-based visual effects company Glassworks, things began to take shape. The goals were to create an anatomically accurate, computer-generated heart model that could instantly produce an authentic ultrasound representation, be animated to beat in real-time to show changes in the heart’s shape during the cardiac cycle, and enable the doctor to view slices of the scene to gain valuable diagnostic insights.
SOLUTION:
Glassworks develops the virtual heart model
The clinicians and Glassworks named the project HeartWorks, and relied on the high performance computing and visualization power of NVIDIA Quadro professional graphics solutions in the development and use of this first-of-its-kind clinical training tool.
A small team of artists and developers at Glassworks began collating a vast set of digital heart images, including attending open-heart surgery to experience the human heart beating live. Wright, Smith, and Martin worked with leading surgeons, cardiac morphologists, ultrasonographers and other experts to consult with Glassworks as the system began to develop.
To create the complex realistic heart model and real-time animations, the Glassworks artists and animators used Autodesk Softimage 3D animation software running on workstations equipped with NVIDIA Quadro graphics processing units (GPUs). These professional graphics solutions provided the processing power necessary to render high quality images at 30 frames per second — keeping the animations realistic and smooth.
“Immersive technology, where users can interact with 3D images on a screen, has been limited historically by the state of graphic technology, and by budgets,” said Hector McLeod, founder of Glassworks. “NVIDIA’s breakthrough with their massively parallel GPU technology has changed that.
What the NVIDIA GPU does within HeartWorks is load and display extremely complicated models, and render them at 30 frames a second, in a series of events that are themselves, complex. In any given frame we’re tapping the GPU to examine the model, slice it, annotate the views and display the two visualizations — the model and the ultrasound — in less than 1/30 of a second.
Glassworks software engineer David Llewellen added, “NVIDIA Quadro is the best professional graphics solution, easily handling the sheer quantity of data we’re pushing through it. The model has a high polygon count — 250,000, and we’re generating two simulations, plus have a whole system of anatomic labels. To get the model to look graphically superior we also have highly detailed textures; the majority using OpenGL and running GL Shader Language. We need all of that processing power at once to make sure HeartWorks delivers a real-time experience.”
Glassworks and the UCLH Heart Hospital clinicians commercialized HeartWorks through Inventive Medical Ltd. in London, which integrates, installs and support the system for hospitals, labs and universities. When it is delivered to clients, the HeartWorks product is a turnkey system that includes the HeartWorks software — the interactive, virtual heart model and ultrasound simulation program; a high-performance workstation equipped with an NVIDIA Quadro professional graphics card; a monitor/keyboard/mouse; and a probe and torso mannequin to enable a hands-on experience for teaching the TOE/TEE procedure.
IMPACT:
HeartWorks in Action At Duke University
One of HeartWorks’ early adopters was Duke University; specifically, the Department of Anesthesiology, Division of Cardiothoracic Anesthesia and Critical Care Medicine. The Department bought the simulator early in 2009 to teach residents and fellows in their advanced Transesophagael Echocardiography program. First, second and third year residents use it primarily to learn basic echocardiographic views and anatomy, and fellows in the advanced program utilize it to examine more subtle features.
“Simulation technology has enabled us to take a quantum leap forward in our teaching,” said Dr. Madhav Swaminathan, MD, FASE, FAHA, of Duke University School of Medicine’s Division of Cardiothoracic Anesthesia. “This particular system essentially simulates the beating heart clearly. To explain how an ultrasound image is formed and how it correlates to anatomical features is extremely difficult. When you’re changing the image plane with a probe it’s hard to understand what parts of the heart you are seeing on the screen — because the heart is three dimensional, and you’re using 3D on a 180 degree plane. A simulator makes it possible to see side by side not only how an ultrasound image is generated, but what the cuts mean in a controlled, relaxed environment where you don’t have to worry about interfering with a patient’s clinical care or taking too much time. This virtual environment technology gives residents a jump start.”
Dr. Swaminathan further explained that the primary advantage of the simulation technology is the real-time interaction residents and fellows experience. “When you can see the heart beating and modify aspects of it, cut it, and manipulate it at will versus being forced to do it the way a teacher wants is a significant breakthrough in echo education.”
Coming Next – A New HeartWorks Application
Following its success with the TOE/TEE application simulator, the UCLH Heart Hospital/Glassworks team began developing another HeartWorks device, a TTE, which simulates the experience of an external ultrasound examination. It utilizes the same virtual heart model as the original HeartWorks application, but the interaction via probe generates a slice plane that goes through the lungs and ribs, which are also visualized.
In the development of the new TTE application, Glassworks is investigating further leveraging NVIDIA’s Quadro GPU technology by writing shader tools in the NVIDIA CUDA programming language to give them an additional performance boost. “We’d love to make further optimizations that make things run even faster, and look even better,” said Glassworks’ Llewellyn. “We will make full use of whatever developments NVIDIA brings along next.”
