David McCutchen is the CTO of Immersive Media, the company behind the recently discussed Dodeca2360. He took the opportunity to sit down and talk to us (via email) about spherical video, Immersive Media, and the Dodeca2360 in an exclusive VizWorld interview.
VW: How did the idea for Spherical Video come about?
DM: When you think about it, everybody actually lives at the center of a spherical field of view. If you could capture that image in every direction, in full color, motion and sound, then you could reproduce what a real experience is like. Lots of people have tried to fake it with virtual reality, but photographic immersive video shows us the real thing.
Click through to read the rest.
VW: How long has ImmersiveMedia been in Spherical Video?
DM: I first started on this back in 1989. The company was founded in 1994, and we shot the first immersive movie in the world in 1995 with an earlier Dodeca camera, which was debuted at SIGGRAPH that year.
DM: It’s a compact ball of sensors with four-channel sound, streaming the images from every direction to be recorded at a full 30 FPS on a high-capacity hot-swappable hard drive. You can also incorporate other data streams such as GPS and external sound into the recording. Our special software automatically turns the recordings into immersive movies, which can be streamed in Flash over the web for everyone to enjoy.
DM: A dodecahedral geometry is a more optically efficient approach for better image quality. It uses what is called the “quality circle’ from each lens image, which is the circular area within the rectangular image that touches the top and bottom of the frame, and the corners are used for the blending between the images. It also has many other advantages, such as a super-simple geodesic geometry, more pixels from more sensors, and even the ability to make an underwater housing with an invisible seam around the equator. (BTW, the Yellowbird looks like the Ladybug, which has been around for years, but is red instead of yellow.)
A Dodecahedron
DM: We started so long ago that digital video didn’t exist. We had to do it the hard way with analog tape decks and analog everything. The development stages have not only been to make it lighter and smaller and more portable, but also to make everything digital. Now the latest models are small and tough enough to do BASE jumping with.
DM: The individual sensors are standard-definition 640×480 square pixel sensors. Hardly any resolution is lost because of the dodecahedral geometry makes the most of the image (as described above).
VW: What’s in the magic black box all of this is connected to? How much horsepower is required for the stitching & blending operations? Is it all done real-time, or is there a lengthy post-processing stage to generate the final movie?
DM: The black box, which we call the Base Unit, has a proprietary operating system and other hardware to get the absolute most out of the hard drive recording and the gigabit ethernet data stream from the camera. It has multiple independent NTSC video monitor channels for looking at the camera or playback image, including in a real-time blended immersive window or even a full-world view. There can also be a real-time full world display, at a lower resolution and frame rate, using a small tablet or PC attached to the gigabit Ethernet output port on the Base Unit. The editing and review process with the Base Unit is very fast, using a game controller, where you can look around with the right joystick while scrubbing back and forth with the left, from slow motion to 120X speed, to find and mark the cuts. Then the Windows software, on anything from a laptop to a workstation, reads the disk to blend and encode the images to make the movies. For best results, the process can take longer than real time, but it’s still pretty fast because it’s using the GPU and CPU resources.
VW: What codecs are used for the resulting spherical video? Is it something mainstream (h264, etc) or something custom?
DM: You can use any codec you like, but many of them can’t handle the unusually large frame sizes of the full immersive frames. We’ve used a relatively uncompressed MJPEG codec for quick review, some WMV, and Flash for streaming, in both the On2 and H.264 varieties.
VW: What are the typical uses for this camera? I see some interesting information about GIS uses for blueprints & planning, along with some interesting stuff on data-fusion. What else? Examples?
DM: Like video in general, it’s got lots of uses, with more coming up all the time. It’s been all over the world for capturing everything from ad campaign action (Red Bull, Mercedes, Armani, etc.), to street surveys (Google Street View). You can certainly do a lot with data fusion, both because of the metadata recorded with it (GPS, IMU stabilization, etc.) and because the immersive movie is based on standard video forms, such as AVIs and Flash.
DM: The new Flash system is real-time immersive streaming video with sync sound, so anybody can tune in and be part of the action. The Armani ad was made from a complex mix of prerecorded pieces. You can also see lots of other examples of Flash movies on our web site.
VW: What functionality is available in the SDK, that’s not already available in the current offerings?
DM: The SDK is available for license with a long list of functions for an immersive player and playback environment.
DM: We have both models that are even smaller, and a larger model that has a very large resolution. We have development facilities in both Portland and Dallas dedicated to the very best in immersive video, and camera crews out shooting around the country.
You can see examples of the Dodeca2360 and the rest of Immersive Media’s spherical video technology at their website, immersivemedia.com .
I’ve been looking into this since seeing the iPhone camera 360 video attachment (where the quality looks as good as this Immersive Media system demos): http://www.engadget.com/2011/04/05/gopano-micro-brings-360-degree-video-recording-to-the-iphone-4/
As for Immersive Media it looks like they went out of business then were repurchased by some of the former management in early 2011. In looking around for Immersive Media video, what was surprising was that they now seem to have porn online?
http://www.digitalplayground.com/v360.php (very NSWF), in a video watermarked with Immersive Media’s name and in their player.
So why would someone buy a very expensive system when you can do the same work with your iPhone now?
Would be nice to see a follow up article on these people. BTW, it’s some of the worst looking porn video despite the 360 spherical feature ever filmed.
@mike_j What are you basing this one? Their Flash Implementation or the real video? Hopefully not the Armani ad, that’s not using their new Flash technology AFAIK.
Interesting idea but really lousy image quality. Its a blurry mess and has buffering problems on a very high speed connection. Give it a few years.