Sean Christmann has some new demo’s up on CraftyMind showing some of the fun stuff that HTML5’s Canvas and Video tags enable.  He shows how you can, relatively easily, interact with live video with a simple “explosion” demo and rotate it in 3D.   He also brings this up:

Don’t ask me why, but copying pixel data out of a video tag is expensive, so expensive that drawing it into a temporary canvas, and then drawing pieces of that temp canvas onto a final canvas is faster then just referencing the video tag repeatedly within the same loop. That’s why you’ll see 2 Canvases in the source code for the demos. I’m sure there’s a technical reason for this duplication process, but it’s a lazy reason.

My guess would be that accessing the Video object directly gets into many levels of very complex API’s for decoding the live stream, simply adding a lot of overhead as it finds keyframes, re-decodes to the requested frame,  and deals with the overhead of various codecs.  Once you copy it into a canvas, access is a simple (x*width+y) operation.

via Blowing up HTML5 video and mapping it into 3D space « Craftymind.