Earlier this week we spoke about CRL’s ‘Eka’ supercomputer in Asia that was used to render “Roadside Romeo”, and boasted an impressive 10x speedup in render times.  Well, one of the guys that works on Eka was kind enough to send in a bit more information about the system and some of the innovations that lead to such an impressive improvement.

There are key aspects in ‘eka’ which differentiate itself from traditional rendering systems:
a) The nodes are external storage are interconnected using a 20 Gbps infiniband as compared a gigabit ethernet connection in majority of other renderfarms
b) The storage throughput that we get is approximately 4 GBps, which means that the compute nodes have to seldom wait for i/o
c) The latency levels are less than 2 micro seconds

All in all, ‘eka’ has been architected as a supercomputer and not a renderfarm. Our main strength is our architecture which has been completely developed in house using off the shelf components. Animation is one of the verticals that we support/extend services to, and also have a lot more complicated applications like engineering, drug discovery, oil and gas which run on the system.

There is definitely truth in the earlier comment that the overall turnaround time can be reduced due to the availability of massively parallel resources (‘eka’ has 1800 compute nodes, each of which is a dual quad core 3 GHz clovertown).

However, we have also seen speed gains even when you do not do split frame rendering/render just one pass where we have seen that passes which taken 2 minutes or so on customer renderfarms have taken 50 – 55 seconds on ‘eka’.

I/O is increasingly becoming the main bottleneck of cluster systems, as Hard Drives have not kept pace with improvements in CPU and Memory.  However, large parallel filesystems and the increasingly supply of SSD’s seem to be alleviating this.

Tags