I met with Ocarina Networks founder Carter George for the first time this week and was genuinely excited about the dramatic data reduction that their cutting edge technology promises. Ocarina aims to reduce the number of bytes that go into file storage through a combination of data deconstruction, deduplication, and compression. Deconstruction breaks complex file types into basic components, deduplication spots block level similarities and replaces them with hashes, and compression squeezes the de-duplicated components beyond what traditional compression techniques can with more complex whole files.
The heavy lifting is done by an out-of-band appliance that applies post processing techniques to create size reduced shadow copies of data with “bit for bit consistency checks” to prevent corruption or loss of data. The key to the approach is a “file type aware approach,” using specialized algorithms and techniques for files like images or text. George claims that the process can reduce the size of most data types by 10X and can even yield a 3X reduction on JPEGs that typically see little benefit from deduplication techniques. For web 2.0 firms that are looking at growing their storage by multiple TB a day, a capability like this would be game changing by reducing their growth costs by leaps and bounds.
There is considerable market resistance to deduplication of primary data, where the industry has long relied on creation of multiple copies to guarantee the protection and availability of critical data. If you are an IT operations professional in the banking or financial services sector, don’t expect your peers to adopt Ocarina’s cutting edge approach any time soon. But, if your firm has less stringent SLAs around data loss and availability and faces staggering growth from day-to-day operations, this could be the missing link that allows you to stay ahead of competitors and offers more rich services to customers without breaking the bank on storage capacity. I can’t speak to the viability of technology approach yet, but suffice it to say that if they can deliver what they promise, this could dramatically shake up the enterprise data storage world and move the industry along the path to keeping up with the astonishing but widely accepted forecasts for growth of file data.
Check out Andrew’s research