Microsoft unzips Zipline, lets world+dog have a go with cloudy storage compression tech

Microsoft used the Open Compute Project (OCP) Global Summit to announce the open-sourcing of the company's cloudy compression technology, Project Zipline.

Disappointingly not a wire strung between data centres, along which techies can whizz with armfuls of USB sticks - an aerial Sneakernet if you will - Project Zipline is all about a compression algorithm, aimed at addressing the exponential growth of data created across the world.

Those playing the Microsoft drinking game - namely taking a swig as soon as "Artificial Intelligence" is mentioned in the company's emissions - will be delighted to know that the company is waving the finger of blame at the technology as more and more datapoints are required to feed algorithms that drive everything from business processes through to taking a stab at predicting a meatbag's upcoming schedule.

Pointing to IDC forecasts that over 22 ZB of storage capacity must ship across all media types from 2018 to 2025 to keep up with storage demand, Microsoft has been working on methods to prevent its Azure data centres bursting at their seams while data-intensive workloads continue to grow.

Enter Project Zipline, a compression algorithm which, along with some optimised hardware, is geared up for the types of data Microsoft has spied in its cloud storage. We checked with Microsoft to find out if that included a look at customer bits and bytes, but have yet to receive a response.

Naturally, the company reckons that it has "been able to simultaneously achieve higher compression ratios, higher throughput, and lower latency than the other algorithms that are currently available" before casting a little shade on the commonly used Zlib-L4 64KB model. Microsoft claims the Project Zipline tech can achieve up to 2X high compression ratios in comparison.

Microsoft is open-sourcing the compression algorithms, hardware specs and Verilog code in the hope that the community and hardware vendors will pick it up and run with it. Contributions from the OCP ecosystem wouldn't go amiss either - after all, everyone benefits, right? "Including Azure and our customers" according to the software giant. ®

* Kids' puppet show on Brit channel ITV

Software News

Jun 20
It looks like your OS goes out of support in seven months. Would you like a new browser?
Jun 20
'Why use a copy when the original is free, proven and battle-tested?'
Jun 20
Same digit could be extended at Google as browser chaps 'evaluate the situation'
Jun 19
Mountain View's white stuff used more than Azure among 7,000 code monkeys
Jun 19
Ugh. Fly In, again? Really? And step away from the Comic Sans
Jun 19
PaaS the parcel: Click-and-forget fortification of Redmond's cloud
Jun 19
Think we're at RISC of overdoing the Intel puns