Microsoft unzips Zipline, lets world+dog have a go with cloudy storage compression tech

Microsoft used the Open Compute Project (OCP) Global Summit to announce the open-sourcing of the company's cloudy compression technology, Project Zipline.

Disappointingly not a wire strung between data centres, along which techies can whizz with armfuls of USB sticks - an aerial Sneakernet if you will - Project Zipline is all about a compression algorithm, aimed at addressing the exponential growth of data created across the world.

Those playing the Microsoft drinking game - namely taking a swig as soon as "Artificial Intelligence" is mentioned in the company's emissions - will be delighted to know that the company is waving the finger of blame at the technology as more and more datapoints are required to feed algorithms that drive everything from business processes through to taking a stab at predicting a meatbag's upcoming schedule.

Pointing to IDC forecasts that over 22 ZB of storage capacity must ship across all media types from 2018 to 2025 to keep up with storage demand, Microsoft has been working on methods to prevent its Azure data centres bursting at their seams while data-intensive workloads continue to grow.

Enter Project Zipline, a compression algorithm which, along with some optimised hardware, is geared up for the types of data Microsoft has spied in its cloud storage. We checked with Microsoft to find out if that included a look at customer bits and bytes, but have yet to receive a response.

Naturally, the company reckons that it has "been able to simultaneously achieve higher compression ratios, higher throughput, and lower latency than the other algorithms that are currently available" before casting a little shade on the commonly used Zlib-L4 64KB model. Microsoft claims the Project Zipline tech can achieve up to 2X high compression ratios in comparison.

Microsoft is open-sourcing the compression algorithms, hardware specs and Verilog code in the hope that the community and hardware vendors will pick it up and run with it. Contributions from the OCP ecosystem wouldn't go amiss either - after all, everyone benefits, right? "Including Azure and our customers" according to the software giant. ®

* Kids' puppet show on Brit channel ITV

Software News

Apr 19
It's not hard, we just need some coding tweaks to make sure Privacy Badger stays sane
Apr 18
New Linux kernel, new build
Apr 18
InterviewCoding lingo represents an effort to realize 'regularized programming'
Apr 18
Mobe software biz suspends service, 12 people held by cops
Apr 15
Social media hits back at suggestion women are too shy to get a smear test
Apr 15
RoundupAlso: Azure goes big in Britain, Bletchley Park to teach tech and Skype shares some more
Apr 12
Something for the Weekend, Sir?Some things are better left without a video greeting