m‎ > ‎4‎ > ‎

m

Developer
  • Microsoft open sources its Azure Container Service Engine and launches deeper Kubernetes integration

    Microsoft open sources its Azure Container Service Engine and launches deeper Kubernetes integration

  • Google’s Firebase developer platform gets better analytics, crash reporting and more

    Google’s Firebase developer platform gets better analytics, crash reporting and more

  • CoreOS introduces Operators to streamline Kubernetes container management

    CoreOS introduces Operators to streamline Kubernetes container management

  • Browse more...

Facebook
  • Facebook threatens LinkedIn with job opening features

    Facebook threatens LinkedIn with job opening features

  • WhatsApp adds GIF support in iOS app

    WhatsApp adds GIF support in iOS app

  • WhatsApp is testing a clone of Snapchat Stories, called Status

    WhatsApp is testing a clone of Snapchat Stories, called Status

  • Browse more...

data storage
  • How storage is changing in the age of big data

    How storage is changing in the age of big data

  • Komprise Raises $6M To Help Companies Rein In Sprawling Data

    Komprise Raises $6M To Help Companies Rein In Sprawling Data

  • NimbusBase Launches An iCloud For Any Platform At Disrupt NY, Lets Users Store Their Data On Dropbox Or Google Drive

    NimbusBase Launches An iCloud For Any Platform At Disrupt NY, Lets Users Store Their Data On Dropbox Or Google Drive

  • Browse more...

data compression
  • MIT develops a new technique to load webpages faster

    MIT develops a new technique to load webpages faster

  • Google Adds Optional Data Compression Feature To Chrome For Mobile, Reducing Your Data Usage By Up To 50%

    Google Adds Optional Data Compression Feature To Chrome For Mobile, Reducing Your Data Usage By Up To 50%

  • Apple Acquires Swedish Firm AlgoTrim, A Company That Does Mobile Media And Data Compression

    Apple Acquires Swedish Firm AlgoTrim, A Company That Does Mobile Media And Data Compression

  • Browse more...

mysql
  • MySQL founder tries a new software licensing model

    MySQL founder tries a new software licensing model

  • Ex-MySQL CEO Marten Mickos On Leadership And The Open Source Revolution

    Ex-MySQL CEO Marten Mickos On Leadership And The Open Source Revolution

  • Linux Ransomware Is Now Attacking Webmasters

    Linux Ransomware Is Now Attacking Webmasters

  • Browse more...

Facebook open sources Zstandard compression algorithm and MyRocks storage engine

Posted Aug 31, 2016 by John Mannes (@JohnMannes)
  • 0

    SHARES
Next Story

Sunrise was the best calendar app ever and now I feel lost

Today, Facebook is releasing its Zstandard compression algorithm into the wild as open source. The lossless compression technology is aimed at replacing existing libraries like zlib that are powered by the outdated Deflate compression algorithm. In addition to Zstandard, Facebook is also dropping its MyRocks storage engine as open source. MyRocks is currently being used by Facebook to improve the storage efficiency of its MySQL databases.

Both releases occurred in coordination with Facebook’s @Scale conference in San Jose. The conference supports engineers working to solve large-scale engineering challenges by bringing them together, side-by-side, with other companies in the industry dedicated to increasing the prevalence of open-source technology. Both Zstandard and MyRocks will enable developers to build platforms that can efficiently scale to large and diverse user bases.

One of the greatest fears of engineers looking to update and replace critical infrastructure is that the new libraries could bring an entire service down. Jay Parikh, vice president of infrastructure engineering at Facebook, is proud of the fact that both Zstandard and MyRocks were tested at scale across the company before being released to the community.

“Everyone here uses all of the products,” added Parikh.

In the six months that Facebook has been testing Zstandard, the company noted impressive results. With the same compression ratio of zlib, Zstandard was five times faster. In a case where the compression time was held constant, files were 10 percent smaller.

MyRocks has also produced some serious storage efficiencies. When compared to compressed InnoDB, MyRocks was able to store the same amount of data on half the server space.

“Making it open source makes the community better,” added Parikh. “It will be adopted more quickly as open source than had if was created as the proprietary technology of a single company.”

Facebook hopes that their open source solutions can become new industry standards.

Featured Image: erhui1979/Getty Images
  • 0

    SHARES
Advertisement Advertisement

Crunchbase

  • Facebook

    • Founded 2004
    • Overview Facebook is an online social networking service that allows its users to connect with friends and family as well as make new connections. It provides its users with the ability to create a profile, update information, add images, send friend requests, and accept requests from other users. Its features include status update, photo tagging and sharing, and more. Facebook’s profile structure includes …
    • Location Menlo Park, CA
    • Categories Internet, Social Media, Social Network, Social
    • Website http://www.facebook.com
    • Full profile for Facebook

Newsletter Subscriptions

Latest Crunch Report

  • Facebook's Election Hub | Crunch Report

    Facebook's Election Hub | Crunch Report

Watch More Episodes
  • data storage
  • Facebook
  • data compression
  • mysql
  • Developer
  • Popular Posts

    #auto

    Subpages (1): 2
    Comments