Vault 2016 has ended
Back To Schedule
Wednesday, April 20 • 10:30am - 11:20am
Facebook Warm Storage – Next Generation Storage for Data Warehouse in Hadoop Ecosystem - Kestutis Patiejunas, Facebook

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Introduction Very few companies run 24/7 clusters at the scale like Facebook using Hadoop/HDFS. As scale increases HDFS shows its limitations in scalability, performance and availability. HDFS was first designed 10+ years ago based on the assumptions that were valid in 2005. Over the years, numerous significant improvements have been made to the system. However storage, network and compute technology changed at different ratios during last decade. In this paper we describe Warm Storage - grounds up solution that exploits technology inflection points that exist in 2015 and we believe will continue into the future. We describe architecture and some of the implementation details of the novel and more efficient and available storage system optimized for DW workloads.


Kestutis Patiejunas

Kestutis Patiejunas is a software engineer at Facebook. He works with a team of engineers on Facebook Warm and Hot Storage – in internal replacement for HDFS. Before he worked on Cold Storage – system for long term data archival where he created hard drive based cold storage and... Read More →

Wednesday April 20, 2016 10:30am - 11:20am PDT
Congressional Room AB