Object Storage on CRAQ: High-Throughput Chain Replication for Read-Mostly Workloads

Abstract

Massive storage systems typically replicate and partition data over many potentially-faulty components to provide both reliability and scalability. Yet many commercially- deployed systems, especially those designed for inter- active use by customers, sacrifice stronger consistency properties in the desire for greater availability and higher throughput. This paper describes the design, implementation, and evaluation of CRAQ, a distributed object-storage system that challenges this inflexible tradeoff. Our basic ap- proach, an improvement on Chain Replication, maintains strong consistency while greatly improving read through- put. By distributing load across all object replicas, CRAQ scales linearly with chain size without increasing consis- tency coordination. At the same time, it exposes non- committed operations for weaker consistency guarantees when this suffices for some applications, which is espe- cially useful under periods of high system churn. This paper explores additional design and implementation con- siderations for geo-replicated CRAQ storage across mul- tiple datacenters to provide locality-optimized operations. We also discuss multi-object atomic updates and multicast optimizations for large-object updates.