Cloudflare

cloudflare Achieves Sub-Millisecond performance with Quicksilver v2,a Next-Generation Key-Value Store

SAN FRANCISCO,CA – November 8,2023 – Cloudflare,a leading provider of internet services,has announced significant advancements in its key-value store technology with teh rollout of quicksilver v2. The upgrade delivers consistently low latency – responding to 90% of requests within 1 millisecond and 99.9% within 7 milliseconds – while managing a massive 1.6 terabyte dataset containing over five billion key-value pairs across its global network of 330 cities.

Originally designed to power Cloudflare’s Workers platform, Quicksilver serves as a critical component for rapidly delivering data to users worldwide. The initial Quicksilver architecture faced challenges in scaling efficiently across Cloudflare’s extensive infrastructure. Quicksilver v2 directly addresses these constraints, maintaining performance while expanding capacity.

The core innovation lies in a multi-layered caching strategy. Developed by Cloudflare engineers Daniel Dort-Golts and Jeroen van de Sanden,the system employs both per-server local caches and data center-wide sharded caches. Each server within a data center maintains its own local cache, alongside a cache representing one shard of the overall dataset. This sharding distributes the data across servers within each data center, creating a distributed, data center-wide cache.

When a key is requested, the system first checks the server’s local cache. If the key isn’t found, it then searches the data center’s sharded cache. Only if both caches miss does the request proceed to the storage nodes. This tiered approach dramatically reduces latency and improves cache hit rates.

According to the authors, the addition of the second caching layer significantly improved data resolution within each data center. The worst-performing instance now boasts a cache hit rate exceeding 99.99%, while all other instances achieve hit rates above 99.999%. This represents a considerable enhancement over previous iterations.

Performance analysis reveals minimal latency difference between proxies and replicas, with proxies occasionally exhibiting faster response times due to smaller on-disk datasets. This optimization is particularly crucial for Cloudflare’s edge computing services, where minimizing latency is paramount.

Source: Cloudflare blog

Background: The Rise of Edge Computing and Key-Value Stores

The demand for low-latency data access is driven by the growth of edge computing, which brings computation and data storage closer to the end-user. Key-value stores, like Quicksilver, are fundamental to edge computing architectures, providing a simple and efficient way to store and retrieve data. Cloudflare’s Workers platform, launched in 2017, exemplifies this trend, allowing developers to deploy serverless code directly to Cloudflare’s edge network.

traditional database systems often struggle to meet the performance requirements of edge computing due to their centralized nature and complex architectures. Key-value stores, with their simplified data model and focus on speed, offer a compelling choice. Quicksilver v2 represents a significant step forward in optimizing key-value store technology for the demands of a globally distributed, low-latency internet.