Cache – Pingora now powers Cloudflare’s cache

Cloudflare’s cache now runs on a new proxy built on Pingora, the Rust-based framework that already serves a significant portion of Cloudflare’s network traffic. The new proxy is faster, more memory-safe, and designed to evolve our cache architecture. It delivers immediate performance improvements and enables new caching capabilities.

What this brings

• Lower latency: The new proxy reduces per-request overhead through improved connection reuse.
• Reduced cache MISSes: Enhanced cache retention improves origin offload.
• Better RFC compliance: Caching behavior more closely follows HTTP caching standards.
• Foundation for future features: The new architecture enables upcoming improvements to cache functionality and efficiency.

New features

• Asynchronous stale-while-revalidate: Every request returns stale content immediately while revalidation happens in the background, instead of the first request after expiry blocking on the origin. Refer to the asynchronous stale-while-revalidate changelog for details.
• Unbuffered bypass by default: Responses that bypass cache are streamed directly to the client without buffering, reducing time-to-first-byte for uncacheable content.

Behavioral changes

The new architecture introduces the following behavioral changes to improve RFC compliance and correctness:

• Vary: * results in cache bypass: According to RFC 9110 Section 12.5.5, a Vary header value of * indicates the response varies on factors beyond request headers and must not be served from cache. Cloudflare now bypasses cache for these responses instead of storing them.
• Set-Cookie stripped on MISS and EXPIRED: For cacheable assets, Set-Cookie is now stripped on MISS and EXPIRED responses, not only on HITs.
• Floating-point TTL values: Floating-point time-to-live values (for example, max-age=1.5) are rounded down to the nearest integer instead of being rejected as invalid.

What’s next

A deeper look at the new cache proxy is coming soon to the Cloudflare blog. For background on the underlying framework, read:

• Open sourcing Pingora: our Rust framework for building programmable network services
• How we built Pingora, the proxy that connects Cloudflare to the Internet

Source: Cloudflare

Latest Posts