Welcome to the Jukebox Podcast from WP Tavern. Nathan Wrigley interviews Saumya Majumder, lead software engineer at BigScoots, about a recent major Cloudflare outage and how BigScoots builds high-performance WordPress solutions integrated with Cloudflare Enterprise.
Who is Saumya Majumder
Saumya is a lead software engineer at BigScoots specializing in high-performance WordPress engineering and Cloudflare-powered architectures. He’s built custom caching engines, migration tools, worker-based automations, and edge computing solutions. He oversees enterprise customers, internal WordPress projects and plugins, and develops scalable developer-friendly hosting solutions.
Origins and focus on caching
Saumya’s interest in WordPress and PHP began in college, leading to work on projects such as Super Page Cache for Cloudflare, later acquired by another company. He explains why CDN-level page caching matters: traditional server-side caches store HTML on the origin server, which still forces every user’s request to travel to that origin (introducing latency). Serving HTML from CDN PoPs worldwide reduces geographic latency, making pages load much faster for remote users.
Cloudflare outage: what happened and why outages occur
Saumya explains that modern internet systems are complex, with many interdependent components. Cloudflare is much more than a CDN—its platform includes many services and a large global footprint. In large systems, a single unexpected issue can cascade. He notes there’s no such thing as an infallible system: Google, AWS, Azure, Cloudflare, or even a private data center can experience failures.
Cloudflare’s incident involved an unexpected file size change and propagation patterns that made the situation appear like a DDoS, which led engineers initially down an incorrect path. Fixing the root cause and stabilizing global control plane and edge propagation takes time. Saumya emphasizes that outages are costly not only for reputation but financially—enterprise SLAs can require credits or refunds when uptime guarantees aren’t met.
He praises Cloudflare for transparency: they publish detailed postmortems explaining causes and mitigations. Cloudflare learns from each outage (e.g., previously removing dependencies on external services and hardening generator failovers) and applies changes to reduce recurrence. But even with reviews and precautions, tiny probability edge cases can still occur.
How BigScoots mitigated impact for customers
For customers whose sites are hosted on BigScoots but proxied through Cloudflare, BigScoots used Cloudflare’s API to temporarily turn off proxying, directing traffic straight to their origin servers until Cloudflare stabilized. Because Cloudflare’s control plane (API) remained available, BigScoots could automate rapid failovers and reduce downtime for customers. This approach doesn’t help customers who host directly on Cloudflare Workers or Cloudflare’s hosting platform, since those run entirely inside Cloudflare’s infrastructure.
Cloudflare Enterprise and CDN-level page caching
Saumya explains CDN-level page caching versus server-based caching:
– Server-based caching stores HTML on the origin server (e.g., via advanced-cache.php). This saves origin compute but still requires requests to traverse the internet to the origin, causing latency for distant users.
– Traditional CDN usage often cached only static assets (CSS, JS, images) while HTML still came from origin.
– CDN-level page caching stores full HTML on CDN PoPs globally. Users fetch HTML from a nearby PoP, dramatically reducing latency.
Cloudflare Enterprise enhances this with very high cache hit ratios, tiered caching, and regional tiers. Tiered caching organizes PoPs into lower and upper tiers; a lower-tier PoP checks an upper tier first before hitting origin. The upper tier can pull from origin and serve multiple lower-tier PoPs, all within Cloudflare’s private backbone (an internal network), which speeds responses and reduces origin load. This architecture reduces redundant origin requests and improves global cache persistence.
BigScoots’ Cloudflare integration and private peering
BigScoots was an early adopter of Cloudflare Enterprise in hosting. They built BigScoots Cache, a proprietary plugin that integrates with Cloudflare page caching and provides granular control. Additionally, BigScoots maintains direct physical connections to Cloudflare via private links (CNI) and dedicated fiber between their data centers and Cloudflare’s network. This private peering ensures that when Cloudflare needs to fetch content from BigScoots origins, it travels over a private, high-bandwidth channel rather than the public internet, improving speed and reliability.
BigScoots Cache plugin and features
BigScoots Cache sits on WordPress sites to orchestrate Cloudflare page caching and provide advanced options:
– Fine-grained cache TTL controls and hooks for developers to adjust TTLs for specific pages.
– Intelligent cache purging: when a post is published or updated, BigScoots Cache clears not just that page but related pages (taxonomies, archives, author pages) to keep content consistent across caches.
– Hooks and APIs: developers can programmatically trigger cache clearing and integrate cache control into other systems. BigScoots Cache exposes a REST API to clear cache from external apps.
– Advanced rules and managed assistance: if a site uses nonstandard archives or complex theme behavior, BigScoots can implement custom logic or provide snippets to keep caches accurate.
BigScoots portal and enterprise control
BigScoots provides an enterprise settings interface with fine-grain Cloudflare controls:
– Toggle protections for login pages, apply proprietary hardening, manage image optimization, enable Rocket Loader, and block or challenge specific bots or AI crawlers.
– Geo controls to block or challenge traffic by country or continent.
– These settings are designed to be simple for non-technical users, while offering advanced hooks and APIs for developers and agencies.
Managed onboarding and enterprise support
BigScoots handles migrations with zero downtime and provides onboarding, performance optimization packages, and engineering services. For enterprise and agency customers, BigScoots offers hands-on migration, performance tuning, and custom engineering—writing code or automation when needed. The portal settings let many tasks be self-served, but BigScoots’ managed support can implement configurations directly for customers who prefer assistance.
Target audience and real-world use
While BigScoots’ features appeal to agencies and enterprise customers, Saumya notes many smaller businesses use the controls too—blocking or challenging traffic from specific countries or resolving analytics issues. The goal is to provide both powerful defaults and accessible tools so a broad range of customers can benefit.
Final notes
Saumya emphasizes that the internet’s complexity makes outages inevitable, and responsible providers mitigate and learn from them. Cloudflare’s scale and engineering make it a powerful platform, with transparent incident reporting and continuous hardening. BigScoots leverages Cloudflare Enterprise, proprietary caching, private peering, and managed services to deliver fast, resilient WordPress hosting and to reduce the impact of upstream infrastructure incidents for their customers.
For more details, the episode and links are available on wptavern.com/podcast.
