Making Next.js caching behave at a million requests a day

Situation

The Goodpods webapp, built on Next.js, served over a million requests a day. Podcasters frequently messaged us saying their podcast page had not updated with their newest episodes, and sometimes users got a stale version of the app itself after a deploy. On top of that, Next.js was eating a lot of CPU and memory serving those pages.

Task

We went digging and traced the problems to our caching layers. I took on getting the caching under control: pages should serve fast from cache, refresh in the background, and never get stuck stale.

Action

The behaviour I wanted was stale-while-revalidate. It was fine to serve an old podcast page to one user while reloading it for the next, because that beat a slow load that made users bounce or hurt our SEO ranking. Next.js did not support stale-while-revalidate, and nginx could serve cached pages much faster than Next.js anyway, so I put an nginx cache in front of the app and let nginx handle the revalidation. I centralised the cache-control headers in one place so podcast and episode pages sent consistent s-maxage values. Next.js’s own caching kept getting in the way. Some pages cached forever, so I added explicit fallback rules to stop caching them. Statically generated pages cached for a year and broke things, so I stopped using static pages entirely. Our rsync deploys were corrupting the .next static folder, which I fixed by preserving the static cache folder and excluding it from the sync. The last symptom was stale JS bundles after a deploy. I tried several rounds of cache busting, including adding a build ID to each bundle item, before finding the real fault was a fallback image component. I fixed that component properly and removed the cache-busting hacks.

Result

Podcast and episode pages served straight from the nginx cache while refreshing in the background, at over a million requests a day, with the serving load taken off Next.js. The year-long static caching that caused more harm than good was gone. The cache-busting workarounds came out once the underlying image component was fixed, which left the caching setup simpler than when I started.