NextJS + Vercel + Dato Cache Tags not always working

I recently swapped a client project over to use the new cache tags – following the setup in this repo:

But it doesn’t always work, (notably when updates are being made to existing content) e.g.

  • My client makes an update to a blog post, publishes and on occasion it doesn’t update, but then if I go in make a content change and publish it shows the updated content
  • Client updates a “featured” article content type, publishes and on occasion, it doesn’t update
  • Client publishes a new article but the listing on the homepage doesn’t always update to show the new blog in the list

I’ve ended up creating a webhook that hits this endpoint to clear all the cache nextjs-with-cache-tags-starter/app/api/invalidate-all/route.ts at main · datocms/nextjs-with-cache-tags-starter · GitHub

But he’s having to use it far too often.

In this post (Introducing DatoCMS Cache Tags: Effortless, surgical page regeneration on content change) it says:

“consistently achieving a cache hit-ratio of over 90%”

Is that what we’re seeing here? Is it that the cache tags can sometimes miss a content type that’s updated?

I guess the first question is; is what we’re seeing expected behaviour and a limitation with the cache tags or have I missed something?

1 Like

Hi @robsimpson, and welcome to the Dato forum!

Sorry that your post was accidentally flagged by the spam bot at first. (It said you typed too quickly – probably copied & pasted? I’ve disabled that rule to be safe.)

Anyway, if I’m understanding you right, the cache tags work fine as long as you’re the one doing the edits, but not if your client does the same edits? i.e. it doesn’t matter what model they’re working on, or what kind of edits they make, etc… as long as you go in after them into the same record to make some minor tweak (like adding/removing a space or whatever), THEN the cache tags work?

Hmm :thinking: I wonder if it might be some sort of roles/permissions/API key type thing, where their Dato user isn’t triggering the same invalidation workflow as your user would…?

Could you please send us the actual project URL (either in a reply post, or in a DM to me, or via email at support@datocms.com – whichever you prefer) so we can take a look at the specific setup and webhook activities log in there?

Edit: Also, can we please see the source code of your invalidate tags webhooks receiver? Not the “invalidate everything” one, but the one that’s supposed to invalidate just the affected tags.

Thank you!

And just to address this part of it:

It’s definitely not supposed to! A cache-hit ratio of over 90% means that the caching is working well (when it’s working right). But that should NOT mean that invalidations are difficult. The whole point of the cache tags is to make the invalidations simpler :slight_smile:

Definitely not! It’s supposed to work the way you envisioned (i.e., either of you make an edit, webhook gets fired, published page updates a few seconds later). If it’s not doing that, it’s either a bug on our part or perhaps a subtle issue in the implementation somewhere? Once you send us the project URL we can take a look together :slight_smile:

1 Like

We are experiencing the same issues. Sometimes the content just doesn’t update. Not sure if it’s really an issue on Datos side, might also be an issue with revalidateTag on Vercel. :man_shrugging:

@kevin,

Is this something you’re able to reliably reproduce? We’ve only received a couple reports of this over the last few months, and nothing that we could consistently replicate :frowning: That makes troubleshooting really difficult, unfortunately…

Either most people aren’t having issues (or not noticing them?) or we don’t have enough people using cache tags yet… hard to say :frowning: But until we can find some replicable edge case where this keeps happening, it’s really hard to track down and fix. In every test we’ve run internally, our systems seem to be behaving appropriately.


UNOFFICIALLY: I’ve been exploring whether we can use other datastores for the cache tag index storage, just for comparison. Officially, we don’t have the resources to write and support a bunch of different adapters to various KV/DB services, but unofficially, the process should be straightforward enough that hopefully any dev can make their own adapter for their favorite service, according to their needs.

Our official example uses Turso because it has high free limits and is easy to set up. But it’s not something we can fully control (it’s not in-house and we don’t have a formal partnership with them yet). In theory it should be possible to drop in any KV store or DB and test reliability & performance on them.

That said… here is an unofficial, unsupported, early pre-alpha, possibly buggy fork of our example that uses Vercel KV instead of Turso. Please note that this is an example only (made for internal testing & debugging), and not something we intend to fully develop & release.

WARNING: Unofficial, unsupported testing branch! Use at your own risk! Click to open...

https://github.com/datocms/nextjs-with-cache-tags-starter/compare/main..arcataroger:nextjs-with-cache-tags-starter:vercel-kv

In my very limited and unscientific personal testing (only a couple hours and a few rebuilds before I ran out of credits on the free Vercel plan), this seemed to be a little more reliable… by that, I mean it successfully revalidated every single time.

Of course, our Turso implementation is supposed to work every time, too. I tried very hard to replicate a failed invalidation on our default Turso example, but after many hours of testing, I only made it happen one time (i.e. an invalidation failed one time, as far as I could tell), but I could not find the root cause and could not replicate it again, and I’m not sure what I did that made it fail =/ I don’t have enough evidence to say whether there’s an edge case issue with their service, a bug in our implementation, some network error, or something else entirely… it’s frustrating, I know.

But in my limited tests with the Vercel fork, it seemed to work without issue. Beware that it’s more expensive than Turso (Vercel KV pricing vs Turso pricing), so depending on your budget/billing plan with Vercel, it may or may not be a realistic option. You can also use another KV/DB of your choice, (Netlify Blobs, Cloudflare Workers KV, Redis on GCP/Azure/AWS, your own Redis/Valkey/Memcached/Postgres, etc.) just to see if it makes any difference at all in reliability.

It’s not something we can officially support, but the underlying mechanisms are not very complex. Most of the underlying datastore provider logic is in just one file: database.ts, that you should be able to modify and hook up to any datastore.

In fact, Vercel KV is basically just a hosted Redis (whitelabeled Upstash), so the commands in my fork should mostly work with any Redis/Valkey-compatible service.

I don’t know if this will help, but providing it just in case any of you want to do some testing of your own…


Sorry, wish I had a better answer here. If anyone does manage to come up with a reproducible example, we will for sure investigate it more!