Hi I m still learning Next and I have used your starter template the Marketing site. I protected many of the pages on the site with Clerk and since than the Revalidate Cache is no longer working and is giving me an 307 or 308 error.
Is there a workaround to revalidate the cache for all pages including the protected pages? A link to a tutorial would be much appreciated. Thank you
This is the response headers:
Date: Wed, 03 Jan 2024 13:48:21 GMT
Which cache are you referring to here? I’m afraid this might be a better question for Clerk and/or Vercel support (if you’re hosting with them). We don’t do that sort of caching on the Dato side (your changes should be immediately visible in our APIs).
Some links that have been helpful for other customers in the past:
Beyond that, I’m afraid that this might be more a question for one of their support teams, since it depends on how exactly you’ve set up your frontend and Clerk integration. On our side, our APIs just give you JSON results (or media assets) and those invalidations should happen automatically and almost instantly.
I’m obviously referring to Vercel cache and I m afraid I am very disappointed with the reply. I don’t know as much yet. I m using this starter Next 13 Company Landing Page Builder — DatoCMS and it has a webhook which stopped working since password-protecting some pages. Nothing to do with Vercel or Clerk. I was merely asking for pointers on how to set the webhook up when some pages are behind the password. Anyway, thanks for taking the time to answer. I now know DATO CMS is not newbie friendly
Okay. Sorry about that. Let’s take another look together and see what we can figure out, okay?
Just for some background, I promise we’re not just trying to brush you off. Next on Vercel has several caching layers (which the links I provided discuss in more detail), and they interact in pretty complex ways. Specifically, Next.js 13 or 14 added a (pretty confusing, IMO) data cacher that monkey-patches the JS fetch() in some non-obvious and unintuitive ways, breaking many third-party tools & buildchains (for example, see this Github issue). Other customers of ours have also been confused by this behavior, but there isn’t much we can do to address it because that’s just how Vercel officially wants Next.js to work.
Our revalidateCache endpoint in our sample project is just a thin wrapper around next/revalidateTag, which tells that monkey-patched fetch() to revalidate. That’s all it does, and it does so using built-in Next.js functionality, not anything special we’re caching on our end. So if ceases to work after adding Clerk, it would really depend on your setup – what’s being cached, which part of the chain isn’t invalidating correctly, etc. Like is it affecting something that Clerk is trying to fetch? Is it causing the API route to be unreachable, maybe…? Hard to say, but happy to look into it with you.
Could you please provide some more details about how your code is set up? The best possible thing would be a minimal reproducible example (with your Clerk integration code added), but if that’s too much work, some snippets would be great too? You can also share your repo with us (view-only is fine) if that’s easier (r.tuan@datocms.com on Github)?
I still don’t believe we at DatoCMS are really the best ones to help you troubleshoot this, but we can still try if you’d like? Honestly, we are not as familiar with Vercel on their data caching mechanics, or the best way to set up a Clerk integration with Next. But we can look with you if you’d like.
Hi Roger thank you for offering to help. It is the Clerk that is causing the issues and causes a 307 error on revalidation.
Sorry too for misreading your first response.
Thank you for explaining the differences in cache and pointing me in the right direction.
No problem! Hope you were able to figure it out? And sorry I wasn’t clearer in my first post!
I think, in general, Next.js caching is a pretty complex topic that many folks find confusing (including several of us here at Dato!). We talked about this internally a bit, thanks to your question and other similar ones. We’re going to try and see if we can improve our documentation and examples for these cases… even if it’s not something directly in our control, maybe at least we can at least provide some guidance, since it’s an increasingly common question for a popular framework.
If there’s anything you’d like to add about how you resolved it with Clerk, it might help other Next.js users too. (And maybe it’s something we can even add to the docs, if it makes sense.)
Thanks again for bringing it up! Please don’t hesitate to reach back out if you need help with anything else
Hi Roger. I hope you are well. The solution was a big oversight on my and the developer’s behalf.
Basically, all we had to do was add the Cache Revalidation URL from the Webhook to the public Clerk URLs.
Now is working perfectly
