Record size limit for localized content

Hi again, sorry for spamming the forum :smiling_face_with_sunglasses: I’m working on a site that’s localized in 5 languages and I need to set up a lengthy ā€œterms and conditionsā€ page. Unfortunately, the size of the record exceeds the allowed 300kb. It appears that the size limit is the sum of the all the content in a record. Is that correct? I would except that 1 localized piece of content is 1 record, instead. Do you have any suggestions for how to proceed?

I’ve stripped down my HTML as much as possible using HtmlWasher, but that wasn’t sufficient.

I also noticed that the HTML editor converts diacritics and other special characters as HTML entity codes, which increases the size of records by a decent amount, in some languages.

For example: Ć  becomes à, an 800% increase in size (idk if that’s how it works, I’m just being dramatic). Ć  would be fine to store as is, right?

Edit: I tried converting my HTML to Markdown but even that wasn’t enough to fit the limit.

Thank you!

Hello @maxime.preaux :slight_smile:

No problem, post as much as you want, that’s what the forum’s for!

However, in this case, this is an technical limit that we have to look at on a case-by-case basis. Could you please send us an email about this at support@datocms.com with your project details and an example of what you’re trying to put in there? In some cases we may be able to suggest a better way to model the same data without running into the record size limit; or in other cases when there is no realistic workaround, we can manually change the record size limit for that specific project (only if there are no other possibilities).

But please send us an email with some details and we’ll take a look at your particular case ASAP!

@maxime.preaux,

I dug around and was able to find your project, and I think found the right records (your legal docs, like the terms of service and privacy policy?). The way you’re using them seems fine to me; they’re just long documents and each locale is about 70 KB, so yeah, after a few locales you’ll naturally reach that limit.

I can increase the limit for your project, but unfortunately not while you’re on a free plan :frowning: Were you considering upgrading to a paid plan at some point? If so, I can change it for you once you do.

Otherwise, if you want to stay on the free plan, you can make each language its own record — not as convenient, I know, but it’s a workaround.

1 Like

Increasing the record limit would be great, I’ll email you once I upgrade to a paid plan @roger :slight_smile:

Any thoughts about the HTML entity encoding? I wonder if removing that encoding could lead to smaller record sizes for everyone.

Hey @maxime.preaux

Can you email us at support@datocms.com with the exact record URL you are hitting those size limits?

This way we can take a look at it and give you some precise recommendations

Thank you!

@roger @m.finamor email sent! Thank you both. For future visitors, it seems like the 2 recommendations for working with the record size limits are:

  1. split up the records. If you have localized content, create 1 record per language, for example, or
  2. be nice to the Dato team and they might increase the record size limit for your project

Just to clarify, being nice never hurts, but in this case it’s more of a technical constraint on our part :slight_smile: So we check these requests on a case-by-case basis mostly to make sure it’s not an accident that’s causing the increased record size. A common culprit is embedded metadata that Figma and other apps add to their HTML on copy, which doesn’t do any good in DatoCMS but drastically increases record sizes.

HTML entity encoding, by contrast, is a tiny increase (just a few chars extra) and not usually worth worrying about. Encoding them is usually a best practice to ensure successful transmission across different networks, languages, libraries, etc… it’s less of an issue these days with better Unicode support, but still not perfect everywhere.

Anyhow, in your specific case, you were doing everything right, and ran into the technical limit ā€œorganicallyā€. That’s the kind of situation where we go, ā€œYep, this is our system’s fault, let’s raise it for this customer so they can keep working.ā€

1 Like