422 Unprocessable Content Validation Error in Form

Describe the issue:

  • We have been using the computed field plugin and working on custom plugins that automate setting the values of a field based on other fields.
  • Iā€™ve noticed that after a value is set in the form, in some cases there appears to be some kind of server-side validation that fails. The browser console shows the error POST https://site-api.datocms.com/items/validate 422 (Unprocessable Content)
  • This causes the form to continue to show a persistent validation message on the field (even though it is valid) until another field is changes. Then it seems to go away.

Iā€™ve been ignoring this issue because it does go away, but we are getting ready to use Dato in production and this could be confusing to our users. One plugin we have is looking up in our custom Author model the author associated with the datocms userā€™s email address.

When I first load the form it seems to not cause the error (maybe because the author field goes from not being set to being set to a value). If I hit the x next to the value, the field becomes empty again so our plugin again retrieves and sets the Author. This is kind of an edge case, but Iā€™ve seen it for other fields as well. In the screenshot below, you can see that the field is populated, but it still says it is required.

Is there something we should be doing to avoid this problem?

More Details:

The error points to this code served by dato

  b.apply(this, T).then(function(U) {
      P.end_time_ms = r.now(),
      P.status_code = U.status,
      P.response_content_type = U.headers.get("Content-Type");
      var z = null;
      c.autoInstrument.networkResponseHeaders && (z = c.fetchHeaders(U.headers, c.autoInstrument.networkResponseHeaders));
      var X = null;
      return c.autoInstrument.networkResponseBody && typeof U.text == "function" && (X = U.clone().text()),
      (z || X) && (P.response = {},
      X && (typeof X.then == "function" ? X.then(function(R) {
          R && c.isJsonContentType(P.response_content_type) ? P.response.body = c.scrubJson(R) : P.response.body = R
      }) : P.response.body = X),
      z && (P.response.headers = z)),
      c.errorOnHttpStatus(P),
      U
  })

The network request that returns the error send this data as the body:

{"data":{"type":"item","attributes":{"call_out_text":null,"content":[],"curation_level":"editorial","custom_seo_settings":null,"expiration_datetime":null,"landing_url":null,"offer_type":null,"sidebar_zone":null,"url":"null","last_verified_datetime":null,"offer_title":null,"rank":null,"related":null,"secondary_call_out":null,"title":"","vendor":null,"author":null,"brief_headline":"null: null","image":null,"monetized_url":null,"rank_override":null,"shipping_text":null,"coupons":[],"hotness":null,"price":-1,"tags":null,"shipping_price":-1,"system_facets":null,"comparison_price":-1,"external_associations":[],"legacy_id":null,"product_categories":null,"product_brand":null,"most_recent_sort_calculated_score":null,"featured_sort_calculated_score":null},"relationships":{"item_type":{"data":{"type":"item_type","id":"2031528"}}}}}

The response from the endpoint (with the 422 error):

{
    "data": [
        {
            "id": "171810",
            "type": "api_error",
            "attributes": {
                "code": "INVALID_FIELD",
                "details": {
                    "field": "author",
                    "field_id": "10489107",
                    "field_label": "Author",
                    "field_type": "link",
                    "code": "VALIDATION_REQUIRED",
                    "options": {}
                }
            }
        },
        {
            "id": "b9a908",
            "type": "api_error",
            "attributes": {
                "code": "INVALID_FIELD",
                "details": {
                    "field": "call_out_text",
                    "field_id": "10413077",
                    "field_label": "Call Out",
                    "field_type": "string",
                    "code": "VALIDATION_REQUIRED",
                    "options": {}
                }
            }
        },
        {
            "id": "e26cfa",
            "type": "api_error",
            "attributes": {
                "code": "INVALID_FIELD",
                "details": {
                    "field": "landing_url",
                    "field_id": "10428557",
                    "field_label": "Landing URL",
                    "field_type": "string",
                    "code": "VALIDATION_REQUIRED",
                    "options": {}
                }
            }
        },
        {
            "id": "35cb0c",
            "type": "api_error",
            "attributes": {
                "code": "INVALID_FIELD",
                "details": {
                    "field": "offer_title",
                    "field_id": "10429782",
                    "field_label": "Offer Title",
                    "field_type": "string",
                    "code": "VALIDATION_REQUIRED",
                    "options": {}
                }
            }
        },
        {
            "id": "b82bf9",
            "type": "api_error",
            "attributes": {
                "code": "INVALID_FIELD",
                "details": {
                    "field": "url",
                    "field_id": "10428571",
                    "field_label": "URL",
                    "field_type": "string",
                    "code": "VALIDATION_FORMAT",
                    "options": {
                        "predefined_pattern": "url"
                    }
                }
            }
        }
    ]
}

(Optional) Can you provide a link to the item, model, or project in question?

(Optional) Do you have any sample code you can provide?

  • I was wanting to get some quick feedback if this issue is known or if there is a known fix before going through the work of trying to create a simple reproducible example. I know that we have seen this issue early on when using the Computed Fields plugin, but have experienced it in our own private plugins as well that set form values.
  • It is worth noting that both the Computed Fields plugin and our private one is setting the field value as suggested in the plugin docs with ctx.setFieldValue(key, value);. So, when we set the author in the example shown, it would be ctx.setFieldValue('author', '<dato-record-id-for-author>')

Hey @Nroth, sorry for the dumb question here, but could you please help me understand the workflow here? Iā€™m not sure I completely understand whatā€™s happening, but it almost sounds like you have several async operations racing each other (is the required Author field is being async fetched by another plugin, while Computed Fields is itself trying to compute another fieldā€¦?)

If the record is trying to save while the Author field is blank (i.e., there is some async operation that hasnā€™t finished fetching it), itā€™s going to fail validation because, well, itā€™s emptyā€¦? Is that not what itā€™s supposed to do?

If the value is getting changed in real-time, we should validate it again after a few seconds (or you can try to save again, or try to validate it via API). Sorry if Iā€™m misunderstanding you hereā€¦?

If itā€™s really a race, would it help to maybe either:

  1. Show a ā€œFetchingā€¦ā€ modal while the async things are in progress, make sure all the relevant fields are valid, and then show the UI again?

or maybe

  1. Override the field presentation with your own, using e.g. Customize record presentation ā€” DatoCMS or even Custom pages ā€” DatoCMS, so you can write your own validators and loading states?

Does that help at all, or am I misunderstanding the situation here?

In this case Iā€™m describing here, this is the only async operation that is occurring. We arenā€™t using the computed fields plugin on the same field that we are setting the author field on. Iā€™m simply saying Iā€™ve seen this issue with the Computed Fields plugin in the past and thought the issue might be related to that plugin

Now that we have this custom plugin setting the author (as described below), Iā€™m seeing the issue there as well.

Setup:

  • We have an Author model that contains a profile picture, bio, email address, etc.

Workflow:

  • We have a datocms editor that opens the form.
  • If the Author field is empty, the plugin queries the author model to find the author with the same email address as the current editor.
  • If we find an author via that query, we ctx.setFieldValue('author', '<dato-record-id-for-author>')
  • Everything is fine at this point
  • If I click the little x next to the linked author, we detect that the field is empty so do the same lookup and set the Author. It seems like dato warns that the author field is required, then the author is set all within a small amount of time and the datocms built-in validation doesnā€™t get re-evaluated.

Thatā€™s it. There is no saving of the record at this point. The 4 fields reference in that error are all set as required by the model. This seems to be built-in datocms validation.

If the value is getting changed in real-time, we should validate it again after a few seconds (or you can try to save again, or try to validate it via API). Sorry if Iā€™m misunderstanding you hereā€¦?

Yes, i think it gets marked as invalid due to being empty. Then the value gets set, but the form never re-validates the contents (even though it should know a change occurred). So, it doesnā€™t validate it again after a few seconds and I didnā€™t see anything in the documentation that weā€™d need to manually retrigger revalidation if we set a value with a plugin.

@roger, I did some more testing on this issue. Here is a more simple way to think about the issue.

Triggering Built-In Validation Logic

  • First, setup a model that has a simple required text field title
  • Click in the text field, then click outside of the text field. The built-in datocms validation logic will be triggered at this point and will say that the field is required
    • I think this is kind of odd behavior. I personally would look for some change in value to trigger validation. What i can tell is that the validation logic is triggered on a user interacting with a field, then that interaction ending, not based on value changes in the form.
    • This happens independent of any plugins or anything

Setting Form Value from Plugin

  • Now, after triggering the validation logic, set the field value via a plugin with ctx.setFieldValue('title', 'foo')
  • You will see now that the field still shows the error that the field is required even though it is populated
  • The validation logic will not be retriggered, unless you click into the required field again then click outside of it or change its value manually by the editor

Issue

  • ctx.setFieldValue('title', 'foo') does not retrigger the validation logic.
  • The built-in validation logic is triggered only when the user manually interacts with a field
  • The result is if you use a plugin to automate setting any required value in the form, the user will see a confusing and inconsistent form state

Suggested Fix

  • Trigger form validation based on value change of the fields with validation settings (with some long trailing edge debouncing)
Other Notes
  • You can also see the same issue by clicking in then out of the required title field in the example
  • Click into the field again then enter some text, but leave the field focused and never click out of the field
  • The required title field will still have the validation message showing that the field is required until you click out of the field
1 Like

I see now! Thank you very much for this thorough example and documentation, thatā€™s much clearer now whatā€™s going on.

Iā€™ll write up a full bug report and let you know once itā€™s resolved (yeah, it should probably check for changes every X seconds debounced, not just onblur). Iā€™ll also see if I can find any workarounds in the meantime.

Really appreciate your patience and troubleshooting here :slight_smile:

Thanks for taking a look. Yeah, i think I originally got sidetracked a bit by the 422 error and our plugin. It sounds like that 422 error is just what Dato returns any time validation occurs. I was thinking at first that it might be that the validation http call failed, which is why the error message continued to show by the field.

Tweaking the form validation so that it does trigger off of even a very debounced change in values and/or triggering revalidation if ctx.setFieldValue is called would definitely improve how responsive it feels.

Yeah, exactly.

We have a preliminary PR for this in review, and Iā€™ll let you know as soon as thereā€™s an update!