Site Search for multiple DatoCMS environments

🆘 Support
1

Hi all!

Asking for some guidance, we will be implementing DatoCMS Site Search for one of our projects, they need a simple site search, hence this looks like the perfect solution.

Context:
We are heavily using DatoCMS environments in our project

  • development (sandbox)
  • testing (sandbox)
  • acceptance (sandbox)
  • production (primary)

We will be using a custom webhook as build trigger, since we are in control of the deployment and we are not using an external service.

My Question:
Build triggers are under Project settings (maining across environments). So the recommended way is to add a build trigger per environment?

This does lead to the following undesirable behaviour:

  • Content editors can trigger builds for development, testing, acceptance environments, while we only want them to trigger builds for production(primary) environment.

Interested on your view, are we implementing this as desired? Could you add a section in the documentation on how to work with Site Seach if you are also using DatoCMS environments, this could prevent such a question in the future :slight_smile:

As I’m understanding it now, we kind of have to see the build_trigger_id for the CMA API as the x-environment from the CDA API. (atleast for our use case.)

Thanks!

1 Like
2

Hello @thomas.vanleynseele

A quick confirmation first: build triggers live at the project level and are not tied to a specific environment, they are project wide. They are simply the connection between your DatoCMS project and a particular frontend URL, and when that frontend is rebuilt we crawl that URL to (re)index the site. That’s more elaborated here: https://www.datocms.com/docs/site-search/configuration and also in this community reply clarifying that build triggers aren’t environment specific: https://community.datocms.com/t/how-can-you-tie-builds-to-environments/2956. The environment your frontend uses is chosen by your build code or config, for example by passing the X-Environment header or initializing the JS client with the right environment as shown here: https://www.datocms.com/docs/content-management-api/setting-the-environment.

Given your setup with development, testing, acceptance, and production, the recommended pattern is to keep one build trigger per frontend you actually want to deploy or crawl, and enable Site Search only on those triggers that correspond to a real site URL you want indexed. Each trigger should have its own Website frontend URL. That way each environment that has a public/staging URL can have its own index. Details on crawling behavior are here: https://www.datocms.com/docs/site-search/how-the-crawling-works.

About your mapping: “build_trigger_id for the CMA API equals x-environment from the CDA API.”

Not really: The build_trigger_id in a Site Search query selects which index to search, and that index is keyed to the build trigger that crawled a specific frontend URL.

The environment is something your frontend used when it was built.

If your staging site is built from the “acceptance” environment and has its own domain, enabling Site Search on the staging trigger means its index reflects acceptance. Then in your search UI you point queries at that trigger’s index. You can see this in our “Perform searches via API” page where build_trigger_id is required: https://www.datocms.com/docs/site-search/base-integration.

Since you’re using a custom webhook for deployments, you can include the target environment name in the JSON payload you send to your CI. Your CI can then set the client’s environment option or X-Environment header during the build. The “Custom webhook” integration is here: https://www.datocms.com/marketplace/hosting/custom-webhook and the environment header details again here: https://www.datocms.com/docs/content-management-api/setting-the-environment. If you ever need to refresh the search index without a full rebuild, you can call the reindex endpoint for that build trigger: https://www.datocms.com/docs/content-management-api/resources/build-trigger/reindex.

Two tiny snippets you can lift as-is.

Trigger a deploy for the production trigger from a script using our current client:

import { buildClient } from '@datocms/cma-client-node';

const client = buildClient({ apiToken: process.env.DATOCMS_API_TOKEN });

// allow only the prod trigger id here
await client.buildTriggers.trigger('<PROD_BUILD_TRIGGER_ID>');

Run a Site Search query against the production index from the browser:

import { buildClient } from '@datocms/cma-client-browser';

const client = buildClient({ apiToken: '<SITE_SEARCH_TOKEN>' });

const { data: results } = await client.searchResults.rawList({
  filter: {
    query: 'hello',
    fuzzy: true,
    build_trigger_id: '<PROD_BUILD_TRIGGER_ID>',
  },
  page: { limit: 10, offset: 0 },
});

So in short, yes, add a build trigger per frontend you care about, enable Site Search on the ones you want indexed. In your search calls always pass the build_trigger_id that corresponds to the index you want. For reference, our deploy overview also notes that with multiple triggers you can manage permissions and logs per environment-like target: https://www.datocms.com/docs/general-concepts/deployment.

Thanks as well for the docs suggestion. We’ll consider adding it in the next docs revamps :slight_smile:

1 Like
3

Hi Marcelo,

Many thanks for the elaborate reply, this clarifies a lot and confirms our approach.

One of the missing pieces was permissions(roles), thanks for pointing me in the right direction for that one.

Have a nice day!

1 Like