I am using the Search API for a project, and it is working well for the most part. However, the excerpts that it returns are not especially useful, as it will combine headings and body sections into a single string which becomes pretty indecipherable.
Could we add some more metadata to the raw data results? If I had access to things like the description and twitter image url, I would find the API a lot more useful.
That search endpoint still doesn’t have that metadata, as it wasn’t a very popular request.
However, you should be able to fetch that separately either from our APIs or from the rendered HTML directly (in its meta tags).
I do think it’d be useful to have that included in the crawler results, but that would probably only happen if this feature requests gets more upvotes.
If it’s important, you might want to consider using a more powerful crawler like https://www.algolia.com/ or https://typesense.org/. Our site search is just a basic implementation… it works, but isn’t as featureful as the big names.