Best Practice when regularly keeping data in Sync?

🆘 Support
1

Hi there!

Great product guys!
I used to use a lot of Cont…ful but after switching…never turning back!
I am building a headless jamstack ecommerce for my families business.
All done, except the last bit = regularly syncing.

Unfortunately ERP doesn’t have webhooks, so I am forced to do interval syncing for products, stock, prices.

Basically I need to confirm the *best practice for regular ‘updating’ with DatoCms?
Get - List all DatoCMS Records ( filtered type )
Map/transform the new data to compare with DatoCMS data.
Create 2 lists of new (mapped) data for Create and for Update methods.
Run Create & Update with segregated new data.

Is that how it should be done?
Or is there some magical import where it will create new, update existing, with option to delete or keep remaining existing data? :smiley:

Thanks so much!

P.S. because I am writing everything in zappier, with their custom data and webhooks apps - I cant find the correct param for allpages=true in the normal (nonclient) request examples (for Get All Records)

2

hey @veljko welcome to Community and thank you for your nice words! :slight_smile:

Yes, what you are saying it’s correct, unfortunately we don’t have a better way for doing that at the moment.

And regarding the allPages option, it’s just for our JS/Ruby clients. If you want to do that with HTTP calls you need to use pagination comparing with the meta.total_count number that should provide the total number of records for a model.

Let me know if you need more details here!

1 Like
3

Got it!
Thank you very much for the warm welcome :smiley:
Ill paginate it through then ^^

1 Like