Describe the issue:
We are currently developing new schema in a secondary environment that will be used to migrate nearly 2,000 mdx records to a model with structured text. The scripting to do this took days to run the first time, with all the related image uploads and api calls. Iām not sure if I can speed this up by running things in parallel or not, but even updating records after tweaking the migration script takes hours to run.
When I migrate this to our primary environment, Iād love to be able to also migrate the content instead of running these migrations scripts again after migrating the schema. Can you point me to documentation about the best way to do this?
@fitzage
Can you share with us the script at support@datocms.com so we can take a look at it?
Updating 2000 records at the 40 request/second rate limit should take about 50 seconds (maybe a couple of minutes depending on the complexity of some of those updates, that can cause some of the update request to linger a bit longer)
Weād like to see what is the central cause for the delay and if it can be improved in any way.
Thank you!
(I know this possibly wonāt fit your use-case and that you already considered this option, but Iāll add this in case anyone else reads this thread)
The easiest way to avoid re-running a long data script on the primary environment is to do the schema changes and the content migration in a sandbox environment, then promote that sandbox to become the new primary. Promotion swaps environments, so the schema, records, and uploads that already exist in the sandbox become the new primary state.
I did some parallelization work which improved things dramatically, but just the image upload alone is going to take a ton of time if we have to run this again.
Running the migration on a clone and then promoting is definitely the way to go, but it requires having the primary environment in maintenance mode for however long this takes, so we want to speed it up any way we can. Is there at least a way to get the media pointers migrated, since the media is accessed through imgix anyway?
I will send the latest migration script in case there are any obvious mistakes you can see that would affect performance.
@fitzage Seems like the script is pretty good, the only thing id add is āskipCreationIfAlreadyExistsā to avoid duplicate uploads, but other than this there isnāt much we can do on our end to improve this 
If promoting the environment doesnāt fit your use-case and the file indeed needs to be uploaded to the environment, the only way to improve the speed is to make sure you are on a machine with good upload bandwidth, but other than this, the file needs to get from your machine to our servers, there are no ways to āmoveā assets from one environment to another post-fork.
Sorry about that