Apologies for the delayed response here @tim1 and @nrothā¦ Iām trying to see if we can rearrange the forum a bit to prevent posts like this from falling through the cracks 
Sorry about that.
I dug more deeply into this, and TLDR, the CREATING_UPLOAD_OBJECT step creates an async job on our server that you shouldnāt have to wait for, but our client makes you wait Iāll flag it for the devs and see if itās an easy fix for themā¦
Good news, though:
In the meantime, here is a workaround that should let you exit early. It reimplements much of the same logic as our real client does (using some of its undocumented exports), but then exits early as soon as CREATING_UPLOAD_OBJECT begins, without waiting for it to finish (which is what was taking so long, but thatās just a serverside op that your client shouldnāt need to wait for).
Thatās all your script or serverless func should need to do. Then, a few seconds later, the image will magically appear in your DatoCMS media areaā¦ but your function wonāt have to keep waiting for that to happen (unlike our official client, which keeps polling until it succeeds, which as you saw can take like 30+ seconds for big enough images).
Hereās a sample implementation that should exit as soon as the ācreateā step begins and the job starts, without waiting for it to finish. The image itself wonāt show up in the media area until about 30-40 secs later, but your client wonāt have to wait:
import {buildClient, downloadFile, LogLevel, uploadLocalFileAndReturnPath} from "@datocms/cma-client-node";
import {makeCancelablePromise} from '@datocms/rest-client-utils';
let jobUrl: string | undefined;
const logParser = (message: string) => {
// Exit early if the job URL is already set
if (jobUrl) return;
// Otherwise, parse the log messages and get back the job ID
const jobUrlMatch = message.match(/GET (https:\/\/site-api\.datocms\.com\/job-results\/.+)$/);
if (jobUrlMatch && jobUrlMatch[1]) {
jobUrl = jobUrlMatch[1];
}
}
const client = buildClient({
apiToken: "YOUR_API_TOKEN",
logLevel: LogLevel.BODY, // Important, keep this! We have to parse it manually to get your job ID :(
logFn: logParser, // This is the function that parses the log to get the job ID
});
async function run() {
const smallFile = "https://upload.wikimedia.org/wikipedia/commons/c/ca/Crater_Lake_from_Watchman_Lookout.jpg";
const bigFile = 'https://upload.wikimedia.org/wikipedia/commons/7/7d/%22_The_Calutron_Girls%22_Y-12_Oak_Ridge_1944_Large_Format_%2832093954911%29_%282%29.jpg'
// First download the file locally
console.log('\nStarting download...')
const downloadPromise = downloadFile(bigFile, {onProgress: handleProgress})
const {filePath} = await downloadPromise;
console.log(`File downloaded to ${filePath}`)
// Then upload it to S3 and get back a path
console.log('\nStarting upload...')
const remotePath = await uploadLocalFileAndReturnPath(client, filePath, {onProgress: handleProgress})
console.log(`File uploaded to ${remotePath}`)
// Tell DatoCMS to link the S3 file to your media area
// Note that we do NOT await it. We will forcibly cancel it later.
console.log(`\nStarting async job to create file in Dato from your S3 upload...`);
const asyncCreatePromise = makeCancelablePromise(
client.uploads.rawCreate(
{
data: {
type: "upload",
attributes: {
path: remotePath
}
}
}
));
console.log('Created the promise, but still waiting for a job URL...');
(function waitForJobURL() {
if (jobUrl) {
console.log(`Found Job URL. Canceling the promise now...`);
asyncCreatePromise.cancel();
console.log(`It's safe to exit now. You can ignore the canceled promise error and manually check job status at ${jobUrl}`);
} else {
setTimeout(waitForJobURL, 50); // Check again in 100ms
}
})();
}
function handleProgress(info) {
console.log("Phase:", info.type);
console.log("Details:", info.payload);
}
run();
Itās really a pretty ugly hack, but it gets the job done. Iāll try to get the devs to update the official client too.
