Javascript fetch, retry upon failure
So recently, I bumped into a situation where the network works/fails randomly. As it affects the consistency of my test results, I decided to implement a fetch_retry function which retries fetch upon failure up to n times.
Introduction
Fetch in Javascript is great. I hope you will agree that it provides a simple yet robust enough interface to do our AJAX requests.
However, network doesn't always work like we want it to, and it might fail randomly. To catch this issue, let's implement a function fetch_retry(url, options, n) which does fetch(url, options) but retries up to n times upon failure. And hence increasing the chance of success.
Let's think
Retrying things sounds like a loop. Why don't we write a for/while loop to do that? Something like the following, perhaps?
function fetch_retry(url, options, n) {
for(let i = 0; i < n; i++){
fetch(url, options);
if(succeed) return result;
}
}
NO! Fetch is an asynchronous function, meaning that the program would not wait for result before continuing! n fetches will be called at the same time (kind of), regardless of whether the previous calls succeed!
This is not what we want. This is not retry upon failure, this is fetching n times simultaneously! (That being said, if written correctly, it could also increase the chance of success. Perhaps with something like Promsie.any? Although I am not a big fan of bluebird. I think native Promise is good enough.)
If you don't know about asynchronous functions and Promise in Javascript, watch this amazing video here, made by Jessica Kerr, before reading on!
Briefly about fetch
So fetch returns a Promise. We usually call it like this.
fetch(url, { method: 'GET' }).then(res => console.log('done'));
console.log('fetching...');
If you understand Promise correctly, you should expect the result to be:
fetching...
done
And if the network fails for some reason, the Promise rejects and we could catch the error as follows:
fetch(url, { method: 'GET' }).catch(err => /* ... */);
So how to implement?
What does fetch_retry do?
We start off with thinking what do we want the function fetch_retry do. We know it has to call fetch somehow, so let's write that down.
function fetch_retry(url, options, n) {
fetch(url, options)
.then(function(result) {
/* on success */
}).catch(function(error) {
/* on failure */
})
}
Now obviously fetch_retry has to be an asynchronous function, since we can't really define a synchronous function out of an asynchronous one. (or could we? Enlighten me.)
Definition: So this means fetch_retry should return a Promise that resolves if any attempt out of n attempts succeed, and rejects if all n attempts failed.
So let's return a Promise now.
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) { // <--- we know it is asynchronous, so just return a promise first!
fetch(url, options)
.then(function(result) {
/* on success */
}).catch(function(error) {
/* on failure */
})
});
}
What if fetch succeeds?
So if the fetch succeed, we obviously can just resolve the promise we are returning, by calling the resolve function. So the code become:
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options)
.then(function(result) {
/* on success */
resolve(result); // <--- yeah! we are done!
}).catch(function(error) {
/* on failure */
})
});
}
What if fetch fails?
What should we do on failure? Doing for/while loop here wouldn't really help, due to the asynchronous property we discussed previously. But there is one thing that we could do what for/while loop does. Does it ring a bell? Yes! Recursion!
My two rules of thumb when doing recursion:
- Do not think recursively. Don't try to follow your code recursively.
- Leap of faith, assume the recursive function you are defining works.
These two points are fundamentally the same! If you have the leap of faith, you wouldn't be thinking recursively into the code.
Ok, so let's try to take the leap of faith and assume fetch_retry will just work, magically.
If it works, then in on failure, what will happen if we call fetch_retry(url, options, n - 1)?
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options)
.then(function(result) {
/* on success */
resolve(result);
})
.catch(function(error) {
/* on failure */
fetch_retry(url, options, n - 1) // <--- leap of faith, this will just work magically! Don't worry!
.then(/* one of the remaining (n - 1) fetch succeed */)
.catch(/* remaining (n - 1) fetch failed */);
})
});
}
fetch_retry(url, options, n - 1) will just work magically by the leap of faith and would return a Promise which, by the definition we discussed previously, resolves if any attempt (out of n - 1 attempts) succeed, and rejects if all n - 1 attempts failed.
So now, what do we do after the recursive call? Notice that since fetch_retry(url, options, n - 1) would work magically, this means we have done all n fetching at this point. In the on failure case, simply resolves if fetch_retry(url, options, n - 1) resolves, and rejects if it rejects.
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options)
.then(function(result) {
/* on success */
resolve(result);
})
.catch(function(error) {
fetch_retry(url, options, n - 1)
.then(resolve) // <--- simply resolve
.catch(reject); // <--- simply reject
})
});
}
Great! We are almost there! We know we need a base case for this recursive call. When thinking about base case, we look at the function arguments, and decide at what situation we could tell the result immediately.
The answer is when n === 1 and the fetch fails. In this case, we could simply reject with the error from fetch, without calling fetch_retry recursively.
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options)
.then(function(result) {
/* on success */
resolve(result);
})
.catch(function(error) {
if (n === 1) return reject(error); // <--- base case!
fetch_retry(url, options, n - 1)
.then(resolve)
.catch(reject);
})
});
}
Clean things up
Redundant function
In our "on success" function, we are simply calling resolve(result). So this function instance is redundant, we could simply use resolve as the "on success" function. So the code would become:
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options).then(resolve) // <--- Much cleaner!
.catch(function(error) {
if (n === 1) return reject(error);
fetch_retry(url, options, n - 1)
.then(resolve)
.catch(reject);
})
});
}
Redundant promise
Now another stupid thing we are doing here is this line:
fetch_retry(url, options, n - 1).then(resolve).catch(reject)
Do you see what is the problem?
Let me put this in context, we are essentially doing this:
new Promise(function(resolve, reject) {
fetch_retry(url, options, n - 1).then(resolve).catch(reject)
});
So this new promise is redundant in this case, because it is resolving if fetch_retry resolves, and rejecting if fetch_retry rejects. So basically it behaves exactly the same as how fetch_retry behaves!
So the above code is basically semantically the same as just fetch_retry by itself.
fetch_retry(url, options, n - 1)
// sementically the same thing as the following
new Promise(function(resolve, reject) {
fetch_retry(url, options, n - 1).then(resolve).catch(reject)
});
It require one more knowledge in order to clean up the code. We could chain promise.thens in the following way. Because promise.then returns a promise as well!
Promise.resolve(3).then(function(i) {
return i * 2;
}).then(function(i) {
return i === 6; // this will be true
});
As you can see, we could pass the processed value onward to the next then and so on. If the value is a Promise, then the next then would receive whatever the returned Promise resolves. See below:
Promise.resolve(3).then(function(i) {
return i * 2;
}).then(function(i) {
return Promise.resolve(i * 2); // also work!
}).then(function(i) {
return i === 12; // this is true! i is not a Promise!
};
The same idea could be applied to catch as well! Thanks to Corentin for the shout out! So this means that we could even resolve a promise when it rejects. Here is an example:
Promise.resolve(3).then(function(i) {
throw "something's not right";
}).catch(function(i) {
return i
}).then(function(i) {
return i === "something's not right";
};
So how could we clean up with these knowledge? The code we have seem to be more complicated.
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options).then(resolve)
.catch(function(error) {
if (n === 1) return reject(error);
fetch_retry(url, options, n - 1)
.then(resolve) // <--- we try to remove this
.catch(reject); // <--- and this
})
});
}
Well, we could resolve the returning promise with the promise returned by fetch_retry! Instead of fetch_retry(...).then(resolve).catch(reject). We could do resolve(fetch_retry(...))! So the code becomes:
function fetch_retry(url, options, n) {
return new Promise(function(resolve, reject) {
fetch(url, options).then(resolve)
.catch(function(error) {
if (n === 1) return reject(error);
resolve(fetch_retry(url, options, n - 1)); // <--- clean, isn't it?
})
});
}
Now we could go even further by removing the explicit creation of the Promise by resolving the promise in catch.
function fetch_retry(url, options, n) {
return fetch(url, options).catch(function(error) {
if (n === 1) throw error;
return fetch_retry(url, options, n - 1);
});
}
Quoting from MDN with some words tweaked for more layman terms:
The Promise returned by catch(handler) is rejected if handler throws an error or returns a Promise which is itself rejected; otherwise, it is resolved.
ES6
I can predict some JS gurus would be hating me for not using arrow functions. I didn't use arrow functions for people who are not comfortable with it. Here is the ES6 version written with arrow functions, I wouldn't explain much.
const fetch_retry = (url, options, n) => fetch(url, options).catch(function(error) {
if (n === 1) throw error;
return fetch_retry(url, options, n - 1);
});
Happy?
ES7
Yeah yeah, Promise is becoming lagacy soon once ES7 async/await hits. So here is an async/await version:
const fetch_retry = async (url, options, n) => {
try {
return await fetch(url, options)
} catch(err) {
if (n === 1) throw err;
return await fetch_retry(url, options, n - 1);
}
};
Which looks a lot neater right?
In fact, we don't have to use recursion with ES7, we could use simple for loop to define this.
const fetch_retry = async (url, options, n) => {
let error;
for (let i = 0; i < n; i++) {
try {
return await fetch(url, options);
} catch (err) {
error = err;
}
}
throw error;
};
// or (tell me which one u like better, I can't decide.)
const fetch_retry = async (url, options, n) => {
for (let i = 0; i < n; i++) {
try {
return await fetch(url, options);
} catch (err) {
const isLastAttempt = i + 1 === n;
if (isLastAttempt) throw err;
}
}
};
Conclusion
To conclude, we have looked at 4 different versions of the same function. Three of them are recursive just written in different style and taste. And the last one with for loop. Let's recap:
Primitive version
function fetch_retry(url, options, n) {
return fetch(url, options).catch(function(error) {
if (n === 1) throw error;
return fetch_retry(url, options, n - 1);
});
}
ES6
const fetch_retry = (url, options, n) => fetch(url, options).catch(function(error) {
if (n === 1) throw error;
return fetch_retry(url, options, n - 1);
});
ES7 async/await recursive
This is my favorite.
const fetch_retry = async (url, options, n) => {
try {
return await fetch(url, options)
} catch(err) {
if (n === 1) throw err;
return await fetch_retry(url, options, n - 1);
}
};
ES7 async/await for-loop
const fetch_retry = async (url, options, n) => {
let error;
for (let i = 0; i < n; i++) {
try {
return await fetch(url, options);
} catch (err) {
error = err;
}
}
throw error;
};
// or (tell me which one u like better, I can't decide.)
const fetch_retry = async (url, options, n) => {
for (let i = 0; i < n; i++) {
try {
return await fetch(url, options);
} catch (err) {
const isLastAttempt = i + 1 === n;
if (isLastAttempt) throw err;
}
}
};
Tell me your feedback in the comments! :D