How to serialize calls to an async function

An example is available on GistRun

Want to learn AWS serverless development? Click here

When to serialize async calls

Let's say you have a function that does some calculation but only if it hasn't been done recently, effectively caching the result for a short amount of time. In this case, subsequent calls to the function affect each other, as when there is a refresh all of them need to wait for it. An implementation that does not take concurrency into account can potentially run the refresh several times in parallel when the cache expires.

This implementation is broken:

// returns a function that caches its result for 2 seconds
// broken, don't use it
const badCachingFunction = (() => {
	const cacheTime = 2000;
	let lastRefreshed = undefined;
	let lastResult = undefined;
	return async () => {
		const currentTime = new Date().getTime();
		// check if cache is fresh enough
		if (lastResult === undefined ||
			lastRefreshed + cacheTime < currentTime) {
			// refresh the value
			lastResult = await refresh();
			lastRefreshed = currentTime;
		}
		return lastResult;
	}
})();

When two concurrent calls come when the function needs to refresh the value, the refresh function is called twice:

const wait = (time) => new Promise((res) => setTimeout(res, time));

const refresh = async () => {
	console.log("refreshing")
	await wait(1000);
};

const badCachingFunction = ...;

badCachingFunction();
badCachingFunction();
// refreshing
// refreshing

The correct way to handle this is to make the second function wait for the refresh process without starting a separate one.

One way to do this is to serialize the calls to the caching function. This makes every function to wait for all the previous ones to finish, so multiple calls only trigger a single refresh process.

This can be done in a variety of ways, but the easiest one is to make the functions run one after the other. In this case, when one needs to refresh the value, the other ones will wait for it to finish and won't start their own jobs.

Another use-case I needed a solution like this is when backend calls needed a token and that token expired after some time. When a call hit an Unauthorized error, it refreshed the token and used the new one to retry. Other backend calls needed to wait for the new token before they could be run. In this case, it wasn't just performance-related as a new token invalidated all the old ones.

Book

Building GraphQL APIs with AWS AppSync

How to design, implement, and deploy GraphQL-based APIs on the AWS cloud

Why await is not a solution

The trivial solution is to use await that achieves serialization easily. After all, that's what that keyword is for.

await badCachingFunction();
badCachingFunction();
// refreshing

But that requires collaboration between the calls, and that is not always possible. For example, when the function is called in response to multiple types of events, await is not possible to coordinate between them:

document.querySelector("#btn").addEventListener("click", () => {
	fn();
})

window.addEventListener("message",  => {
	fn();
});

In this case, the function call must do the coordination.

The async serializer pattern

The solution is to keep a queue of Promises that chains them one after the other. It is just a few lines of code and it is general purpose, allowing any function be serialized:

const serialize = (fn) => {
	let queue = Promise.resolve();
	return (...args) => {
		const res = queue.then(() => fn(...args));
		queue = res.catch(() => {});
		return res;
	};
};

The Promise.resolve() is the start of the queue. Every other call is appended to this Promise.

The queue.then(() => fn(...args)) adds the function call to the queue and it saves its result in res. It will be resolved when the current and all the previous calls are resolved.

The queue = res.catch(() => {}) part makes sure that the queue won't get stuck in rejection when one part of it is rejected.

Wrapping the caching function with this serializer makes sure that a single refresh is run even for multiple calls:

const fn = serialize((() => {
	const cacheTime = 2000;
	let lastRefreshed = undefined;
	let lastResult = undefined;
	return async () => {
		const currentTime = new Date().getTime();
		// check if cache is fresh enough
		if (lastResult === undefined ||
			lastRefreshed + cacheTime < currentTime) {
			// refresh the value
			lastResult = await refresh();
			lastRefreshed = currentTime;
		}
		return lastResult;
	}
})());

fn();
fn();
// refreshing