Asynchronous array functions in Javascript

An example is available on GistRun

Want to learn AWS serverless development? Click here

Asynchronicity in Javascript

The new async/await is an immensely useful language feature that hides the complexity of asynchronous code and makes it look like it's working in a synchronous way. The code example below does not look complicated at all, just a few awaits here and there. But under the hood it's sending commands to a separate process and controlling its lifecycle in an event-driven way:

const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto("https://www.google.com");
const divsCounts = await page.$$eval("div", divs => divs.length);
console.log(divsCounts);
await browser.close();

Since Javascript is single-threaded, asynchronicity is a lot more important than in other languages. In Java, for example, you can create a new Thread and then it does not matter if some commands block it. But in Javascript, blocking is not an option, that's why there is no sleep function, just a callback-based setTimeout.

This becomes overly apparent when you do too much computation and it freezes the page which is when Chrome shows the "Page unresponsive" popup. Apart from web development, when I was working with the ESP8266 WiFi-enabled hobby IoT chip, my program needed to stop executing every few milliseconds so that the WiFi code could do its thing, otherwise it would lose the connection.

That's why callbacks and Promises, an abstraction over callbacks, is so prevalent in the language. And async/await is another language feature that makes it a lot easier to write asynchronous code.

Book

Building GraphQL APIs with AWS AppSync

How to design, implement, and deploy GraphQL-based APIs on the AWS cloud

Async collection processing

But while async/await is great to make async commands look like synchronous ones, collection processing is not that simple. It's not just adding an async before the function passed to Array.reduce and it will magically work correctly. But without async functions, you can not use await and provide a result later which is required for things like reading a database, making network connections, reading files, and a whole bunch of other things.

Let's see some examples!

When all the functions you need to use are synchronous, it's easy. For example, a string can be doubled without any asynchronicity involved:

const double = (str) => {
	return str + str;
};

double("abc");
// abcabc

If the function is async, it's also easy for a single item using await. For example, to calculate the SHA-256 hash, Javascript provides a digest() async function:

// https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto/digest
const digestMessage = async (message) => {
  const msgUint8 = new TextEncoder().encode(message);
  const hashBuffer = await crypto.subtle.digest("SHA-256", msgUint8);
  const hashArray = Array.from(new Uint8Array(hashBuffer));
  const hashHex = hashArray.map(b => b.toString(16).padStart(2, "0")).join('');
  return hashHex;
};

await digestMessage("msg");
// e46b320165eec91e6344fa10340d5b3208304d6cad29d0d5aed18466d1d9d80e

This looks almost identical to the previous call, the only difference is an await.

But for collections, handling asynchronicity becomes different.

To calculate the double for each element in a collection of strings is simple with a map:

const strings = ["msg1", "msg2", "msg3"];

strings.map(double);
// ["msg1msg1", "msg2msg2", "msg3msg3"]

But to calculate the hash of each string in a collection, it does not work:

const strings = ["msg1", "msg2", "msg3"];

await strings.map(digestMessage);
// [object Promise],[object Promise],[object Promise]

This is an everyday problem as async functions make a bigger category than synchronous ones. A sync function can work in an async environment, but an async one can not be converted to work in a synchronous way.

For a more realistic scenario, querying a database is inherently async as it needs to make network connections. To get the score for a single userId from a hypothetical database is simple:

const getUserObject = async (id) => {
	// get user by id
};

const userId = 15;
const userObject = await getUserObject(userId);
const userScore = userObject.score;

But how to query the database for a collection of userIds?

const userIds = [15, 16, 21];
// const userObjects = ???

And it's not just about using map to transform one value to another. Is the user's score above 3?

const userObject = await getUserObject(userId);
const above3 = userObject.score > 3;

But is any of the users' score above 3?

// userIds.some(???)

Or what is the average score?

// userIds.reduce(???)

To make things more interesting, adding an async to a map at least gives some indication what is changed:

// synchronous
[1, 2, 3].map((i) => {
	return i + 1;
});
// [2, 3, 4]

// asynchronous
[1, 2, 3].map(async (i) => {
	return i + 1;
});
// [object Promise],[object Promise],[object Promise]

But a filter just does something entirely wrong:

// synchronous
[1, 2, 3, 4, 5].filter((i) => {
	return i % 2 === 0;
});
// [2, 4]

// asynchronous
[1, 2, 3, 4, 5].filter(async (i) => {
	return i % 2 === 0;
});
// [1, 2, 3, 4, 5]

Because of this, async collection processing requires some effort, and it's different depending on what kind of function you want to use. An async map works markedly differently than an async filter or and async reduce.

In this series, you'll learn how each of them works and how to efficiently use them.

for iteration

But first, let's talk about for loops!

I don't like for loops as they tend to promote bad coding practices, like nested loops that do a lot of things at once or continue/break statements scattered around that quickly descend into an unmaintainable mess.

Also, a more functional approach with functions like map/filter/reduce promote a style where one function does only one thing and everything inside it is scoped and stateless. And the functional approach is not only possible 99% of the time but it comes with no perceivable performance drop and it also yields simpler code (well, at least when you know the functions involved).

But async/await is in the remaining 1%.

For loops have a distinctive feature, as they don't rely on calling a function. In effect, you can use await inside the loop and it will just work.

// synchronous
{
	const res = [];
	for (let i of [1, 2, 3]){
		res.push(i + 1);
	}
	// res: [2, 3, 4]
}

// asynchronous
{
	const res = [];
	for (let i of [1, 2, 3]){
		await sleep(10);
		res.push(i + 1);
	}
	// res: [2, 3, 4]
}

As for loops are a generic tool, they can be used for all kinds of requirements when working with collections.

But their downside is still present, and while it's not trivial to adapt the functional approach to async/await, once you start seeing the general pattern it's not that hard either.

Coming next

In the following articles in this series, we'll look into how the commonly used array functions can be used in an async way. Stay tuned!