Generators in Javascript

Generators

In the previous post, we've covered iterables. We've seen that they have a function that returns iterators, which are then used to access the elements of a collection. But they require a lot of boilerplate code to make them work.

This post introduces generator functions, which is a standard and well-supported way to create iterators and do that with minimal additional code.

Book

Building GraphQL APIs with AWS AppSync

How to design, implement, and deploy GraphQL-based APIs on the AWS cloud

The problem case

To see what's the problem with iterables, let's revisit an example from the previous post (Try it):

const iterable = {
	[Symbol.iterator]: () => {
		let num = 0;

		return {
			next: () => {
				return num < 3 ? {value: ++num, done: false} : {done: true};
			}
		}
	}
}

for (let i of iterable) {
	console.log(i); // 1, 2, 3
}

It's 11 lines, and most of it is boilerplate that is only there to make the protocol happy.

Generators to the rescue

Generator functions are denoted with an asterisk as part of the function declaration. Whenever the generator yields a value, the function is paused. And later, when the next() is called, it is resumed.

This pause/resume behavior is unique in JS, as these are the only type of functions that offer this behavior. It also opens the possibilities for other use cases besides iterables. But in this post, we'll concentrate only on this use case.

The above iterable defined as a generator function (Try it):

const gen = function*() {
	for (let num = 1; num <= 3; num++) {
		yield num;
	}
}

for (let i of gen()) {
	console.log(i); // 1, 2, 3
}

There are a few things to notice about generators:

First, the generator function has to be called to produce an iterable. As a result, it can handle arguments, in contrast with the Symbol.iterator function

Second, the protocol boilerplate is completely missing. Since it produces standard iterators, the value and the done properties are there, but they are handled inside the generator function and the for..of loop.

And finally, the function's end finishes the iterator.

Generators also support infinite iterators, just like the manually-written iterables. Since the function is paused, without endlessly calling the next() method, it won't stuck in an infinite loop.

For example, a generator that outputs all natural numbers (Try it):

const numbersGen = function*() {
	let current = 0;

	while (true) {
		yield current++;
	}
}

The while(true) construct is fairly common when working with generators. It takes a little time to override the old habits and get used to them.

Iterators or iterables?

You might have noticed something strange. Calling the generator function returns an iterable, as it can be used in a for.of loop, and it can produce multiple iterators. But one call starts only one pausable function call. What happens when the iterable produced by a generator function creates multiple iterators?

The unfortunate answer is that generators are both iterators and iterables, and their Symbol.iterator function simply returns itself.

Why is this a problem? When working with an iterator, you know it's a one-shot thing. Each element will appear only once, and after the iterator is exhausted, it's useless.

On the other hand, iteration over an iterable should not produce side effects. And exhausting its elements is surely that.

For example, multiple iterations over an array are possible, but not on an iterable generated by a generator (Try it):

// iteration over an array
const arr = [1, 2, 3];

for (let i of arr) {
	console.log(i); // 1, 2, 3
}
for (let i of arr) {
	console.log(i); // 1, 2, 3
}

// iteration over an iterable generated by a generator
const gen = function*() {
	for (let num = 1; num <= 3; num++) {
		yield num;
	}
};

const generated = gen();

for (let i of generated) {
	console.log(i); // 1, 2, 3
}
for (let i of generated) {
	console.log(i); //
}

Because of this, you can not simply pass generators around instead of other iterables, as iterations will change them.

To remedy this, wrap the generator into an iterable that makes a fresh instance for every iteration:

const gen = function*() {
	for (let num = 1; num <= 3; num++) {
		yield num;
	}
};

const wrapGenerator = (gen) => ({
	[Symbol.iterator]: () => {
		return gen();
	}
});

const wrapped = wrapGenerator(gen);

for (let i of wrapped) {
	console.log(i); // 1, 2, 3
}
for (let i of wrapped) {
	console.log(i); // 1, 2, 3
}

Conclusion

Generators are a powerful and terse way to define iterables. Their pause/resume model makes them a lot easier to be understood that traditional next() calls. All while retaining the ability to define infinite collections, which are a lot more useful than you might initially think.

The drawback of generators is their liberal approach of being both iterables and iterators, and blurring the lines between the two. In everyday programming, it's more common to make one-shot generator functions than reusing them, but this is something you need to look out for if you choose to use them.