Generators and ImmutableJS

Motivation

Iterables are great in theory, but there are only so many language constructs supporting them. For loops are great if you still live in the 90's, but map, filter, reduce and the like will be missed instantly otherwise. In practice, the first thing you'll do is to convert them into something more useful.

ImmutableJS embraces the iterable protocol. Not only that all data structures are standard iterables, but they also allow construction from another iterable. And since generators make iterables, they are fully supported too.

Book

Building GraphQL APIs with AWS AppSync

How to design, implement, and deploy GraphQL-based APIs on the AWS cloud

Building a List from a generator

Let's revisit the generator function from the last post (Try it):

const gen = function*() {
	for (let num = 1; num <= 3; num++) {
		yield num;
	}
}

To construct a List, pass the generated iterable:

console.log(Immutable.List(gen())); // List [1, 2, 3]

The same works for a Set too:

console.log(Immutable.Set(gen())); // Set {1, 2, 3}

This way, you have full-fledged Lists or Sets with their rich API directly from a generator.

Generators with Seqs

But generators can be infinite, which is not suitable for traditional Lists or Sets. On the other hand, Seqs are lazy, and in turn support lazy collections, making them suitable to handle potentially infinite iterables too. At first sight, Seqs are the ideal tool to augment iterables.

For an infinite generator, let's pull one of our previous examples (Try it):

const numbersGen = function*() {
	let current = 0;

	while (true) {
		yield current++;
	}
}

To construct a Seq from a generator, pass the generated iterable:

const numbersSeq = Immutable.Seq(numbersGen()); // Seq

From now on, it is just like any other Seq:

// Sum the first 10 even numbers
numbersSeq
	.filter((n) => n % 2 === 0)
	.take(10)
	.reduce((acc, n) => acc + n, 0); // 90

In case the generator does nothing but produces a stream of numbers, ImmutableJS also has a built-in class called Range:

Immutable.Range()
	.filter((n) => n % 2 === 0)
	.take(10)
	.reduce((acc, n) => acc + n, 0); // 90

They work the same, but Range does not need the generator function. For simple cases, use Range, but for more complex ones that can not be covered with a Range, for example, a Fibonacci sequence, use a generator.

Iterable caching

Let's do an experiment with Seqs constructed from generators!

In the previous post, we've seen that generators can be iterated over only once. What about a Seq, initialized from a generator?

After calculating the sum of the first 10 even numbers, let's reuse the same Seq and calculate the first 20 even numbers too! (Try it):

// Sum the first 20 even numbers too
numbersSeq
	.filter((n) => n % 2 === 0)
	.take(20)
	.reduce((acc, n) => acc + n, 0); // 380

It works. But why? The Seq can not pull the first 10 even numbers twice from the generator, but somehow it still manages to calculate the correct result.

It turns out that the Seq caches the iterable so that it can produce reproducible results.

Usually, it's a good thing. In particular, they fix the generators' peculiarity of being both iterables and iterators. On the other hand, ignored elements still consume memory.

For example, skip the first 100000 elements and get only 1 item after them. The generator and the Range produce the same Seq:

numbersSeq
	.skip(100000)
	.take(1); // Seq [100000]

Immutable.Range()
	.skip(100000)
	.take(1); // Seq [100000]

But while the Range does not consume more memory than before, the Seq with the generator is. Keep this in mind when you need to skip a lot of elements.

Another problematic use case is when you have a generator that is continuously running in your app, for example, a random number generator. If you wrap it in a Seq, it will slowly leak memory.

Conclusion

ImmutableJS supports iterables and generators with all its collection types. You can build a List from a generator, or wrap with a Seq for a more developer-friendly API.

This will work, but watch out for the iterable-caching behavior of Seqs. In some rare cases, this could make an algorithm memory-bound, or the app to leak.