Iterate over any Iterable in Java

Eclipse Collections supplies iteration patterns for any Iterable type.

Photo by Tim Johnson on Unsplash

Iterating over collections

Iterating over collections is a fundamental feature of any modern programming language. There are many common iteration patterns that have well known names and alternatives like filter (select/reject), map (collect), reduce (injectInto), groupBy. There are four approaches to implementing iteration patterns — eager/serial, eager/parallel, lazy/serial, and lazy/parallel. Java Streams provide lazy/serial and lazy/parallel iteration patterns. Terminal operations like forEach, collect, any/all/noneMatch, force iteration execution to happen. Terminal operations are eager operations.

Eclipse Collections has offered eager/serial and eager/parallel iteration patterns for a very long time. Eager iteration patterns execute immediately. They are the equivalent of iteration code developers would write by hand using for loops.

Eclipse Collections also offers lazy/serial and lazy/parallel iteration patterns. I will not discuss lazy/parallel iteration in this blog. Instead I wanted to describe the eager/serial, eager/parallel and lazy/serial iteration patterns that were implemented in Eclipse Collections via utility classes that works with any java.lang.Iterable type. These classes still exist in Eclipse Collections today and can be useful for executing eager or lazy operations against any Iterable type.

If you’d like to understand more about eager and lazy iteration, the following blog explains the differences in detail.

Eager is Easy, Lazy is Labyrinthine

Looping with Iterable

Since Java 5, the Java Collections framework has had the java.lang.Iterable interface, which is the parent interface of java.util.Collection. In Java 5, the enhanced for loop was added that would work with any implementation of Iterable. The enhanced for loop allowed developers to write more concise loops when iterating over Java Collection implementations and other Iterable types.

Before Java 5, we had to write code like this to iterate over a java.util.Collection using a for loop.

Collection<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
for (Iterator<Integer> it = list.iterator(); it.hasNext(); )
{
    Integer each = it.next();
    System.out.println(each);
}

After Java 5, we could use the less verbose enhanced for loop as illustrated in the following example.

Iterable<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
for (Integer each : list)
{
    System.out.println(each);
}

Recharging Iterable with forEach

Eclipse Collections was initially developed using JDK 1.4. The first iteration method in Eclipse Collections, named forEach, was added to a utility class named Iterate and took a functional interface type called Procedure as a parameter. Before Java 5, we could use Iterate and forEach with any java.util.Collection. We had to resort to using anonymous inner classes to work with methods like forEach. After Java 5, the Iterate utility was updated to work with java.lang.Iterable instead of java.util.Collection.

The following code shows how you could use the Eclipse Collections Iterate utility and its forEach method after Java 5 and before Java 8.

Iterable<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
Iterate.forEach(list, new Procedure<Integer>()
{
    @Override
    public void value(Integer each)
    {
        System.out.println(each);
    }
});

If your reaction to this code is “yuck!”, then you would not be alone. Why would anyone agree to write code like this? I believed since I started programming in Java that it was inevitable that Java would eventually get lambdas, and this style of coding with anonymous inner classes would eventually be replaced with something much more concise and readable. Once Java 8 provided support for lambdas and method references to the Java development community, coding patterns using anonymous inner classes were able to be converted using automated refactoring tools.

The same method when used with Java 8 or above looks as follows.

Iterable<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
Iterate.forEach(list, each -> System.out.println(each));

This code can be further simplified by using a method reference shown below.

Iterable<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
Iterate.forEach(list, System.out::println);

Since the release of Java 8, the need for Iterate.forEach was lessened by the addition of the default implementation of forEach that was added to Iterable. With the new default forEach method, the following code works with Iterable.

Iterable<Integer> list = Arrays.asList(1, 2, 3, 4, 5);
list.forEach(System.out::println);

Eager Iteration methods for any Iterable type

The Iterate utility class provides much more than just forEach. There are many eager iteration methods provided for any Iterable type. Browse the Javadoc below to find out what methods are available.

Iterate (Eclipse Collections - 11.1.0)

Structure

A more compact view of the methods on Iterate is available by using the Structure view in IntelliJ. There are over 130 methods available on the Iterate class. I include a code example of a method from the Iterate class contained in each screenshot in the sections that follow.

Methods from addAllIterable to collectInt

Examples — any and all

The methods anySatisfy and allSatisfy are the equivalents of Java Streams anyMatch and allMatch. The methods anySatisfyWith and allSatisfyWith take an extra parameter which makes it possible to use them with more method references.

@Test
public void anyAndAllSatisfy()
{
    List<String> list = List.of("cat", "bat", "rat");

   // Java Streams
    Assertions.assertTrue(
            list.stream().anyMatch(each -> each.contains("at")));
    Assertions.assertTrue(
            list.stream().allMatch(each -> each.contains("at")));

    // Eclipse Collections Iterate
    Assertions.assertTrue(
            Iterate.anySatisfy(list, each -> each.contains("at")));
    Assertions.assertTrue(
            Iterate.allSatisfy(list, each -> each.contains("at")));

    // Eclipse Collections Iterate "With"
    Assertions.assertTrue(
            Iterate.anySatisfyWith(list, String::contains, "at"));
    Assertions.assertTrue(
            Iterate.allSatisfyWith(list, String::contains, "at"));
}

Methods from collectLong to getOnly

Examples — detect

The method detect finds the first element that matches a Predicate. There are also detectOptional, detectWith, and detectWithOptional versions.

@Test
public void detect()
{
    List<String> list = List.of("cat", "bat", "rat");

    // Java Streams
    Assertions.assertEquals(
            "cat",
            list.stream()
                    .filter(each -> each.contains("at"))
                    .findAny()
                    .orElse(null));

    // Eclipse Collections Iterate
    Assertions.assertEquals(
            "cat",
            Iterate.detectOptional(list, each -> each.contains("at"))
                    .orElse(null));
    Assertions.assertEquals(
            "cat",
            Iterate.detect(list, each -> each.contains("at")));

    // Eclipse Collections Iterate "With"
    Assertions.assertEquals(
            "cat",
            Iterate.detectWithOptional(list, String::contains, "at")
                    .orElse(null));
    Assertions.assertEquals(
            "cat",
            Iterate.detectWith(list, String::contains, "at"));
}

Methods from groupBy to reject

Example — makeString

The method makeString is the equivalent of Collectors.joining. One notable difference is that makeString does not require an Object to be converted to a String first.

@Test
public void makeString()
{
    List<Integer> integers = List.of(1, 2, 3);

    // Java Streams
    Assertions.assertEquals(
            "1, 2, 3",
            integers.stream()
                    .map(Object::toString)
                    .collect(Collectors.joining(", ")));

    // Eclipse Collections Iterate
    Assertions.assertEquals(
            "1, 2, 3",
            Iterate.makeString(integers, ", "));
}

Methods from reject to take

Example — sumOfInt

The method sumOfInt returns the sum of some IntFunction applied to each element of the Collection. The difference between IntStream.sum and sumOfInt is that IntStream returns an int, which may quietly overflow. The sumOfInt method widens to a long, which will handle summing much larger numbers.

@Test
public void sumOfInt()
{
    List<Integer> integers = List.of(1, 2, 3);

    // Java Streams
    Assertions.assertEquals(
            6,
            integers.stream()
                    .mapToInt(Integer::intValue)
                    .sum());

    // Eclipse Collections Iterate
    Assertions.assertEquals(
            6L,
            Iterate.sumOfInt(integers, Integer::intValue));
}

Methods from toArray to zipWithIndex

Example — zip

The method zip takes two Iterable instances, and creates a List of Pair instances. There is no equivalent in Java Stream, but this does work with any two Iterable instances.

@Test
public void zip()
{
    List<Integer> integers = List.of(1, 2, 3);
    List<String> strings = List.of("1", "2", "3");

    // Eclipse Collections Iterate
    List<Pair<Integer, String>> zipped =
            Iterate.zip(integers, strings, new ArrayList<>());

    List<Pair<Integer, String>> expected = List.of(
            Tuples.pair(1, "1"),
            Tuples.pair(2, "2"),
            Tuples.pair(3, "3"));
    Assertions.assertEquals(expected, zipped);
}

Optimized by type

Iterate does its best to optimize each eager iteration method by type. There are instanceof checks that look for ArrayList, List, and RandomAccess.

Select with a targetCollection optimized by type

The Futility of Utility

Utility classes can be very useful for extending the capabilities of types without having the expand the interface of the types. Unfortunately, utility classes also have a problem — what type should they return?

You only get one shot

The Iterate utility takes Iterable as a parameter, and usually returns Collection as a result. Collection is the most usesful abstract type to return. Unfortunately, Collection is not as useful as List or Set in terms of communicating the capabilities of the result.

There are methods on Iterate that take the return result as a parameter, and return the same result. These methods, although slightly more verbose, are the most useful as they provide the most specific return type.

The following code example illustrates the differences of return type between the overloaded forms of collect on the Iterate class. The method collect in Eclipse Collections is the equivalent of map on Stream. The name collect used in Eclipse Collections comes from the same method name in the Collections framework in the Smalltalk-80 programming language, which has been around since 1980.

@Test
public void collectOnIterate()
{
    Set<Integer> set = Set.of(1, 2, 3);
    Set<String> expected = Set.of("1", "2", "3");

    // Return type of Collection
    Collection<String> collect =
            Iterate.collect(set, Object::toString);

    Assertions.assertEquals(expected, collect);

    CopyOnWriteArraySet<String> target = new CopyOnWriteArraySet<>();
    
    // Return type is the same type as the target parameter
    CopyOnWriteArraySet<String> collectWithTarget =
            Iterate.collect(set, Object::toString, target);

    Assertions.assertEquals(expected, collectWithTarget);
}

If I had the opportunity to re-build Iterate from scratch, I would only provide the methods which take a target collection as a parameter and return the same type as the target collection. While these methods are more verbose, they are also more versatile and useful.

Iterate and other static utility classes

In addition to Iterate, there are several other *Iterate static utility classes.

Iterate and friends

ParallelIterate is the eager/parallel equivalent of Iterate. ParallelIterate also takes Iterable as a parameter.

The following code example shows the equivalent of collect using ParallelIterate.

@Test
public void collectOnParallelIterate()
{
    Set<Integer> set = Set.of(1, 2, 3);
    Set<String> expected = Set.of("1", "2", "3");

    // Return type of Collection
    Collection<String> collect =
            ParallelIterate.collect(set, Object::toString);

    Assertions.assertEquals(expected, collect);

    CopyOnWriteArraySet<String> target = new CopyOnWriteArraySet<>();

    // Return type is the same type as the target parameter
    CopyOnWriteArraySet<String> collectWithTarget =
            ParallelIterate.collect(set, Object::toString, target, true);

    Assertions.assertEquals(expected, collectWithTarget);
}

ListIterate returns MutableList instead of Collection. If you know you are iterating over a List, then using this utility class gives you access to methods that work with any List, and returns a MutableList, which has the extensive Eclipse Collections API available.

@Test
public void collectOnListIterate()
{
    List<Integer> list = List.of(1, 2, 3);
    List<String> expected = Lists.mutable.of("1", "2", "3");

    // Return type of MutableList
    MutableList<String> collect =
            ListIterate.collect(list, Object::toString);

    Assertions.assertEquals(expected, collect);
    // Extensive Eclipse Collections API available on MutableList
    Assertions.assertEquals("1, 2, 3", collect.makeString());

    MultiReaderList<String> target = Lists.multiReader.empty();

    // Return type is the same type as the target parameter
    MultiReaderList<String> collectWithTarget =
            ListIterate.collect(list, Object::toString, target);

    Assertions.assertEquals(expected, collectWithTarget);
}

Over the years, many developers have opted for using ListIterate instead of Iterate. This is primarily because Iterable as a type has never seen widespread use. List and Set are much more commonly used types than either Iterable or Collection. ParallelIterate continues to demonstrate good performance for use cases where parallelism has been proven to be useful.

Lazy Iteration methods for any Iterable type

While Iterate and ParallelIterate provide eager iteration patterns for any Iterable, LazyIterate provides lazy iteration patterns for any Iterable. LazyIterate will create a LazyIterable that then requires a terminal operation to iterate over the collection, similar to how Java Stream operates. Browse the Javadoc below to find out what methods are available.

LazyIterate (Eclipse Collections - 11.1.0)

LazyIterate was created much later than Iterate and ParallelIterate. By the time LazyIterate was created, Eclipse Collections already had an extensive RichIterable interface hierarchy, which included the LazyIterable interface.

Structure

A concise view of the methods is available by using the Structure view in IntelliJ. In this case, concise refers to the method signatures which contain all of the information without as much of the structure.

Example — adapt

The method adapt, adapts any Iterable as a LazyIterable, which has the complete API of RichIterable, which LazyIterable extends.

@Test
public void adaptOnLazyIterate()
{
    List<Integer> list = List.of(1, 2, 3);
    List<String> expected = Lists.mutable.of("1", "2", "3");

    // Adapt the List as a LazyIterable
    LazyIterable<Integer> iterable = LazyIterate.adapt(list);
    // Return type of MutableList
    MutableList<String> collectToList = iterable
            .collect(Object::toString)
            .toList();

    Assertions.assertEquals(expected, collectToList);
    // Extensive Eclipse Collections API available on MutableList
    Assertions.assertEquals("1, 2, 3", collectToList.makeString());
}

Example — collect

The lazy form of collect is very similar to map in Java Stream in terms of how it behaves. The difference between Stream and LazyIterable on the other hand is enormous.

@Test
public void collectOnLazyIterate()
{
    List<Integer> list = List.of(1, 2, 3);
    List<String> expected = Lists.mutable.of("1", "2", "3");

    // Return type of LazyIterable
    LazyIterable<String> collect =
            LazyIterate.collect(list, Object::toString);
    // Return type of MutableList
    MutableList<String> toList = collect.toList();

    Assertions.assertEquals(expected, toList);
    // Extensive Eclipse Collections API available on MutableList
    Assertions.assertEquals("1, 2, 3", toList.makeString());
}

The following is the equivalent code for comparison using Java Stream and the map method.

@Test
public void mapOnStream()
{
    List<Integer> list = List.of(1, 2, 3);
    List<String> expected = Lists.mutable.of("1", "2", "3");

    // Return type of Stream
    Stream<String> map =
            list.stream().map(Object::toString);
    // Return type of List
    List<String> toList = map.toList();

    Assertions.assertEquals(expected, toList);
    Assertions.assertEquals("1, 2, 3", toList.stream()
            .collect(Collectors.joining(", ")));
}

The Utility of Utility

Utility classes allow you to extend the behavior of types you have no direct control over. In the cases described above, Eclipse Collections has extended the possibilities for all Iterable types in Java. Java Stream is great, but does not work directly with Iterable. Stream also provides mostly lazy behavior.

The Iterate, ParallelIterate and ListIterate utility classes provide eager behaviors for Iterable and List. There can be a challenge when it comes to choosing return types when using utility classes (you get one choice), but ListIterate and LazyIterate show that it is possible to bridge abstract and anemic types like Iterable with rich and fluent types like MutableList and LazyIterable.

Iterate was the first class in Eclipse Collections to provide a rich eager/serial API initially to java.util.Collection, and later to java.lang.Iterable. It has survived a long time in the library, and still continues to provide useful behaviors.

Thank you for reading this blog! I hope you have found it interesting and informational!

I am the creator of and committer for the Eclipse Collections OSS project, which is managed at the Eclipse Foundation. Eclipse Collections is open for contributions.