Partitioning a Stream in Java

Omozegie AziegbeApril 8th, 2024Last Updated: April 8th, 2024

1 215 4 minutes read

Java Streams is a powerful abstraction for processing collections. Often, we need to divide a stream into smaller chunks for further manipulation. This article explores various techniques to partition a Java 8 Stream based on a fixed maximum size.

1. Partitioning with Lists

When dealing with a List, partitioning based on a fixed maximum size can be relatively straightforward using Java 8 Streams. here’s how it works:

Calculate the number of partitions: Determine the number of partitions required based on the size of the list and the desired partition size.
Create partitions: Iterate over the range of partition indices and extract sublists of the original list.
Collect partitions: Collect the sublists into a list of lists.

import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class ListPartitioning {

    public static <T> List<List<T>> partitionList(List<T> list, int partitionSize) {
        return IntStream.range(0, (list.size() + partitionSize - 1) / partitionSize)
                .mapToObj(i -> list.subList(i * partitionSize, Math.min(list.size(), (i + 1) * partitionSize)))
                .collect(Collectors.toList());
    }

    public static void main(String[] args) {
        List<Integer> numbers = IntStream.rangeClosed(1, 20).boxed().collect(Collectors.toList());
        List<List<Integer>> partitions = partitionList(numbers, 5);
        partitions.forEach(System.out::println);
    }
}

To test this method:

We use the IntStream.range to create a stream of integers representing partition indices.
For each index, it extracts a sublist of the original list using list.subList.
The collect method is then used to gather the sublists into a list of lists.

Fig 1: Output from Partition List example

2. Partitioning a Stream Using Guava

Partitioning a stream in Java can be efficiently achieved using Guava. We can leverage Guava’s Lists.partition method to partition a stream into fixed-size chunks. First, let’s ensure we have the Guava library added as a dependency in the project.

2.1 Importing Guava

If you are using Maven, you can include Guava by adding the following dependency to your pom.xml file:

    <dependencies>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>33.0.0-jre</version> <!-- Replace with the latest version -->
        </dependency>
    </dependencies>

For Gradle, include Guava in the build.gradle file:

implementation 'com.google.guava:guava:33.0.0-jre' // Replace with the latest version

Guava’s Lists.partition method allows us to partition a list into fixed-size sublists. This method can be directly applied to a stream to achieve stream partitioning.

import com.google.common.collect.Lists;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class GuavaPartitioning {

    public static void main(String[] args) {
        List<Integer> numbers = IntStream.rangeClosed(1, 10)
                                         .boxed()
                                         .collect(Collectors.toList());

        // Define the partition size
        int partitionSize = 3;

        // Partition the stream using Guava
        List<List<Integer>> partitions = Lists.partition(numbers, partitionSize);

        // Print each partition
        partitions.forEach(System.out::println);
    }
    
}

In this example:

We generate a list of integers from 1 to 10 using IntStream.rangeClosed and collect them into a List<Integer>.
We specify the desired partitionSize (in this case, 3).
We use Lists.partition(numbers, partitionSize) to partition the numbers list into sublists of size partitionSize.
The resulting partitions are stored in a list of lists (List<List<Integer>>), which can be further processed as needed.

The output of this code shows the partitioned sublists, each containing a specified number of elements based on the partition size. In this instance, we partition a list of integers ranging from 1 to 10 with a partition size of 3, and the output is:

[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
[10]

3. Partitioning a Stream Using Guava’s `Iterables.partition`

Partitioning a stream in Java using Guava’s Iterables.partition method also provides an efficient and convenient way to partition/divide a stream or sequence into fixed-size chunks. Here is how we can use it:

import com.google.common.collect.Iterables;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class GuavaPartitionExample {

    public static void main(String[] args) {
        List<Integer> numbers = IntStream.rangeClosed(1, 10)
                                         .boxed()
                                         .collect(Collectors.toList());

        // Define the partition size
        int partitionSize = 3;

        // Partition the stream using Iterables.partition
        Iterable<List<Integer>> partitions = Iterables.partition(numbers, partitionSize);

        // Print each partition
        partitions.forEach(System.out::println);
    }
}

This approach is similar to the one used above using the List.partition method. The only distinction lies in how they handle the input data structure and the resulting partitioned output. In this example:

We use Iterables.partition(numbers, partitionSize) to partition the numbers list into sublists of size partitionSize.
The result is an Iterable<List<Integer>> containing partitions of the original list.

4. Key Points of Partitioning a Stream Using Guava

Simplicity: Guava’s Lists.partition and Iterables.partition methods simplify the task of partitioning a stream into fixed-size chunks.
Flexibility: This approach is versatile and works well with various types of data and partition sizes.
Readability: By leveraging Guava utilities, the code becomes more readable and concise compared to manual partitioning implementations.
Performance: Guava is well-optimized and can efficiently handle partitioning even for large streams.

4.1 Difference between `Iterables.partition` and `List.partition`

The difference between Iterables.partition and List.partition using Guava lies in how they handle the input data structure and the resulting partitioned output.

4.1.1 Iterables.partition

Input: Iterables.partition operates on any Iterable type, not just specifically on List. This means it can work with any collection that implements the Iterable interface, such as List, Set, Queue, etc.
Output: The output of Iterables.partition is an Iterable<List<T>>, where each List<T> represents a partition of the original sequence. This is suitable for lazily partitioning and processing large or infinite sequences without loading the entire data into memory.
Usage: Iterables.partition is more versatile and can be applied to a broader range of data structures, making it suitable for scenarios where you need to work with different types of collections or streams.

4.1.2 List.partition

Input: List.partition specifically operates on List collections. It partitions the list into sublists of a fixed size.
Output: The output of List.partition is a List<List<T>>, where each List<T> represents a partition of the original list. This method is useful when you have a List and want to partition it into smaller chunks, typically when you need random access to the elements within each partition.
Usage: List.partition is ideal when you specifically have a List and want to divide it into smaller lists based on a fixed partition size. It’s straightforward for use cases where you are working exclusively with List collections.

4.2 Choosing Between Them

Use Iterables.partition when you need to partition any Iterable sequence (not limited to List) lazily or when dealing with various types of collections.
Use List.partition when you specifically have a List and want to partition it into smaller sublists for random access or when you prefer to work with List collections exclusively.

5. Conclusion

In conclusion, partitioning a Java 8 Stream based on a fixed maximum size can be achieved using various techniques, ranging from basic Stream operations to utilizing external libraries like Guava. Depending on the specific requirements and constraints of your application, you can choose the most suitable approach to efficiently partition your streams.