Reactive programming with Java 8 and simple-react : sharding

If you want to split a Stream of data into separate Streams for processing simple-react provides sharding functionality.

Example :

Imagine we have a Stream of User data, we want to process that data and then store it in different files. By sharding the data into different Streams we can create Streams that will ultimately write data to a single file, removing the need for any locking or synchronisation that might be required with a different threading approach.

First lest’s build an async.Queue for each Shard

Map<Integer,Queue<User>> shards = new HashMap<>();
 shards.put(1,new Queue<>());
 shards.put(2,new Queue<>());
 shards.put(3,new Queue<>());

Now if we process a Stream of Users we can assign them to a different shard based on some attribute (for example the modulus of their user id).

Map<Integer,LazyFutureStream<User>> sharded =        
                   LazyFutureStream.sequentialBuilder()
                        .react(()->loadUserData())
                        .flatMap(Collection::stream)
                        .shard(shards,user -> user.getUserId()%3);

The Tutorial : Reactive programming with Java 8