GSoC’22 Final Report

In the last three months, as a part of GSoC, I have been contributing to JBoss Community as a student developer. This blog post is a report of the work that I have accomplished in this GSoC period.

Introduction

Infinispan is an open-source in-memory data grid that offers flexible deployment options and robust capabilities for storing, managing, and processing data. Infinispan provides a key/value data store that can hold all types of data, from Java objects to plain text. Js-client is an asynchronous event-driven Infinispan client for Node.js. The results of the asynchronous operations are represented using the Promise instance. This project aims to add more features to the js-client.

Project Details

Organization:- JBOSS
Source Code:- Infinispan js-client

Primary Mentor:- Vittorio Rigamonti
Student:- Neeraj Gartia

Work done

All the newly added feature are described below.

1. Incorporated sasl library code and removed dependency

This was my first PR that got merged. It was a very basic task so did not take much time. The code for the js-client relied on the sasl mechanism(npm package) to perform sasl authentication. So my task was to remove the dependency by importing the source code to the codebase itself.

2. Implemented PING v3.0 operation

Hot Rod is a binary TCP client/server protocol that gives fast access to data in remote infinispan caches. Before this PR the js-client was compatible up to Hot Rod protocol v2.9. This PR extends client compatibility up to v3.0. This extends the PING operation response with additional information :

• the Hot Rod server version
• the list of the Hot Rod operations supported by the server

More details about the Hot Rod protocol can be found here.

3. Added support of protobuf encoded cache

Protocol Buffers (a.k.a., protobuf) are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. You can find protobuf’s documentation on the Google Developers site. Infinispan supports keys and values as protobuf encoded data types. Protostream is a Protobuf-based serialization library, that is used by Infinispan to serialize cache entries preserving their type. The client should support protobuf encode/decode for media type application/x-protostream.

This feature was implemented in four parts:

1). Discussion on the design pattern of the new feature.

The main challenge faced in implementing this feature was to decide the design pattern and deciding the scope of the user. My mentor and I discussed a lot about this. His idea was great. The finalized design pattern to handle protobuf mediatypes can be found here.

We decided to restrict the implementation only to protobuf.js Message instances. The user can turn his js object into a Message instance just by calling MyMessage.create(myJsObj) and the user has to provide the message instance for a different kind of operation in application/x-protostream caches. The message instance has a $type property that has all the methods of a protobuf root instance. So, behind the scene, the js-client will encode the instance i.e. instance.$type.encode(instance) and this will be wrapped in a wrappedMessage Object and will be sent to the server. The definition of the wrappedMessage.proto file can be found here.

2). Protobuf encoding

This part was mainly focused on encoding the media. The pipeline for sending the encoded data to the server was already there and was compatible with the new mediatype also. So the main work was to encode the protobuf instance, wrap it inside the wrappedMessage instance and send it to the pipeline. The commits related to this can be found here. After implementing the encode functionality simple tests for PUT operation were added.

3). Protobuf decoding

In this part decoding the response received from the server was implemented. The pipeline for decoding the headers was compatible with the new mediatype so the main focus was on decoding the body. The body was a protobuf encoded binary data encoded using wrappedMessage.proto. And the content of this can be further decoded using the protobuf root instance provided by the user.

4). Extending the encoding and decoding to basic data types

The application/x-protostream data format also supports the basic data types(integer, boolean, string) as the key and value. But the encoder and decoder added were not compatible with the basic data types. In this part, the support of basic data types was added. Also in this part, the code was refactored in the functional programming paradigm. And the code for the encoder and decoder were shipped to the codec module.

4. Added the support of querying

You can query the infinispan server using its query language Ickle. See here for more details about Ickle. The implementation of the query in js-client can be divided into two parts :

• Query without projection i.e.
  from sample_bank_account.User u where u.name = "John"
• Query with projection i.e.
  select u.name, u.surname, u.age from sample_bank_account.User u where u.age is null

1). Query without projection

The addition of the protobuf encoder and decoder makes the implementation of the query easier. The query request and response proto definition can be found here. Also, the request-response format for the Hot Rod protocol can be found here. The response received from the server is a protocol buffer and it further can be decoded to get the array of protocol buffers encoded using the schema provided by the user. These array of buffers can be decoded using the user schema to obtain the response in JSON format.

2). Query with projection

Infinispan also supports query with projection. Projection queries allow you to query datastore for just those specific properties of an entity that you need, at lower latency and cost than retrieving the entire entity.

In the js-client, the result of select u.name, u.surname, u.age from sample_bank_account.User u where u.age is null will look like this -[ [ name1 , surname1 , null ] , [name2 , surname2 , null ] ].

Query with projection the response needs to be decoded again i.e it is only decoded once using the query.proto and outputs the array and the array is then processed to final output.

5. Typescript adaptor

Currently, the js-client doesn’t provide the typescript support. A Typescript adaptation layer should be provided to ease infinispan-based Typescript application developments. The pull request for this feature can be found here. Since this PR is not merged yet so, I will provide more information regarding this in a separate blog.

The working code example of all these features can be found here.

A lot of code written, a lot of feature addition and a lot of brainstorming along with a great deal of learning is what describes the work done in brief.

Challenges

The most challenging part of this journey was getting acquitted with the functional programming paradigm. The project was written in functional programming which reduces code redundancy, improves modularity, and increases maintainability. But functional programming needs a completely different thinking process than procedural programming or OOP. To adopt a functional programming style I followed the book Functional Javascript which can be found here.

Another major challenge was to decide on the architecture to implement these new features. We had to make the architecture as simple as possible and also we had to restrict the user to certain methods only. But the documentation of Hot Rod and Ickle made the implementation of new features easier.

Experience

I have learned a lot during the GSOC period and the code speaks for itself. With several PRs merged and several approaches discussed, I think that I have become a better developer and a team worker. I communicated with my mentors and met amazing people along the way and learned amazing stuff from them.

This GSoC with JBOSS was an enjoyable experience. The mentors I have been assigned were very friendly, helpful, and supportive. The community as a whole is fabulous and enthusiastic.

The coding experience was enlightening. I have learned a lot of best practices in coding and Open Source. The work was always enjoyable and insightful. Mentors were very generous in explaining why I should be doing things in a particular way. The Overall experience was just awesome. The mentors were very knowledgeable. I don’t have words to portray how fantastic they were.

Acknowledgment

I am very thankful to my mentor Vittorio Rigamonti. Without him, the work never would have been this joyful and rewarding. He was always available to answer my questions on time and also actively ensured that my Pull Requests were reviewed and that the coding standards were up to the mark. He also provided appropriate links to resources and made sure that the deadlines were met through regular meets. Also, I would like to thank Katia Aresti for her support and guidance. Despite her busy schedule, she was ready for a meeting whenever we needed her. I love the JBoss Community for its welcoming and enthusiastic nature.

Finally, thanks to the GSoC program, without which, I wouldn’t be a part of this awesome project and gained this memorable experience. I am thankful to them for creating this kind of opportunity for students.

Future Work

1. The typescript adaptor part can be extended.
2. Linter can be added for code consistency.
3. The protobuf encoder and decoder can be extended to enum data types.
4. The result of the query with projection is an array of an array consisting of only the values. The result of the query in JSON format can be more user-friendly.
5. The base code was written before the evolution of ECMAScript 6. But in ES6 new features were added which should be added to the project. So code re-factoring will be good considering the release of ES6.