CRUD operations and Message Transformation with Ballerina

Using Ballerina’s drag-and-drop interface for some of the most important services involved in integration

Note: Following content is applicable for version 0.8

Ballerina brings out powerful, flexible yet simple capability to the table that makes integration solutions viable through less effort and incorporates diverse audiences. More on Ballerina…[1], [2], [3] . In the integration space, CRUD operations and message transformations become a necessity in scenarios that integrate applications using different message formats. Development of these services needs mastering of a particular language area or a mechanism. Ballerina provides a solution where such services can be implemented using a simple drag and drop user interface, with minimum learning effort for the solution provider.

Power of Ballerina choreographing — The Composer

Ballerina is designed to achieve it’s choreographing by a very powerful yet simple mechanism, which is the good old sequence diagram. Everyone in need of an integration solution understands sequence diagrams as it’s a basic concept used every day. Ballerina comes with a composer that lets you draw your integration scenarios using sequence diagrams, which makes it’s easy to understand and hassle free way to express in discussions.

A user interface with drag and drop capability is provided to make code creation experience a simple and fun one. In addition to this the developer has the capability to switch between Source and Swagger views. This enhances user experience as the developer has the freedom to use what he is most comfortable in using. More on composer….[4]

A Choreograph, which is Simple yet Powerful and Flexible

CRUD operations and message transformation are very common when integrating most systems as introduced before. Therefore, this article is written to demonstrate how easily this can be achieved using Ballerina.

Scenario

FransEdu is an educational entity that needed an integration solution to connect it’s traditional databases with its day to day operations such as registering students, changing student status, etc; Currently they have a web app that directly talks to the database but now they want to move forward and expand their IT infrastructure in order to meet their expansion goals planned for the future.

Figure 1: Current FransEdu IT infrastructure

Their new solution will be to expose the database using data service that provides CRUD operations. Furthermore, they will decouple their web app and the actual business logic, so that the web app will be lightweight. The payload that the FransEdu web page is sending in a request does not directly matches to the required message format by the data service.

Payload sent by FransEdu website:

<studentDetails>
<studentId>stu_col_0001</studentId>
<studentName>Jenny Healer</studentName>
<studentAddress>123,Strreet lane, Down road,MV,SL</studentAddress>
<studentContact>0710000011</studentContact>
<studentRegDate>2016–05–02</studentRegDate>
<studentStatus>active</studentStatus>
</studentsDetails>

Payload required by the Data Service:

{“students”:{“student_id”:”stu_col_0001", “student_name”:”Jenny Healer”, “student_address”:”123,Strreet lane, Down road,MV,SL”, “student_contact”:”0710000011", “student_reg_date”:”2016–05–02", “student_status”:”active”}}

In order to solve the above problem of accommodating we have the following options

  1. Change the FransEdu website so that it sends out the needed payload
  2. Change the Data Service so that it incorporates the payload sent by the web page directly
  3. Use a message transformation mechanism so that neither the Data Service or the FransEdu web page has to be changed.

For an enterprise, adjusting its current infrastructure from scratch to adjust to changes will be hard. As change is inevitable, the continuous change of the existing infrastructure to accommodate the new additions in place will take considerable amount of redesign, rewrite, etc. will be a concern when it comes to waste of effective hours. Thus, the best approach is to leave the existing infrastructure as it is and use a mechanism that can let the existing components communicate with each other easily. To a situation as this, the best available option would be the above mentioned third point as it leverages the minimum impact to the existing infrastructure. An integrator web service can be used to perform message transformations so that the FransEdu web site and the Data service can communicate with each other. With an integrator in place, it will be an interface for further extensions when needed as it provides capability to incorporate various enterprise components to achieve a certain integration aspect.

Figure 2: The IT infrastructure FransEdu is expecting to develop

Ballerina can be utilized to make this integration scenario possible with a simple yet flexible approach as depicted below

The Data Service

In this scenario the data service has to expose an API where CRUD operations can be performed through the service on the database. Ballerina provides the following capabilities

  • JDBC Connectivity with retrieve, insert and update capabilities
  • Ability to design a service with HTTP methods GET, POST, PUT, DELETE, HEAD, PATCH.
  • Ability to design services as needed with multiple resources.
  • Ability to model the service and its operations visually using the Composer, where the Ballerina code will be generated automatically. Alternatively, directly write the code using the Composer or your favourite IDE (Ballerina release consists of plugins for IntelliJ IDEA, Atom, Vim, Sublime and VS Code [5]

Using the above capabilities in Ballerina the data service is basically drawn in the canvas using drag and drop as depicted in the below diagram using the Composer with JDBC connectivity.

Figure 3: Composer view of the Data service — Insert Operation

Let’s take a look at the components available in the resource depicted above.

StudentDataService — This is the component that creates a service on the canvas. A simple drag and drop of the service component lets the developer create a service with ease as the code is automatically generated in real time while you drag and drop. A service can consist of multiple resources so that it represents different database operations separately. The service has configurable parameters such as BasePath, QueryParams, etc… and the Composer provides much flexibility in providing all what a developer needs so that he has to just enter desired values as needed. Let’s take a closer look at the individual components of this service.

Figure 4: Service configuration for StudentDataService

insertStudent — This resource can be used to insert student records in the DB. The configuration for this particular resource is illustrated below

Figure 5: Resource configuration for insertStudent resource

Client — This is the lifeline of the client. It depicts requests and responses. This clearly illustrates at which points the request comes in and the reply goes back to the client.

Resource Worker — Lifeline of the program statement execution. Within the lifeline of the resource worker, the relevant logic is executed.

Start — Starts the invocation when a request is generated by the client

json payload = messages:getJsonPayload(m) — Retrieves the json payload from the incoming message. There is no coding needed. It’s just a drag and drop of getJsonPayload component under ballerina.lang.messages

insertToStudent(payload) — This is a function invocation. A drag and drop of the function invocation construct let’s this be created.

message response = {} — Initialise response message .

reply response — This is the end execution of the resource worker life line. Reply returns response to the client.

Next let’s take a look at how the automatically generated Ballerina code through drag and drop in Composer performs the JDBC insert operation. As depicted in the figure 3, the insertToStudent function invocation calls the relevant function that performs database insert. Following is the code that performs the insert operation on the database

Figure 6: Code snippet that performs JDBC insert.

The relevant packages that addresses JDBC connections are ballerina.lang.sql and ballerina.lang.datatables. In order to understand the above code better please find the below interpretation

Special pointers in code

Line 10: A JDBC connection the database is initialized and obtain through a function invocation. In order to maintain abstraction, in this sample the connection is made in another ballerina file. Below is the code for the connection.

function getConnectionDetails()(sql:ClientConnector dbConnector){
map propertiesMap = {“jdbcUrl”:”jdbc:mysql://localhost:3306/sample?useSSL=false”, “username”:”baladmin”, “password”:”baladmin”};
dbConnector = create sql:ClientConnector(propertiesMap);
return;
}

Rest of the code: This code reads values from the incoming json payload (lines 11–23), and sets them as parameters to be used for the sql insert operation (lines 25–31). After defining the insert query as a string value, the parameters, the query is passed to the Sql Client Connector using it’s update method.

Similarly update operation on database is defined through the dataservice as given below

Figure 7: Compose view of the Data Service — Update Operation

The Composer is designed in such a way that when the user is drawing the diagram, the code is created in real time with each drag and drop in the source view. This gives the capability for the user to edit either in the Design view or the Source view.

Figure 8: Source view of Data Service in Composer

The Integrator

As per the current expectation, the integrator should integrate the web page of FransEdu and the above data service while providing abstraction to the data service. Furthermore the integrator is expected to provide an interface so that message type conversions are provided. This will enable the existing web page to be unchanged and integrated with much more power as it doesn’t change the existing systems at all.

Ballerina currently provides capability to perform type conversions from and to xml, json, string. Furthermore Ballerina is written in such a way that the developer can perform this conversions using his/her own code written via Ballerina that provides more flexibility. Basically a developer either can use the default Type converters in Ballerina or draw a converter to suit his/her customized needs.

In the scenario that we are using, the FransEdu web site sends in payload in form of xml and as discussed earlier, the Data service requires it in json. Hence we need to perform a message transformation which is the reason the Integrator service is developed. Below sample code illustrates how a xml payload is read and then converted to a json.

function transformPayloadToJson(xml payload)(json transformedPayload){
transformedPayload = `{“students”:{“student_id”:”dummyVal”, “student_name”:”dummyVal”, “student_address”:”dummyVal”, “student_contact”:”dummyVal”, “student_reg_date”:”dummyVal”, “student_status”:”dummyVal”}}`;
string studentId = xmls:getString(payload, “/studentDetails/studentId/text()”);
string studentName = xmls:getString(payload, “/studentDetails/studentName/text()”);
string studentAddress = xmls:getString(payload,”/studentDetails/studentAddress/text()”);
string studentRegDate = xmls:getString(payload,”/studentDetails/studentRegDate/text()”);
string studentStatus = xmls:getString(payload,”/studentDetails/studentStatus/text()”);
string studentContact = xmls:getString(payload,”/studentDetails/studentContact/text()”);
jsons:set(transformedPayload,”$.students.student_id”,studentId);
jsons:set(transformedPayload,”$.students.student_name”,studentName);
jsons:set(transformedPayload,”$.students.student_address”,studentAddress);
jsons:set(transformedPayload,”$.students.student_contact”,studentContact);
jsons:set(transformedPayload,”$.students.student_reg_date”,studentRegDate);
jsons:set(transformedPayload,”$.students.student_status”,studentStatus);
system:println(strings:valueOf(transformedPayload));
return;
}

The Integrator which provides an abstract layer is drawn using the Composer as shown below

Figure 9: Composer view of the Integrator — Register student

Figure 10: Composer view of the Integrator — Deactivate student

The service representation using the Swagger is as below. The Swagger too gets created in real time while the user is creating the service in design view which provides flexibility for him to switch and see how the swagger is made through each drag and drop.

Figure 11: Swagger view of the Integrator using Composer

Conclusion

Every dancer has his or her unique style of movement. . A great talented choreographer takes these beautiful dancers and creates a performance that brings out the beauty of dancing on stage that creates a powerful impact on the audience. In the integration world Ballerina weaves an integration among different applications, databases, resources etc; by providing the capability to draw what needs to be coded. With Ballerina, a powerful, flexible and beautiful solution can be created in the simplest available manner using the best Composer in the world of Integration.

References

[1] Conceiving Ballerina — From Dataflow to Sequence Diagrams

[2] Ballerina — Let’s dance !!!!!

[3] Ballerina, the programming language for geeks, architects, marketers and rest

[4] The Ballerina Composer: the visual tool that lets us code with diagrams

[5] Ballerinalang.org