Business to Business integration with Ballerina

Ballerina 2201.6.0 (Swan Lake Update 6) is used in this article.

Let’s consider the following scenario: Voltex, an electronics appliance manufacturer, supplies its high-tech products to several retailers, including TechTown. Voltex also depends heavily on several suppliers for parts. One day, TechTown launches a significant promotion for Voltex’s products. However, outdated manual ordering systems prevent TechTown from placing additional orders to Voltex in a timely manner. When Voltex gets these late orders, their own manual systems slow them down, and they couldn’t quickly arrange more supplies. This coordination failure triggers a devastating supply-demand mismatch, causing order cancellations, customer dissatisfaction, and a tarnished reputation.

Root cause of this disastrous situation?

Lack of proper integrations among business partners.

B2B integrations

Integrations can be categorized into three types: internal, Business to Consumer (B2C), and Business to Business (B2B). Internal integrations involve varied standards, often connected via an internal integration layer (e.g., service bus). Here, the data exchanges are relatively flexible, facilitated by the collaborations among developers. In B2C communications, organizations generally control the interface design (APIs) and data exposure. In contrast, B2B integrations necessitate stricter adherence to data format standards and stringent controls over the data exchanged. Alterations here are infrequent, necessitating inter-organizational coordination. However, as we saw in the above example, B2B integrations and timely orchestrations across partner systems also play a key role in digital transformation initiatives.

Let’s take an example from the retail domain. A retail store may have an online shopping portal and a mobile app, which are backed by a REST API. Here, the retail store (i.e., business) interacts with consumers, which is a B2C integration. This retail store may contain many internal systems to handle various business operations such as Sales, procurement, inventory, etc. These systems also need to communicate with each other, which is an internal integration. Then this retail store needs to communicate with manufacturers to order goods, where for example, the retail store’s procurement system connects with a sales service of the manufacturer to place orders. This is an example of a B2B integration.

Internal, B2C, and B2B integrations in retail domain

Of course, we do have popular data formats, such as JSON and XML, used for internal and B2C integrations. So why don’t we use those and come up with some schemas for B2B integrations? There are a few issues in doing so…

• B2B data formats and schemas require consensus among all involved companies.
• A single B2B message, often containing a vast amount of data, necessitates complex schema definitions. This labor-intensive process typically requires multiple iterations.
• Pre-JSON/XML era standards for B2B integrations were widely adopted, making it unlikely for new schema sets to gain acceptance across organizations.

Therefore, using current B2B standards is the only way for B2B integrations in most situations.

Electronic Data Interchange — The B2B format

Electronic Data Interchange (EDI) is the de facto standard for B2B integrations, heavily adopted by organizations across multiple industries. Below is an example EDI message:

HDR*ORDER_1201*ABC_Store*2008–01–01~
ITM*A-250*12~
ITM*A-45*100~
ITM*D-10*58~

Each line in the EDI message is called a segment. The above message starts with a header segment (HDR), specifying it as order number 1201 from ABC_Store dated 2008–01–01. The subsequent segments (ITM) are the individual items within the order. Each ITM segment lists a product code (e.g., A-250, A-45, D-10) and the corresponding quantities ordered (e.g., 12, 100, 58). The tilde (~) marks the end of each segment.

In reality, EDI message structures are far more complex than the above example and can contain numerous types of segments and segment groups. This complexity, along with the need for agreement among organizations, has led to the standardization of EDI message types under various specifications like X12 and EDIFACT. For instance, X12 has established standards for over 350 B2B message types. These include an invoice message (EDI 810), purchase order (EDI 850), payment order (EDI 820), shipment notice (EDI 856), and insurance claim (EDI 837), among others.

B2B integration architecture

Now, let’s focus on how we can adopt EDI for B2B integrations with our partners. Below are some points we need to consider in this process:

• Usually, EDI is used only for B2B communications
• Integrations among internal systems use formats such as JSON, XML, and gRPC/protobuf.
• Although EDI message types are standardized, partners may use custom variations of those standards.
• EDI messages are typically exchanged over AS2 and FTP protocols.

Due to these differences between B2B and internal integrations, a B2B gateway is used to bridge B2B and internal communications as shown below:

B2B gateway architecture

In the above architecture, all external communications go through the partner management and authentication module. This module manages partner certificates and authenticates incoming messages using identifiers like partner IDs and digital signatures. A separate module is used per each trading partner, which converts partner-specific EDI formats into a standardized internal format. EDI processing and transformation module performs validations, filtering, enrichment, etc., on EDI messages and converts those to relevant internal message formats such as JSON and XML. This module also sends EDI payloads as well as metadata to the message persistence and EDI tracking module, which further processes those and stores in a data store for auditing, replaying, and monitoring purposes.

When considering the actual implementation, the components described above may be deployed as multiple runtime components, especially in microservices environments. Above B2B gateway functionality may also be integrated into the application logic itself depending on the technology stack used.

EDI processing with Ballerina

Now, let’s consider the actual implementation step. Among many options, in this article we choose Ballerina language, mainly due to its built-in support for EDI and suitability of Ballerina for data-oriented business application development. Ballerina language simplifies the handling of complex EDI messages by converting them into Ballerina records and vice versa based on a specified schema. Once converted, EDI data can be manipulated just like any other record, facilitating any data processing operation. For instance, users can validate EDI fields against organization-specific criteria, extract relevant fields to create new records, save selected data elements into a database, or route data based on EDI field values.

Let’s consider the simple EDI schema for a sales order below:

This schema consists of two main sections: header segment and itemDetails segment group. The header segment contains required fields such as code, orderId, organization, and an optional field date. The itemDetails segment group can occur multiple times and contains two segments: item and supplier. The item segment requires fields like code, item, and quantity (with an integer data type). The supplier segment requires fields code, supplierCode, and an optional promotionCode.

An example EDI message of this schema is shown below:

We can generate Ballerina records and utility methods for this EDI schema and process EDI messages as shown in the steps below:

Use the below command to get the Ballerina EDI tool:

bal tool pull edi

Create a new Ballerina package using the following command:

bal new order_processing

Go to the order_processing folder and use the below command to generate Ballerina code for the above schema:

bal edi codegen -s resources/schema.json -o sales_order.bal

Above command assumes the schema is in the resources folder. However, it can be placed anywhere in the file system.

This will generate a file containing all records required for representing data in sales order EDI messages. It also contains two functions to convert EDI messages to records and vice-versa.

Ballerina code generated by the EDI tool

Now we can use the generated code to load EDI messages into Ballerina records and apply any operation on those records.

Ballerina program for printing the order Id, customer organization name and total number of ordered items from an EDI message

This simplified EDI handling capabilities and flexibility in deployment allows developers to use Ballerina to implement EDI processing according any preferred architecture. For example, it is possible to use a scalable Ballerina application as a centralized B2B gateway. It is also possible to implement separate Ballerina microservices to handle different EDI capabilities, such as bridging message variations, EDI to internal format conversion, validation, transformation, persistence, etc., and deploy these as cloud-native applications (e.g. in Choreo). On the other extreme, especially for simple applications, it is possible integrate EDI processing steps into the application logic itself.