Some have suggested that Finland should ditch the X-Road and build everything from the scratch and again with REST only. A few months back I would have been with the herd, but now I don’t. X-Road has benifits and it’s tested and therefore I would not let this opportunity to go by. Instead, let’s build a way to use REST/JSON in the current version of X-Road.
X-Road will enable quicker start with the digitalisation of services than building it all over again, finding and fixing all the nasty bugs. If you are a developer, you know that the last 5% of the app takes 95% of your time to get things done. With using a adapter server which is already a part of the X-Road usage, we can go forward and start thinking of approriate moment to make transition to REST-only X-Road.
Furthermore, Estonia is building an international interest group and development institution around the X-Road and at least Nordic countries + UK has been interested. That is another reason to go with the flock, use the momentum and share development costs. In addition, having similar systems in use in neighboring countries, creating cross-border applications and services is a lot easier.
First of all a little rant. I hate to see one-sided one-liners about the uselessness of SOAP and that REST is the answer. Try to get it: REST is not a silver bullet, nor is JSON. Those one-liners are often thrown around by middle-managers and presentation generators, in other words people who I refer to as repeaters. Repeaters don’t know the implementation pain, they just repeat what they have heard someone say at the Whatever Repeaters’ Annual Conference™. Don’t get me wrong, I love REST and JSON and prefer to use those in systems, but nevertheless SOAP has it’s use cases. REST is a fundamentally different paradigm from SOAP.
Use REST when…
To put it briefly, areas that REST works really well for are (taken from http://www.infoq.com/articles/rest-soap-when-to-use-each):
- Limited bandwidth and resources; remember the return structure is really in any format (developer defined). Plus, any browser can be used because the REST approach uses the standard GET, PUT, POST, and DELETE verbs. Again, remember that REST can also use the XMLHttpRequest object that most modern browsers support today, which adds an extra bonus of AJAX.
- Totally stateless operations; if an operation needs to be continued, then REST is not the best approach and SOAP may fit it better. However, if you need stateless CRUD (Create, Read, Update, and Delete) operations, then REST is it.
- Caching situations; if the information can be cached because of the totally stateless operation of the REST approach, this is perfect.
SOAP is for…
That covers a lot of solutions in the above three. So why would I even consider SOAP? Again, SOAP is fairly mature and well-defined and does come with a complete specification. The REST approach is just that, an approach and is wide open for development, so if you have the following then SOAP is a great solution:
- Asynchronous processing and invocation; if your application needs a guaranteed level of reliability and security then SOAP 1.2 offers additional standards to ensure this type of operation. Things like WSRM — WS-Reliable Messaging.
- Formal contracts: if both sides (provider and consumer) have to agree on the exchange format then SOAP 1.2 gives the rigid specifications for this type of interaction.
- Stateful operations: if the application needs contextual information and conversational state management then SOAP 1.2 has the additional specification in the WS* structure to support those things (Security, Transactions, Coordination, etc). Comparatively, the REST approach would make the developers build this custom plumbing.
As shown above, each technology approach has their uses. Some say that you should use REST all of the time unless you are dealing with critical operations like banking transaction. End of rant. That said, we can move on to X-Road and REST/JSON support.
Mythodology over Adapter server
The earlier post (How to build REST support to X-Road — laughing version ) which demonstrated one possible model to get REST/JSON support to X-Road was an educated best guess.
That guess was not too far from the (boldly said) right approach. In architecture drawings shown in PowerPoints and presentations you might see Adapter Server next to Secure Server. What is this adapter server and what can it do? That was one of the tasks for me to find out from my Estonian contacts.
The adapter server of the database modifies queries arriving from X-Road to a form that can be processed by the database’s data server (which is independent of X-Road), and returns the data server’s response in a form suitable for X-Road.
I had a nice 30 minute chat over Skype with X-Road Architect/Area Manager Heiko Vainsalu and found out that Adapter Server is just a drawing, it’s not an implemented part of the X-Road like Secure Server.
It seems that some of miscommunication is coming from the fact that adapter server is considered to something pre-built… while in fact it is just a concept… or other words.. it is a problem of the x-road member :)
Each X-Road member is responsible for implementing Adapter Server to enable for example SAP — SOAP or JSON — SOAP conversions. In other words the plan presented in previous post containing suggestion of bolting Mule ESB to X-Road adapter server was more or less the right answer. The only minor adjustment is that Mule ESB is the Adapter Server. This was confirmed as right answer and recommended approach by Heiko Vainsalu.
About Mule ESB
In fact we can use any ESB solution available on the market. Mule ESB is currently the prefered approach because Mule is vendor-neutral, so different vendor implementations can plug in to it. You are never locked in to a specific vendor when you use Mule. In addition Mule ESB community edition is open source.
Mule ESB is a lightweight Java-based enterprise service bus (ESB) and integration platform that allows developers to connect applications together quickly and easily, enabling them to exchange data. Mule ESB enables easy integration of existing systems, regardless of the different technologies that the applications use, including JMS, Web Services, JDBC, HTTP, and more.
The key advantage of an ESB is that it allows different applications to communicate with each other by acting as a transit system for carrying data between applications within your enterprise or across the Internet. Mule ESB includes powerful capabilities that include:
- Service creation and hosting — expose and host reusable services, using Mule ESB as a lightweight service container
- Service mediation — shield services from message formats and protocols, separate business logic from messaging, and enable location-independent service calls
- Message routing — route, filter, aggregate, and re-sequence messages based on content and rules
- Data transformation — exchange data across varying formats and transport protocols
You can read more about Mule ESB from http://www.mulesoft.org/
Road ahead
I would 1) use the X-Road service management as it is. To do that REST/JSON interfaces need to be defined and added to service catalog with WSDL files. We could encourage all business to define their REST/JSON interfaces with RAML and develop a tool to generate WSDL description files from those. We would also use the security features it has. The concept of secure server is solid.
We need to seek and explore by implementing small tests, are there possibilities 2) to create common features in the adapter server. I’m a bit skeptical since all REST/JSON interfaces are different, there is no standard. Yet it might be possible for example in sectors; Education industry might be able to find something identical in their processes. For example they more or less agree of the standard which defines the learning object.
3) Defining transformation rules in Mule ESB as our Adapter Server. It would be needed that most likely all need to create the mappings for their services. Of course there can be some sharing, since for example all SAP services behave pretty much the same.
4) Some of the transformations or at least best practices and patterns could be shared among the companies. This kind of meta information sharing in often the most valuable asset.
It would most likely help us all in Finland if we 5) decide to use same ESB and share the knowledge. That way not all have to learn the same stuff again and again or make the same mistakes again and again.
6) Sharing is the key here since we all have to have Adapter server and Secure Server. But all that has to be tested and explored.
I wil continue with Mulesoft people to have more clearly defined task list and estimates of needed work to do. Our next scheduled meeting is 19th August and after that we should have a rather conchise plan (technically) how to proceed. Before the Mulesoft meeting, all comments are welcome. I’m ready to meet people and have a chat about this if needed. I’m also approaching CSC to find out whether we can have sandbox version of X-Road for this purpose and what kind of efforts that requires.
Consensus needed
It might be a good idea to organize a meeting around the end of August/early September to discuss the overall plan. In that meeting we should have participants from 5–6 dynamic and respected ICT companies, Ministry of Finance, Ministry of Communication, CSC — IT Center for Science Ltd, Mulesoft Ltd and Estonian Information System Authority. In the meeting, the rough plan would be discussed and if possible accepted. After that it would be implemented. Those willing to be part of building the solution for Extended X-Road Interface support, contact me (jarkko.moilanen [ät] minedu.fi) or leave comment below.
