New logic app mapper for VS code
Data mappers, or message translator as the pattern also is known, is an integration pattern enabling systems using different data formats to communicate with each other using messaging. Low code tools for creating mappers is appealing as domain experts with deep knowledge of processes and systems could create integration without much coding skills. Such tools can also be used by architects to hand of specifications that is more than just dead documentation. Graphical low-code tools for developing data mappers has therefore been around in integration platforms going on decades.
For customers of Azure Integration Services this has until now been limited to the “BizTalk Mapper”. While it is possible to run maps developed in the BizTalk mapper in Logic apps this require BizTalk installed and older versions of Visual Studio. In addition it for this to work, users needs to enable the premium feature in logic apps for “Integration accounts”.
Microsoft recently announced a new mapper for logic app in VS Code. This allow developers to create maps in Visual studio code and use them in standard workflows. How does it work and is it worth trying out in your project (in my opinion)?
How does the mapper work?
The mapper allows users to select source and target schemas, which could be either JSON schema or XSD. The current version does not support flat file formats. This is a useful feature in the BizTalk mapper when working with legacy systems. We hope that this feature will be added to the roadmap for this tool as well.
The visual editor itself is quite nice to work with. It is implemented in HTML using React. Unlike the old BizTalk Mapper, the source code for the UI is open source and available on GitHub. The code-behind is in a compact and easy-to-understand YAML format.
The YAML is converted to XSLT 3.0, and it’s actually this file that is executed when running the map in a Logic App workflow. The produced XSLT is clean and can be used by frameworks and tools with XSLT 3.0 support. I have tested running the XSLT produced by the map described later in this post with XmlSpy.
There is a set of built-in functions to work with, ranging from string manipulation to logic operations useful for conditionally creating elements based on conditions. There is also an option to run custom XSLT. However, compared to the old BizTalk Mapper, some important functions are missing to handle more than the most simple transformations. The VS Code mapper lacks built-in functions for flattening or restructuring through intermediate data structures such as tables. There is no option for running custom code or calling functions in reference DLLs. This might be intentional to keep the XSLT compatible with other tools. Developers are better off doing this kind of work in Azure Functions.
Plenty of preview bugs
For this blog post I tried implementing a simple scenario converting from json-orders to XML-orders. The code can be found in this GitHub repo. I came across several bugs while implementing this scenario. Creating loops breaks the entire map once the editor is closed and open again.
Also the index function, used for retuning the current index in a loop, does not seem to work as expected, and I get a runtime error when I try to test the map. It appears that the underlying XSLT code is missing a variable declaration for the index variable. I submitted these issues to the project on github. Status on the issues can be found here and the issue with the loop here. Hopefully these issues get resolved quickly. The tool is not very usable for json-to-xml scenarios at the time of writing.
Overall
XSLT itself is no longer as relevant due to the decline of XML and the popularity of new flexible APIs such as OData and GraphQL. Also, performance has always been an issue with XSLT, and this is becoming more pressing as the amounts of data and the number of messages in our integration keep growing. However, I believe there will be huge amounts of integrations where the processes and tools related to XSD and XSLT will be relevant in the years to come within enterprise integration. Low code is undoubtedly relevant for organizations due to the cost of developing in languages such as C#.
What I find appealing about creating integration using Logic Apps and the new data mapper is that developers are guided through a process that requires them to build proper schemas, even for JSON data. This, in turn, makes validation and general data quality in the data flow much easier.
The process of mapping between data models is at its core an activity within the realm of architects or domain experts. I love the fact that the VS Code mapper can empower these roles, which may have limited time and/or coding skills, to create artifacts that will work effectively and serve as important pieces of documentation.
Middleware offerings for data mapping, in general, have the problem of lock-in and can be difficult to migrate to other platforms or systems based on tailor-made software. This is especially true for BizTalk Maps, and the significant investment in creating complex mappings is one of the reasons why many organizations struggle to migrate their integrations to the cloud today.
Microsoft has taken huge steps in the direction of open source. Logic Apps Standard can run as containers, and the mapper itself is open source. The underlying XSLT seems to be fully portable, and as mentioned earlier, I managed to run my simple use case in XmlSpy.
I am definitely positive and welcome this product, as I believe XLST still has an important role to play in the years to come, and that this area of systems development is suited for low-code/no-code solutions.
The mapper itself is far from being a finished product, with several bugs and glitches. Hopefully, this project will get attention and become a robust tool.
