Web Service Client in Spring Boot

Well, I am a REST api guy. That’s all I have used and developed until one day someone gave me a WSDL & a certificate and asked me to write a SOAP web service client to fetch some information using it from the server. I started with these two things in hand and let’s see what I learnt from that experience!

As I started googling and got terms like service-oriented architecture(SOA), Web Services Description Language(WSDL), Simple Object Access Protocol(SOAP), and Universal Description, Discovery, and Integration (UDDI).

This page well explained me everything!

Introduction to Web Services Technologies: SOA, SOAP, WSDL and UDDI | Web services and the…

A web service is much like a REST Api in terms of functionality. A web service is like a script running on a web server which receives requests/queries in XML format and replies/sends a response to the corresponding request in XML format using SOAP over HTTP/HTTPS protocol.

source: https://www.service-architecture.com/articles/web-services/web_services_explained.html

WSDL, UDDI and SOAP formed the original SOAP Web Services specification.

A SOAP web service is described using a WSDL document. It basically provides a template of the requests and responses that will be flowing over the network to share information between a service provider and a client.

There is also a logically centralized directory of services. The registry provides a central place where developers can publish new services or find existing ones. This service discovery is handled via UDDI. It is a specification for a distributed registry of web services.

Now, to explain how the three things SOAP, UDDI and WSDL is related, I will enlist a few steps in which things work.

1. A service consumer issues one or more queries to the central directory of services to locate a service and determine how to communicate with that service(WSDL is provided by the service registry defining how to communicate with the service).
2. A part of the WSDL provided by the service provider is passed to the service consumer. This tells the service consumer what will be the format of the requests and responses for the service provider.
3. The service consumer uses the WSDL to send a request to the service provider in SOAP format. SOAP at one time stood for Simple Object Access Protocol. Now, the letters in the acronym have no particular meaning.
4. The service provider provides the expected response to the service consumer in XML format.

That’s it! This is all we need to know to prepare a service consumer. Now we will see —

1. How to construct a request message
2. How to send the message to the service provider using SSL certificates
3. How to parse and use the response

Before we start with our first step, we will discuss two important points here in detail — SOAP and WSDL

Learning XML

SOAP and WSDLs are all about XMLs and their schema definition: XSDs. I am mentioning two links which I found very useful for the same —

XML Schema Tutorial - Defining Elements and Attributes

Tutorial: Namespaces | XML Schema Tutorial | Webucator

I am covering an important concepts here — Namespaces. We know XML format is used to transfer data between to independent systems that may not know anything about each other(Service provider and client in our case). Provider will specify a format(called “Schema”) in which it will accept the data that will be exchanged. This schema(.xsd files) is shared with clients and the provider will validate the message(.xml files) against this schema before accepting it for further downstream processes and sending the response.

So, let’s suppose we have 3 schema for Address, Owner and Tenant. All the XML requests that provider receives contains an object/document of one of these schema as a query. Thus we need to create a Address/Owner/Tenant kind of XML object/file that should now strictly follow these schema.

Namespaces

Now, we have 3 schema for CommonTypes, Owner and Tenant(We have not seen how a schema looks yet). We can have all three schema together in one XSD! However, usually the schema in broken into several files so as to create re-usable definitions that can be used across several projects. They make definitions easier to read and version as they break down the schema into smaller units that are simpler to manage.

We will pull these into a single “Main” schema where we will define our elements.

Now, if different teams start working on schema, there could be a case where we have “Name” attribute in the Owner schema as well as Tenant schema. Then if we pull them together in the Main schema, there will be a conflict! Hence, for every schema file we will put an attribute targetNamespace into the schema element in the XSD file. Now the Name in Owner’s schema will have a sully qualified name like — Owner:name and in the Tenant’s schema will be like Tenant:name. You see, no conflicts now!

<?xml version="1.0" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           targetNamespace="Owner">
    ...
</xs:schema>
//  The value of targetNamespace is simply a unique identifier, typically a company may use their URL followed by something descriptive to qualify it. Yes, that why you see URLs! But, it's not mandatory to have URLs here!  In principle the namespace has no meaning, but some companies have used the URL where the schema is stored as the targetNamespace, and so some XML parsers will use this as a hint path for the schema.

Placing the targetNamespace attribute at the top of your XSD schema means that all elements defined in it are part of this namespace. So in our example each of the 3 schema files could have a distinct targetNamespace value.

Now we can arrive at a definition of what Namespaces are. To group elements are attributes that relate to each other and to keep XML tags globally unique, namespaces are defined.

Sample schema:

Namespaces are defined using the xmlns attribute that is present in the root tag like this —

xmlns:prefix=namespace

where namespace COULD BE a unique URI.

In here — <xs:schema xmlns:xs=“http://www.w3.org/2001/XMLSchema” xmlns=“http://www.example.com/Owner” targetNamespace=“http://www.example.com/Owner”>

1. The xmlns:xs(discussed above) attribute indicates that global elements, attributes, types, and groups in the XML Schema namespace can be used in this document, but they must be qualified with the xs: prefix. It indicates that the elements and data types used in the schema come from the “http://www.w3.org/2001/XMLSchema” namespace. It also specifies that the elements and data types that come from the “http://www.w3.org/2001/XMLSchema” namespace should be prefixed with xs. Eg: If we want to declare type of an element as string, this would we wrong to do — <element name=“ApartmentNo” type=“string”/> and we should do this <xs:element name=“ApartmentNo” type=“xs:string”/>. In the later case, XML parser knows the namespace from where “element” and “string” keyword is coming from.
2. We already know what targetNamespace it. It uniquely defines the data types that we define in our schema file. In this example, “ApartmentNo” is from the Owner.xsd file belongs to namespace : owner. Then we have a third declaration xmlns attribute. This indicates that global elements, attributes, types, and groups in the Owner namespace can be used in this document without a prefix (i.e, unqualified). The xmlns attribute, thus, is defining a default namespace here. The above CommonTypes.xsd schema would be invalid if we remove the xmlns=“http://www.example.com/Owner” because ApartmentInfo and Address element declarations have child elements that refer to other elements declared somewhere else within the same schema. We can only reference elements that are declared globally in namespaces used in the document. The Owner namespace in Owner.xsd schema then is the default namespace(because we have kept targetNamespace and xmlns as the same). There can only be one default namespace in an instance document. Suppose, if we make this as our default namespace — xmlns=“http://www.w3.org/2001/XMLSchema”, then we need not to use xs: prefix anywhere with element or complexType or string etc!
3. The other way(apart from default namespace way using xmlns) we can declare and reference elements in the same schema by putting a prefix for it in there and then further references to the declared elements should be done via fully qualified names. The following version of CommonTypes.xsd is same as the above one but we do not have any default namespace here —

Now, there are a few things to be noted down in the sample schema I have provided —

1. See how we are declaring and referencing ApartmentInfo type in the same schema file
2. See how we are importing and using CommonTypes.xsd schema in Tenant and Owner schema. We are doing two things here — first we are importing it via an import tag and then we are declaring a prefix in the root schema tag that we will used to refer to elements of CommonType namespace.

Why I went through this detail? Because multiple schema can be sitting next to each other in the <types> </types> tag within the WSDL. They can import each other as we are doing it in Main.xsd and they will refer to entities defined in each other. Example:

<types>
    <xs:schema {commonTypes...}></xs:schema>
    <xs:schema {Owner...}></xs:schema>
    <xs:schema {Tenant...}></xs:schema>
</types>

SOAP

I assume everyone knows what web services are — either SOAP or REST. XML or JSON is widely used to exchange data between two applications. But there are no standard specifications on use of XML across all programming languages for data exchange. That is where SOAP comes in. SOAP was designed to work with XML over HTTP and have some sort of specification which could be used across all applications.

What is the difference between SOAP and HTTP? We can serve any content over HTTP like HTML, Images, sound, videos etc. SOAP is an XML-based encoding of messages that are typically sent over HTTP, i.e., SOAP messages are placed as the HTTP payload. Just like HTTP sits on top of TCP/IP, SOAP sits on top of HTTP. Although there is the overhead of HTTP, it has the advantage that it is a protocol that is open to firewalls, well-understood and widely-supported. Thus, web services can be accessed and exposed via technology already in-place.

As we will see further, a SOAP request has both layers, HTTP headers at the top followed by the SOAP message.

Author in the below stackoverflow answer has given a very intuitive explanation of the same —

Why does a SOAP message have to be sent over HTTP?

SOAP Message Format

The SOAP specification defines a SOAP message and this is what, is travelling over the network. The SOAP message consists of a —

1. SOAP Envelope element. It identifies the XML document as a SOAP message. This is the root element in the SOAP message.
2. Then there is a header element followed by a body element.

<?xml version='1.0' ?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/SOAP-envelope" env:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
    <env:Header> 
    </env:Header>
    <env:Body>
        <env:Fault>
        </env:Fault>
    </env:Body>
</env:Envelope>

All of it is well explained in the below link. If you know about XML markup, it would not be difficult to understand the things like namespaces and encoding attributes in the above template!

SOAP Message Structure

Header: The header contains metadata, It can contain documentation about the particular soap message and any other elements a developer might like to include. Some common uses of the soap header, include API keys, or log-in credentials to validate requests, or timestamps in responses indicating when data was created, or how long it’s good for.

Body: contains the application-defined XML data being exchanged in the SOAP message. In our use case, the format of the body will be defined by the WSDL and we will create our body according to it. I will later show you how!

Fault: If the request fails for any reason, the server sends back a response containing a Fault element which is a child element of the body. The body typically contains either valid response data or a fault, but not both. In SOAP, the Fault element has a number of child elements with fixed names, including Code, Reason, Role, and Detail.

This is how a SOAP over HTTP request look like —

POST /InStock HTTP/1.1
Host: www.example.org
Content-Type: application/soap+xml; charset=utf-8
Content-Length: nnn

<?xml version="1.0"?>

<soap:Envelope
xmlns:soap="http://www.w3.org/2003/05/soap-envelope/"
soap:encodingStyle="http://www.w3.org/2003/05/soap-encoding">

<soap:Body xmlns:m="http://www.example.org/stock">
  <m:GetStockPrice>
    <m:StockName>IBM</m:StockName>
  </m:GetStockPrice>
</soap:Body>

</soap:Envelope>
-- FROM https://www.w3schools.com/xml/xml_soap.asp

Communication Model

All communication by SOAP is done via the HTTP protocol like this —

1. The client would format the information regarding the procedure call and any arguments into a SOAP message and sends it to the server as part of an HTTP request. This process of encapsulating the data into a SOAP message was known as Marshalling.
2. The server would then unwrap the message sent by the client, see what the client requested for and then send the appropriate response back to the client as a SOAP message. The practice of unwrapping a request sent by the client is known as Unmarshalling.

WSDL

As said earlier, a WSDL document is an XML document that is used to describe a web service. This description is required, so that client applications are able to understand what the web service actually does and how to use it.

• The WSDL file contains the location of the web service.
• The methods which are exposed by the web service.
• The message formats that is exchanged between the service provider and client.

This is what it looks like —

<definitions>
   <!-- abstract definitions -->
   <types>
      definition of types........
   </types>

   <message>
      definition of a message....
   </message>

   <portType>
      <operation>
         definition of a operation.......  
      </operation>
   </portType>
   <!-- concrete definitions -->
   <binding>
      definition of a binding....
   </binding>

   <service>
      definition of a service....
   </service>
</definitions>

Let’s talk about these terms —

<types>

The data types to be used in the messages are in the form of XML schemas, i.e., in <types> </types> section we need to put an XSD of the datatypes we will be using in the messages. It could be a simple datatype or a complex data type as you can see in the following example.

I have tried to create a complex example for better understanding. In above snippet, I have two objects — productRequest and productResponse Object. Observe how complicated it looks but it is similar to how you create classes in and OOPs based programming language.

<message>

It is used to define the messages that are exchanged between the service provider and the client. It is a collection of the data types (that we defined above) that make a single message (whether request or response). Example:

<message name="ProductRequest">
    <part name="parameters" type="pdt:getProductReq"/>
</message>
<message name="ProductResponse">
    <part name="parameters" type="pdt:getProductReq"/>
</message>

Each message has a <part> element which is used to describe the parameter used by the request and response message. In our case, we have encapsulated our parts in the dataType object. However, the following is similar to above way —

<message name="ProductRequest">
    <part name="productCode" type="xsd:String"/>
    <part name="categoryCode" type="xsd:String"/>
    <part name="transactionId" type="xsd:String"/>
</message>
<message name="ProductResponse">
    <part name="productCode" type="xsd:String"/>
    <part name="categoryCode" type="xsd:String"/>
    <part name="productName" type="xsd:String"/>
    <part name="price" type="xsd:decimal"/>
</message>

<portType>

This tag is used to group messages into one operation. For example, the above shows the message template of Request of a particular query. They are part of an operation getLatLong(). Client is supposed to deal with this operation and he will sent a request message like “LatLonListSubgridRequest” and get a response like “LatLonListSubgridResponse”. We group these two in an operation.

<portType name="Pdt_Txn_Port_Type">
    <operation name="ProductTxnDetails">
        <input message="ProductRequest"/>
        <output message="ProductRespose"/>
    </operation>
</portType>

<binding>

This tag binds a particular operation to a portType. In essence, it provides the actual details of how the operation will be transmitted over the network. Port types that we defined above are just like interfaces. We did not mention how it will be transmitted — via HTTP GET, HTTP POST or SOAP. So if a client application needs to use a web service they need to use the binding information to ensure that they can connect to the interface provided by that web service. In our case, we will use SOAP protocol and corresponding binding will be <soap:binding> which will be transported on the top of HTTP.

<binding name="PdtTxnBinder" type="portType">
    <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="ProductTxnDetails">
        <soap:operation soapAction="PdtTxnBinder_ProductTxnDetails" style="document"/>
        <input>
            <soap:body use="literal" parts="parameters"/>
        </input>
        <output>
            <soap:body use="literal" parts="parameters"/>
        </output>
    </operation>
</binding>

<soap:binding>: Tells that the portType will be made available via SOAP. Style attribute tells the overall SOAP message format and the transport attribute indicates how the messages will be transported, i.e., over HTTP or SMTP etc.

<soap:operation>: It is binding specific operation to specific SOAP implementation.

<soap:body>: It is used to specify the details of input and output message. In the above example it is specifying the encoding style and the namespace associated with the service.

It is explained in the oracle site with a great diagram. I am putting it here

Source: https://download.oracle.com/otn_hosted_doc/jdeveloper/1012/web_services/ws_wsdlstructure.html

<service>

It defines the ports supported by the web service. So if there are multiple supported protocols, there would be multiple port elements. It is wise to say that a service element is, thus, a collection of ports! Via the <service> element, the client knows where to access the web service and the ports that should be used. The ports refers to bindings & operations which ultimately tells what messages are supported over the network.

We will focus on <port> .. </port> now. It defines a SINGLE endpoint for a particular binding.

From our example:

<service name="TxnLedger">
    <port name="TxnLedger_Port" binding="PdtTxnBinder">
        <soap:address location="https://example.com/process.php"/>
    </port>
</service>

Also look at WSDLs of these publicly available services where we have multiple portTypes and bindings —

http://www.thomas-bayer.com/axis2/services/BLZService?wsdl

http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL

Available Technology Stacks for Web Services

Now we know almost everything we need to know about SOAP, WSDL and the Service Oriented Architecture(SOA). Yes, I have not talked about creating web services. In this write up, I am only concerned about creating a client given a service is already live!

Coming back — we got some hints about what a request should look like and how the message is transported to and interpreted by — the server. Great! Now I will show how to create a client service that will help us to connect to our web service that will then serve our queries!

I will create my client using Spring Boot and show the libraries and packages available to do the same.

Available Technologies In Picture

There are many technologies out there for web services development. It was difficult for me to understand which one to use in the beginning. So, I’m putting a short discussion on what I found.

1. Apache Axis
2. Apache CFX
3. JAX-WS
4. Spring WS

Apache CFX > Apache Axis2: Discussion is available here —

Difference between Apache CXF and Axis

Spring WS: Has least number of WS-* Specifications(which are standard specifications developed for web services that frameworks/implementations should follow) supported (does not fully compliant with JAX-WS — standard one provided by Sun) and is still evolving in comparison to CFX.

Well, all I see is CFX and JAX-WS as competing players in this market. CFX has a simple integration with Spring boot and a lot of other features that it provides at one place. CXF also supports usage of both JAX-WS and JAX-RS annotations together without any trouble.

You may read more about the difference between CFX and JAX WS on your own. Right now, I want to focus on getting things up and running! I will use JAX-WS.

WSDL to Java Classes

There are two ways you can construct SOAP messages — First, you can generate the SOAP XML string by carefully observing a WSDL and finalizing the structure of message. But, there is no point in making things difficult when we have an easier option #2. The easier way is to create Java classes and convert them to SOAP XML messages. Java objects to XML binding is very simple and intuitive. Java provides an in-built function to convert WSDLs to Java classes. You can see a wsimport tool in the ${JAVA_HOME}/bin directory. This tool maps the WSDL to Java Class. We will do it for our WSDL also. But, let us first see how the mapping looks like. This page well explains the mappings —

Default mapping from WSDL to Java

I will summarize the important ones that you will see —

namespace → package name

This is how packages will be created in Java based in WSDL targetNamespace. If you remember from our earlier discussion, we have target namespace at two places in the WSDL. First at the WSDL root tag itself and then within the <types>. Within the <types> we can have multiple schemas each with different namespaces. Hence, for every schema, we will have a java package created. Classes related to <portType>, <binding>, <service> will be created within the package created corresponding to the root targetNamespace.

<portType> → Interface
<operation> → Interface Methods
<binding> → Interface Method Annotations

<portType> → Interface

<operation> → Interface Methods

<binding> → Interface Method Annotations

For every <portType>, we will have interfaces. <portType> constitutes multiple <operation>. For every operation, we will have a method defined in the Java interface.

Operation further constitutes input and output messages. Input message will be argument to the interface method and the output message will be the return type of the interface method.

Messages in turn, contains Parts. These parts refer to one of the simple/complex data type that is declared under the <types> tag. Hence, the arguments and return type of the interface methods will be Java classes that were created corresponding to the schema.

This is how classes and elements are generated for them —

<types> → Java Beans

As I said earlier, for every schema file, we will have a package that will contain the Java beans generated corresponding to the data types. We will also have a ObjectFactory.java class in these packages. This class contains factory methods to create instances of schema derived classes programmatically.

This is how your interface will look like. Generate one for your WSDL see it property values closely —

@WebService(name = "<portType> tag Name attribute", 
targetNamespace = "targetNamespace of the root tag - <definitions>")
@XmlSeeAlso({Comma separated list of ObjectFactory.java classes created for each schema defined under the <type> tag})
public interface portTypeName {
    @WebMethod(action = "Binding name corresponding to this portType + underscode + operation name")
    @WebResult(name="", targetNamespace="")
    @RequestWrapper(localName="", targetNamespace="", className="")
    @ResponseWrapper(localName="", targetNamespace="",className="")
public MessageOutputTypeClassName operationName(
         @WebParam(name = "", targetNamespace = "") 
               MessageInputTypeClassName variableName);
}

Now the last one — <service> → Class Xyz extends Service {}

wsimport will create a class, in the package corresponding to targetNamespace of root <definitions>, extending the service class of JAX-WS. This will be the class you will use to interact with the provider as a client.

Creating a WS Client

We will use this public available calculator web service for demo:
http://www.dneonline.com/calculator.asmx?wsdl

I will put the codes in my Github repository here. I will refer to the code at each step via a commit id to track the development step by step.

Step 1: Create a spring boot project

We create a demo project using spring boot initializr. Here is how my repository looks in the beginning — step-1

Step 2: Create classes from WSDL

As said earlier, we have wsimport tool provided with JDK to do the same. We will use it to create the Java Classes. More about it is here.

wsimport -keep -s <source_classes_output_path> -verbose <path_to_wsdl>

Note: you can provide URL directly in the <path_To_wsdl>. However, there might be cases when the wsdl HTTPS link is provided by untrusted parties. The certificate at the URL might not be recognized by the Java tool and it will throw exception. So, copy-paste the contents of the WSDL from your browser to a plain-text file(In notepad, use encoding as UTF-8 while saving the file) and pass the file path instead of URL.

Look at the classes generated — we have request and response classes for Add, Subtract, Multiply & Divide. We also have an ObjectFactory.java class as discussed before.

We have a CalculatorSoap.java interface that was created with <portType>, <operation>, <binding>. We have four functions for four operations in this interface.

From the <service> tag, we also have a Calculator class that extends the JAX-WS service class.

We put the created classes in our src/main/java/ directory. Step-2 code.

Step 3: Make the wsdl location relative to the project

As you can see in the Calculator.java class, the WSDL location is absolute and we cannot have this. Later if we run the file in a different machine, it won’t work. So we make a code change to add it to our classpath. You just have to put the wsdl file in the src/main/resources/ directory.

Step 4: Prepare a client class

In this simple case, the methods in interface for portType are very simple. For Eg: add(int a, int b) returns the added value that comes from the service. we can simply call this method from the port and get it. You can see the code here.

Your client service is now working. But, wait! This case was very simple and it might be very complicated in your case. Here are a few tips.

Tip 1: The operation method in the interface accepts and returns class objects

We have to pass two integers in the operations defined in the interface and the response was a simple value. There can be scenarios where the request args for these methods are java objects with nested and hierarical classes. In such cases, only thing that is to be ensured is that while you create the object, provide the values for all the variables marked as required in the classes.

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "ProcessType", propOrder = {
    "category",
    "subCategory",
    "productCode",
    "releaseVersion"
})
public class ProcessType {
    @XmlElement(required = true)
    protected String category;
    @XmlElement(required = true)
    protected String subCategory;
    protected String productCode;
    protected String releaseVersion;
    // getters and setters
}

When it comes to reponse, note here that even when something goes wrong, an exception object is captured in the java classes in your WSDL. Check that in your program as well.

Case 2: Logging

You can see the XML string created for your request and response Object by passing the request object and its class to this method—

public <T> String convertToXml(Object source, Class<T> type) throws IOException {
    String result;
    StringWriter sw = new StringWriter();
    try {
        JAXBContext jaxBContext = JAXBContext.newInstance(type);
        Marshaller m = jaxBContext.createMarshaller();
        m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, false);
        m.marshal(new JAXBElement<T>(QNAME, type, (T)source), sw);
        result = sw.toString();
    } catch (JAXBException e) {
        throw new RuntimeException(e);
    } finally {
        sw.close();
        sw.flush();
        sw = null;
    }
    return result;
 }

If you want to log the SOAP message along with the header, you can follow this —

Tracing XML request/responses with JAX-WS

Put the above statements before calling the operation from the web service.

Case 3: Configure service endpoint at runtime

In enterprises, you will get two WSDLs — one for development and one for production environment. Both WSDLs will be the same except for the URL end point and certificates/credentials(if they are there)

To configure it, you need not to change the client code much. Just before you call the service method, you need to the following —

BindingProvider bp = (BindingProvider)port;            bp.getRequestContext().put(BindingProvider.ENDPOINT_ADDRESS_PROPERTY, "http://www.dneonline.com/calculator.asmx");

If there are username/passwords required for authentication, you can provide them here only —

bp.getRequestContext().put(BindingProvider.USERNAME_PROPERTY, "username");
bp.getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, "password");

Case 4: Communication encryption using SSL certificates

In case you have signed certificates — keystores or truststore or both, you can configure them in your client in a two step process —

First, create a SSLSocketFactory —

Second, provide it in the binding provider —

bp.getRequestContext().put("com.sun.xml.internal.ws.transport.https.client.SSLSocketFactory", sslSocketFactory);

And, that’s all folks! Hope this helps.

References

WSDL Tutorial: Web Services Description Language with Example

WSDL Elements

SOAP Web Services - javatpoint

SOAP