Overview

Spring Boot & Apache CXF – XML validation and custom SOAP faults

No Comments

What about XML? Can’t we validate our data with XML easily? Just take the XML schema and … erm. What about the reaction to the validation´s outcome? Most of the time, we have to tailor this response and build a custom SOAP fault. But how does this work with Spring Boot and Apache CXF?

Spring Boot & Apache CXF – Tutorial

Part 1: Spring Boot & Apache CXF – How to SOAP in 2016
Part 2: Spring Boot & Apache CXF – Testing SOAP web services
Part 3: Spring Boot & Apache CXF – XML validation and custom SOAP faults
Part 4: Spring Boot & Apache CXF – Logging & Monitoring with Logback, Elasticsearch, Logstash & Kibana

In the preceding parts we learned how a SOAP web service is configured and tested in detail with Spring Boot and Apache CXF. Now we want to look at a more particular case. There are some big web service specifications out there (look e.g. at the BiPro specs) that require our SOAP endpoint to react with a 100% XML schema compliant response in every situation – even when somebody sends bad XML requests which generate errors inside Apache CXF´s XML processing.

Given a request that could be processed successfully, our response will always be 100% XML schema compliant. We only have to follow this blog series first part´s guidelines and generate the Java classes from our WSDL and XSDs. As usual, there is a new GitHub project waiting in our tutorial repository – if you want to give it a try. 🙂

As a starting point we´ll use the preceding part´s project for now and fire up the SimpleBootCxfApplication with the help of a “Run as…” . Once our SOAP endpoint is up and running (check http://localhost:8080/soap-api/WeatherSoapService_1.0), we send a valid request against it with SoapUI:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:gen="http://www.codecentric.de/namespace/weatherservice/general">
   <soapenv:Header/>
   <soapenv:Body>
      <gen:GetCityForecastByZIP>
         <gen:ForecastRequest>
            <gen:ZIP>99998</gen:ZIP>
            <gen:flagcolor>bluewhite</gen:flagcolor>
            <gen:productName>ForecastProfessional</gen:productName>
            <gen:ForecastCustomer>
            <gen:Age>30</gen:Age>
            <gen:Contribution>5000</gen:Contribution>
            <gen:MethodOfPayment>Paypal</gen:MethodOfPayment>
            </gen:ForecastCustomer>
         </gen:ForecastRequest>
      </gen:GetCityForecastByZIP>
   </soapenv:Body>
</soapenv:Envelope>

Also, the reply should look like a valid SOAP response:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <GetCityForecastByZIPResponse xmlns="http://www.codecentric.de/namespace/weatherservice/general" xmlns:ns2="http://www.codecentric.de/namespace/weatherservice/datatypes" xmlns:xmime="http://www.w3.org/2005/05/xmlmime" xmlns:ns4="http://www.codecentric.de/namespace/weatherservice/exception">
         <GetCityForecastByZIPResult>
            <Success>true</Success>
            <State>Deutschland</State>
            <City>Weimar</City>
            <WeatherStationCity>Weimar</WeatherStationCity>
            <ForecastResult>
               <ns2:Forecast>
                  <ns2:Date>2016-06-06T17:17:06.903+02:00</ns2:Date>
                  <ns2:WeatherID>0</ns2:WeatherID>
                  <ns2:Desciption>weather forecast Weimar</ns2:Desciption>
                  <ns2:Temperatures>
                     <ns2:MorningLow></ns2:MorningLow>
                     <ns2:DaytimeHigh>90°</ns2:DaytimeHigh>
                  </ns2:Temperatures>
                  <ns2:ProbabilityOfPrecipiation>
                     <ns2:Nighttime>5000%</ns2:Nighttime>
                     <ns2:Daytime>22%</ns2:Daytime>
                  </ns2:ProbabilityOfPrecipiation>
               </ns2:Forecast>
            </ForecastResult>
         </GetCityForecastByZIPResult>
      </GetCityForecastByZIPResponse>
   </soap:Body>
</soap:Envelope>

Standard SOAP faults

Approaching the topic for the first time, one could google something like “configure XML schema validation Apache CXF” or the like. The results are somewhat misleading. For example, look at the Apache CXF FAQ. You´ll find thousands of different variants to activate XML schema validation in Apache CXF. And even worse, the examples provided nearly all show up with Spring XML configuration, which we left behind already. But the configuration of the XML schema validation isn´t our real problem. And funnily enough, XML schema validation is already activated in our setup with Spring Boot and CXF. Just fire a non-valid XML request against our endpoint (which we´ll do in a minute). Instead we should shift our focus to the reaction onto a validation error. All the web service specifications define not if, but the terms how to react.

In case of an error, Apache CXF reacts in form of a standard SOAP fault. We´ll try that out now. This time we´ll send a request against our endpoint that doesn´t comply to our XML schema. Actually our SOAP request´s root element has to be spelled as GetCityForecastByZIP according to the weather-general.xsd, which is imported into our WSDL. But because we want to provoke an error, we´ll change the root tag to GetCityForecastByZIPfoo and send it against our endpoint:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:gen="http://www.codecentric.de/namespace/weatherservice/general">
   <soapenv:Header/>
   <soapenv:Body>
      <gen:GetCityForecastByZIPfoo>
         <gen:ZIP>99425</gen:ZIP>
      </gen:GetCityForecastByZIPfoo>
   </soapenv:Body>
</soapenv:Envelope>

Our (still running) endpoint should react with a SOAP response like this:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <soap:Fault>
         <faultcode>soap:Client</faultcode>
         <faultstring>Unexpected wrapper element {http://www.codecentric.de/namespace/weatherservice/general}GetCityForecastByZIPfoo found.   Expected {http://www.codecentric.de/namespace/weatherservice/general}GetCityForecastByZIP.</faultstring>
      </soap:Fault>
   </soap:Body>
</soap:Envelope>

Now our web service specification defines its own type of exception in case of an error. It´s called WeatherException, which is also defined inside the mentioned weather-exception.xsd. This exception is appended to the SOAP operations with the wsdl:fault tag inside our WSDL. It defines the following structure:

<s:element name="WeatherException">
    <s:complexType>
        <s:sequence>
            <s:element name="Uuid" type="s:string"/>
            <s:element name="timestamp" type="s:dateTime"/>
            <s:element name="businessErrorId" type="s:string"/>
            <s:element name="bigBusinessErrorCausingMoneyLoss" type="s:boolean"/>
            <s:element name="exceptionDetails" type="s:string"/>
        </s:sequence>
    </s:complexType>
</s:element>

Our specification states that the element WeatherException and its children should be put beneath the soap:Fault element, which resides inside the detail tag. Such standards are well known from “enterprise WSDLs”. So we´ll have to implement this requirement to provide a specification-compliant SOAP endpoint.

Not XML schema compliant vs. invalid XML

In case of an error, our WeatherException should be returned inside the soap:Fault/detail – whatever the error might be. So our implementation should be capable of handling not only the errors based on non schema compliant XML, but also if somebody sends XML that is per se completely incorrect. Here are some example requests with a broken XML header – starting with a missing right angle bracket:

<?xml version="1.0" encoding="UTF-8"?

…a non-concluded tag somewhere inside the document:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:gen="http://www.codecentric.de/namespace/weatherservice/general">
   <soapenv:Header/>
   <soapenv:Body>
      notRelevantHere />
   </soapenv:Body>
</soapenv:Envelope>

…a broken SOAP header:

<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:gen="http://www.codecentric.de/namespace/weatherservice/general">
   <soapenv:Header/
   <soapenv:Body>
   ...

and many more.

Looking at these examples, the requirement to handle such broken requests makes a lot of sense. Especially if we follow the ideas from part 1 of this tutorial. Because then, we rely on 100% XML schema compliant messages – which also implies that those messages contain correct XML. Our framework would otherwise react with cryptic fault messages if the requests aren´t valid.

Apache CXF interceptor chains

So how do we approach this problem? As always many roads lead to Rome. In the following I´ll show a way that worked quite well and has been approved in my current projects. Let´s first have a look at Apache CXFs architecture. The docs show that CXF´s processing relies on interceptors, which are invoked sequentially, one at a time. They are arranged like pearls on a string and are organized in phases. There´s an incoming interceptor chain, where at the end our web service implementation is invoked, as well as an outgoing interceptor chain, which handles the response processing. A image says more than a thousand words, so let´s look at this one:

apache_cxf_interceptor

The incoming web service calls always run through these chains. If an error occurs inside an incoming chain interceptor, the outgoing interceptor chain is provided with the error information and run through in opposite direction. So if for example an error occurs in the phase READ, the succeeding incoming interceptor chain´s phases aren´t invoked any more. By the way: Apache CXF provides an extra outgoing fault interceptor chain specifically for handling every error. And luckily enough, we can hook our own custom interceptors into these chains, so that we are able to react on every event that might appear.

This knowledge is quite useful in our case. If an request contains incorrect XML, the incoming chain stops in the UNMARSHAL phase at the latest and the outgoing fault interceptor chain is called. So we just have to implement an Interceptor who witnesses as many errors as possible and enables us to react on them. The eye-catching phase here is org.apache.cxf.phase.Phase.PRE_STREAM. With that phase we are as far ahead in the chain as possible to not miss any error. We have to derive our Interceptor from org.apache.cxf.binding.soap.interceptor.AbstractSoapInterceptor and override the method void handleMessage(T message) throws Fault. Additionally we provide the phase inside the constructor while calling the super() method:

public class CustomSoapFaultInterceptor extends AbstractSoapInterceptor {
 
    private static final SoapFrameworkLogger LOG = SoapFrameworkLogger.getLogger(CustomSoapFaultInterceptor.class);
 
    public CustomSoapFaultInterceptor() {
        super(Phase.PRE_STREAM);
    }
 
    @Override
    public void handleMessage(SoapMessage soapMessage) throws Fault {
        Fault fault = (Fault) soapMessage.getContent(Exception.class);
        Throwable faultCause = fault.getCause();
        String faultMessage = fault.getMessage();
 
        if (containsFaultIndicatingNotSchemeCompliantXml(faultCause, faultMessage)) {  
            WeatherSoapFaultHelper.buildWeatherFaultAndSet2SoapMessage(soapMessage, FaultConst.SCHEME_VALIDATION_ERROR);
        }
        else if (containsFaultIndicatingSyntacticallyIncorrectXml(faultCause)) {
            WeatherSoapFaultHelper.buildWeatherFaultAndSet2SoapMessage(soapMessage, FaultConst.SYNTACTICALLY_INCORRECT_XML_ERROR);          
        }
    }
 
    ...

Detecting XML validation errors

First of all we extract the faultCause and faultMessage inside the overridden method handleMessage(SoapMessage soapMessage). Looking at the latter you could find the exact same inside the standard SOAP fault tag faultstring. Based on those two variables, we are able to detect which error occured.

Sadly the CXF API does not provide any help and we have to implement the methods containsFaultIndicatingNotSchemeCompliantXml() and containsFaultIndicatingSyntacticallyIncorrectXml() ourselves. To figure out how Apache CXF reacts to non schema compliant or incorrect XML, one could think about many test cases and send each of them against our SOAP endpoint. This could get quite sophisticated and cumbersome. Luckily there is already a bunch of test cases inside our example project that we could use. As we try every one of them out, some patterns emerge that we could use to build our detection methods:

1. non XML-schema-compliant

The faultCause contains a javax.xml.bind.UnmarshalException if the request message does contain non schema compliant XML. Besides we check if there´s a missing closing tag. In this case we have to check whether the faultMessage contains an “Unexpected wrapper element”:

private boolean containsFaultIndicatingNotSchemeCompliantXml(Throwable faultCause, String faultMessage) {
    if(faultCause instanceof UnmarshalException
        || isNotNull(faultMessage) && faultMessage.contains("Unexpected wrapper element")) {
        return true;
    }
    return false;
}

2. generally incorrect XML

There are three kinds of errors indicating that the request message contains incorrect XML itself. Either the faultCause contains a com.ctc.wstx.exc.WstxException, the wrapped cause is a com.ctc.wstx.exc.WstxUnexpectedCharException or the faultCause contains an IllegalArgumentException:

private boolean containsFaultIndicatingSyntacticallyIncorrectXml(Throwable faultCause) {
    if(faultCause instanceof WstxException
        // If Xml-Header is invalid, there is a wrapped Cause in the original Cause we have to check
        || isNotNull(faultCause) && faultCause.getCause() instanceof WstxUnexpectedCharException
        || faultCause instanceof IllegalArgumentException) {
        return true;
    }
    return false;
}

Building custom SOAP faults

OK, now ne “know” that some invalid or incorrect XML request tried to frighten our endpoint. Let´s now tailor our custom SOAP fault. The class WeatherSoapFaultHelper is able to change the SOAP fault to our needs. The method buildWeatherFaultAndSet2SoapMessage(SoapMessage message, FaultConst faultContent) extracts the org.apache.cxf.interceptor.Fault out of the org.apache.cxf.binding.soap.SoapMessage. Now we have the fault where we could set our desired message and detail:

public static void buildWeatherFaultAndSet2SoapMessage(SoapMessage message, FaultConst faultContent) {
	Fault exceptionFault = (Fault) message.getContent(Exception.class);
	String originalFaultMessage = exceptionFault.getMessage();
	exceptionFault.setMessage(faultContent.getMessage());
	exceptionFault.setDetail(createFaultDetailWithWeatherException(originalFaultMessage, faultContent));
	message.setContent(Exception.class, exceptionFault);
}

It uses the other class WeatherOutError inside the package transformation to arrange the actual WeatherException. As you for sure remember, our specification states that the WeatherException has to be put into the detail tag in our soap:Fault:

private static final de.codecentric.namespace.weatherservice.exception.ObjectFactory objectFactoryDatatypes = new de.codecentric.namespace.weatherservice.exception.ObjectFactory();
 
public static WeatherException createWeatherException(FaultConst faultContent, String originalFaultMessage) {
    // Build SOAP-Fault detail <datatypes:WeatherException>
    WeatherException weatherException = objectFactoryDatatypes.createWeatherException();        
    weatherException.setBigBusinessErrorCausingMoneyLoss(true);
    weatherException.setBusinessErrorId(faultContent.getId());
    weatherException.setExceptionDetails(originalFaultMessage);
    weatherException.setUuid("ExtremeRandomNumber");
    return weatherException;
}

There´s one interesting aspect about this: Apache CXF kicks out the root element of that piece of XML that one tries to set into soap:Fault/detail. Therefore we have a short look into the code of the method createFaultDetailWithWeatherException(String originalFaultMessage, FaultConst faultContent) of our WeatherSoapFaultHelper (exception handling is left out of this here for readability reasons):

private static Element createFaultDetailWithWeatherException(String originalFaultMessage,  FaultConst faultContent) {
	Document weatherExcecption = XmlUtils.marhallJaxbElementIntoDocument(WeatherOutError.createWeatherException(faultContent, originalFaultMessage));
	return XmlUtils.appendAsChildElement2NewElement(weatherExcecption);
}

With a little help of our XmlUtils we marshal the WeatherException into a org.w3c.dom.Document. Because the method Fault.setDetail() expects a org.w3c.dom.Element and kicks the root element, we prepend our WeatherException Document with a mock root element which can be thrown away by Apache CXF later on.

Is there a way to build test cases using invalid XML requests?

Now we have an implementation that is said to detect all the cryptic errors Apache CXF produces when requests are sent containing invalid or incorrect XML. Additionally we have a bunch of test cases that we could manually send to our SOAP endpoint (e.g. with SoapUI). But do we have to believe the author? Fortunately not 🙂 Just think of all the things that could break if there´s a small incline of a version number in some used libraries or Apache CXF itself.

Here our knowledge from the preceding part Testing SOAP web services comes in handy. We just have to write some automatically executable tests. Ideally some single system integration tests that fire up our SOAP server endpoint inside the test´s execution.

And as we saw in the paragraph “How to deal with your test cases”, we could also load our test files and marshal them directly into the appropriate object. Or couldn´t we? No, sadly not. I guess you already know why. Because we want to send requests containing invalid XML, we´re not able to use the power of our JAX-B marshallers. If we try to marshal them, we would get similar errors as Apache CXF would throw in its outbound chains in case of an error triggered from invalid XML.

But we still want to be able to write some tests that we could automate. And there´s a way to do it. The core problem is just to send text messages containing our invalid XML via HTTP POST against our endpoint. All we need is a mature HTTP client and we are able to use our invalid XML test files. OK, let´s go! First of all we extend our pom by adding two new dependencies: org.apache.httpcomponents.httpclient and org.apache.httpcomponents.fluent-hcs. But before starting to use our HTTP client, we´ll have a short look at a SOAP message including its HTTP headers that is SOAP 1.1 compliant (in SoapUI just click on “Raw”):

POST http://localhost:8080/soap-api/WeatherSoapService_1.0 HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "http://www.codecentric.de/namespace/weatherservice/GetCityForecastByZIP"
Content-Length: 289
Host: localhost:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
 
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:gen="http://www.codecentric.de/namespace/weatherservice/general">
   <soapenv:Header/>
   <soapenv:Body>
      notRelevantHere />
   </soapenv:Body>
</soapenv:Envelope>

Besides the content type, what´s really important here is the HTTP header field SOAPAction. It should contain the SOAP operation, according to the SOAP specification. If not present, our endpoint will complain. But what´s the exact value we have to put in there? This is defined inside the WSDL in the soap:operation´s tag attribute soapAction, which both belong to the wsdl:operation definition. So if we like to use our HTTP client – e.g. via the fashionable fluent API – we have to set the SOAPAction HTTP header correctly. And don´t forget the quotes (which you have to escape)!

Response httpResponseContainer = Request
            .Post("http://localhost:8090/soap-api/WeatherSoapService_1.0")
            .bodyStream(xmlFile, ContentType.create(ContentType.TEXT_XML.getMimeType(), Consts.UTF_8))
            .addHeader("SOAPAction", "\"http://www.codecentric.de/namespace/weatherservice/GetCityForecastByZIP\"")
            .execute();
 
HttpResponse httpResponse = httpResponseContainer.returnResponse();

These few lines of code suffice so that we can torture our endpoint with weird XML requests. As a somehow enhanced version the class SoapRawClient inside our example project does exactly that. We just configure it as a Spring bean in the WebServiceSystemTestConfiguration and provide it with our generated Service Endpoint Interface (SEI). It then dynamically derives the correct SOAPAction header form the SEI. Additionally the method callSoapService(InputStream xmlFile) will give us a SoapRawClientResponse, which is really helpful while crafting our test cases.

single system integration tests with invalid XML

Now we have all the tools in place to finally write our desired test cases. Therefore we again use our knowledge from the last part of this tutorial – especially considering single system integration tests. Because they automatically fire up our SOAP endpoint for the period of the test´s execution. Besides, we know how to load our XML testcases and provide them as InputStream via Spring’s org.springframework.core.io.Resource in a really straightforward manner, without having to grapple with the file handling ourselves.

Our testcase WeatherServiceXmlErrorSystemTest is based on the same principles we know from the last part´s WeatherServiceXmlErrorSystemTest. So let´s look into the details. We inject our SoapRawClient and configure the test files to load:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes=SimpleBootCxfSystemTestApplication.class)
@WebIntegrationTest("server.port:8090") 
public class WeatherServiceXmlErrorSystemTest {
 
    @Autowired private SoapRawClient soapRawClient;
 
    @Value(value="classpath:requests/xmlerrors/xmlErrorNotXmlSchemeCompliantUnderRootElementTest.xml")
    private Resource xmlErrorNotXmlSchemeCompliantUnderRootElementTestXml;
 
    @Value(value="classpath:requests/xmlerrors/xmlErrorSoapBodyTagMissingBracketTest.xml")
    private Resource xmlErrorSoapBodyTagMissingBracketTestXml;
 
    // ... and many more

After that we provide every test file with its own test method. All these methods refer to the generalized method checkXmlError(), to which we forward the appropriate test file and the expected kind of error class. The latter are defined in FaultConst, which we already used in the paragraph “Building custom SOAP faults” to build our SOAP fault:

@Test
public void xmlErrorNotXmlSchemeCompliantUnderRootElementTest() throws InternalBusinessException, IOException {
    checkXMLError(xmlErrorNotXmlSchemeCompliantUnderRootElementTestXml, FaultConst.SCHEME_VALIDATION_ERROR);
}
 
@Test
public void xmlErrorSoapBodyTagMissingBracketTest() throws InternalBusinessException, IOException {
    checkXMLError(xmlErrorSoapBodyTagMissingBracketTestXml, FaultConst.SYNTACTICALLY_INCORRECT_XML_ERROR);
}
 
// ... and many more

And finally we´ll see some asserts in action. 🙂 Inside the checkXmlError() method we check our resulting SOAP faults thoroughly. Among others we look for a HTTP status code 500 and the tag faultstring should contain a message from our FaultConst. For simplicity reasons we use the SoapRawClientResponse´s method getFaultstringValue(), which gets us the faultstring out of the HTTP message. With the help of the convenient getUnmarshalledObjectFromSoapMessage(Class jaxbClass) we also get our WeatherException out of the HTTP message. After that we can throw our assert statements:

private void checkXmlError(Resource testFile, FaultConst faultContent) throws InternalBusinessException, IOException {
    // When
    SoapRawClientResponse soapRawResponse = soapRawClient.callSoapService(testFile.getInputStream());
 
    // Then
    assertNotNull(soapRawResponse);
    assertEquals("500 Internal Server Error expected", 500, soapRawResponse.getHttpStatusCode());
    assertEquals(faultContent.getMessage(), soapRawResponse.getFaultstringValue());
 
    de.codecentric.namespace.weatherservice.exception.WeatherException weatherException = soapRawResponse.getUnmarshalledObjectFromSoapMessage(de.codecentric.namespace.weatherservice.exception.WeatherException.class);       
    assertNotNull("<soap:Fault><detail> has to contain a de.codecentric.namespace.weatherservice.exception.WeatherException",  weatherException);
 
    assertEquals("ExtremeRandomNumber", weatherException.getUuid());
    assertEquals("The correct BusinessId is missing in WeatherException according to XML-scheme.", faultContent.getId(), weatherException.getBusinessErrorId());
}

One important note here: Using the fully qualified name of your exception (e.g. de.codecentric.namespace.weatherservice.exception.WeatherException) you avoid confusion with identically named classes (de.codecentric.namespace.weatherservice.WeatherException). One could argue here that this should be handled by renaming the exceptions. But sadly this is something you´ll not be able to do in real world projects, where the WSDL is a given and immutable artifact. Just have a look at big enterprisey web services like BiPro.

Now we achieved everything we wanted: Our framework validates the XML requests against the XML schemas and we decide what the SOAP faults will look like. At the same time we can test all this automatically while being able to send arbitrary strange XML requests against our SOAP endpoint. Overall this solution is rather complex from an Apache CXF user´s point of view. Configuring custom SOAP faults as a reaction to XML validation errors could be something far more easy to do. But for now it should work very well for us.

As always, there are things left to look at. Just think about the weird namespaces or the monitoring of SOAP messages with the elastic stack. We´ll have a look at these topics in the upcoming blog posts.

Jonas Hecht

Trying to bridge the gap between software architecture and hands on coding, Jonas hired at codecentric. He has deep knowledge in all kinds of enterprise software development, paired with passion for new technology. Connecting systems via integration frameworks Jonas learned to not only get the hang of technical challenges.

Share on FacebookGoogle+Share on LinkedInTweet about this on TwitterShare on RedditDigg thisShare on StumbleUpon

Comment

Your email address will not be published. Required fields are marked *