Reading Time: 22 minutes

In spite of JSON's reign as the king of API data format, XML still remains the exchange data format of choice for a number of systems. Any service exposing functionality through SOAP, and many application built years ago (or even nowadays) still depend on XML to share data – to such an extent that in April 2013 the W3C published a new spec for version 3.0 of the XPath, XSLT and XQuery standards. We decided it was time to update the platform's support for these standards and fix a couple of things while at it.

At the moment of releasing mule 3.5.0, we were in a situation in which:

• We had only partial support for XQuery 1.0 and XSLT 2.0
• Users had a very inconsistent experience when dealing with XPath:
• When processing an XSLT template, we supported XPath 2.0
• When using the xpath() MEL function or the Xpath: expression evaluator, we only supported XPath 1.0
• The xpath() function and expression evaluator are like a box of chocolates: you never know what you're gonna get. The return type changes depending on how many results the query finds and whether or not it's a simple type or a node.
• The jxpath-filter and jxpath-extractor-transformer elements, which are supposed to only process POJOS, falls back to an actual XPath 1.0 expression through the use of dom4j if the message payload is an XML document

So, mea culpa. This was a mess, no shame in admitting it as long as we go and fix it. And that's why for Mule 3.6 we aimed to:

• Provide state of the art, 100% compliance support for XPath 2.0, XSLT 2.0, and XQuery 1.0
• Provide basic support for version 3.0 of the XML specs
• Reuse the existing XSLT and XQuery elements and functions we have (xpath-filter, xslt-transformer, xquery-transformer, etc) so that they can be used regardless of the targeted version spec
• Deprecate our current XPath support and provide a new, more usable and consistent solution allowing to use either XPath 2.0 and 3.0
• Deprecate all JXPath support in favor of simple MEL expressions.

Before we begin, a few words on the 3.0 spec

The XML specs 3.0 are still a recommendation, not yet approved by the W3C committee. However, they're on “last call” status, which means that they're highly unlikely to receive any substantial changes.

About XPath 3.0

XPath 3.0 is backwards compatible with 2.0. However, it's not fully compatible with version 1.0. Although a compatibility mode exists, it's doesn't cover all cases. This is one of the main reasons why although we'll provide a new API for Xpath processing, we'll still support the xpath() function which currently works with XPath 1.0 until mule 4 .0

What does basic support means?

Before we used the term “basic support” when referring to the 3.0 specs. By basic support we mean all features which don't rely on:

• Schema awareness
• High order functions
• Streaming

Improvements on XPath

As previously stated, we found that in our strive to provide the best experience possible we couldn't leverage Mule's existing xpath support, reasons being that we had an inconsistent and unusable mixture of Xpath 1.0 and 2.0, and that Xpath 3.0 is not backwards compatible with 1.0.

So, in the spirit of cleaning up we decided to deprecate the following components:

xpath: expression evaluator

xpath2: expression evaluator

bean: expression evaluator

jxpath filter

jxpath extractor transformer

jaxen-filter

Implicit things to take notice on:

Because XPath 3.0 is completely backwards compatible with 2.0, this function will also serve those wanting to use 2.0 expressions

This doesn't guarantee support on Xpath 1.0 expressions. The simpler ones will work, but the ones which are not compatible will not. Since XPath 1.0 is dated all the way back to 1999, we consider it deprecated and won't officially support it. Compatibility mode will be disabled.

Because we want this function to have predictable return types, we need to create a new xpath3() function. We considered adding a compatibility flag to the current function, but our analysis indicated that the impact was way too great for that to make sense. Therefore, a new xpath3 function was created and the existing xpath() one is deprecated

The new xpath3() function is of the following form:

xpath3(xpath_expression, input_data, return_type)

Let's take a closer view:

expression (required String)

The Xpath expression to be evaluated. Cannot be null or blank.

input (optional Object, defaults to the message payload)

The input data on which the expression is going to be evaluated. This is an optional argument, it defaults to the message payload if not provided

This function supports the following input types:

org.w3c.dom.Document

org.w3c.dom.Node

org.xml.sax.InputSource

OutputHandler

byte[]

InputStream

String

XMLStreamReader

DelayedResult

If the input if not of any of these types, then we'll attempt to use a registered transformer to transform the input into a DOM document or Node. If no such transformer can be found, then an IllegalArgumentException is thrown.

Additionally, this function will verify if the input is a consumable type (streams, readers, etc). Because evaluating the expression over a consumable input will cause that source to be exhausted, in the cases in which the input value was the actual message payload (no matter if it was given explicitly or by default), we will update the output message payload with the result obtained from consuming the input.

Output type (optional String, defaults to ‘STRING')

When executing an XPath expression, a developer might have very different intents. Sometimes you want to retrieve actual data, sometimes you just want to verify if a node exists. Also, the JAXP API ( JSR-206 ) defines the standard way for a Java application to handle XML, and therefore, how to execute XPath expressions. This API accounts for the different intents a developer might have and allows choosing from a list of possible output types. We consider this to be a really useful features in JAXP, and we also consider that many Java developers that are familiar with this API would appreciate that Mule accounts for this while hiding the rest of the API 's complexity.

That is why there's a third parameter (optional, String), which will allow specifying one of the following:

BOOLEAN: returns the effective boolean value of the expression, as a java.lang.Boolean. This is the same as wrapping the expression in a call of the XPath boolean() function.

STRING: returns the result of the expression converted to a string, as a java.lang.String. This is the same as wrapping the expression in a call of the XPath string() function.

NUMBER: returns the result of the expression converted to a double as a java.lang.Double. This is the same as wrapping the expression in a call of the XPath number() function.

NODE: returns the result the result as a node object.

NODESET: returns a DOM NodeList object. Components like the foreach, splitter, etc, will also be updated to support iterating that type.

Query Parameters

Another XPath feature that will now be supported is the ability to pass parameters into the query. For example, consider the following query which returns all the LINE elements which contains a given word:

//LINE[contains(., $word)]

the $ sign is used to mark the parameter. As for the binding, the function will automatically resolve that variable against the current message flow variables. So, if you want to return all the occurrences of the word ‘handkerchief', all you have to do is:

xpath3('//LINE[contains(., $word)]', payload, 'NODESET')

NamespaceManager

Unlike its deprecated predecessor, the xpath3 function will be namespace-manager aware, which means that all namespaces configured through a namespace-manager component will be available during the xpath evaluation.

For example, suppose you want to do an XPath evaluation over this document: