Querying XML data with XQuery

By Scott Mitchell

Introduction
Let's face it, one of the primary tasks we, as Web developers, are faced with is querying data from some data store and allowing users to view and/or manipulate the information via a Web interface. typically, the data stores that we query from are traditional relational databases, like Microsoft SQL Server or Oracle. with relational databases, the de facto means for querying data is SQL. however, with the ever-continuing rise in the popularity of Web Services, and the need for a platform-independent, Internet-Transferable data representation format, XML data stores are becoming more and more popular. SQL was never designed for querying semi-structured data stores, and therefore is not suitable for querying XML data stores. (For more information on creating and using Web Services in ASP. net be sure to read: Creating and consuming a web service. for more information on XML be sure to read the xml faq category on aspfaqs.com .)

So, how does one query an XML data store and retrieve results from such a query? Most developers currently use XSLT and XPath to accomplish this task. XPath is a syntax for accessing only parts of an XML document that meet a certain criteria; XSLT is a technology that transforms an XML document from one form to another.

While XSLT and XPath have been in use for a while now, there is a new kid on the block: XML query, or XQuery for short. XQuery is a querying language designed specifically to work with XML data stores using a SQL-like syntax. as of July 2003, XQuery 1.0 is still under development by the W3C standards body. however, the core features and syntax or XQuery are solid, so now is as good a time as any to learn about XQuery, especially since Yukon, the next version of ms SQL Server, will have built-in XQuery support.

In this article we will examine the XQuery syntax and examine some use cases. following this we'll examine Microsoft's XQuery classes, which are currently available. with these classes, you can start using XQuery in your asp. net web applications today!

XQuery Basics
XQuery is used to query an XML document, so, first things first, we need an XML document to talk about while examining our varous queries. for this article, let's use the following XML document which describes the structure of a file system:

<?xml version="1.0" encoding="utf-8" ?><filesystem> <drive letter="C"> <folder name="My Programs" /> <folder name="Games"> <folder name="Quake"> <folder name="SavedGames"> <file>game1.sav</file> <file>game2.sav</file> </folder> <file>quake.exe</file> <file>README.txt</file> <file>EULA.txt</file> </folder> </folder> <folder name="Windows"> <folder name="System32" /> <file>win.exe</file> </folder> </drive> <drive letter="D"> <folder name="Backup"> <file>2003-06-01.bak</file> <file>2003-06-07.bak</file> </folder> </drive></filesystem>

The root element of this XML document is<filesystem>, Which contains an arbitrary number<drive>Elements. Each<drive>Element, in turn, contains an arbitrary number<folder>Elements, and each<folder>Element contains an arbitrary number<folder>Elements and<file>Elements.

In its simplest form, an XQuery query can simply be an XPATH expression. (If you're unfamiliar with XPath, I wowould strongly encourage you want to work through this XPath tutorial before continuing .) for example, if we wanted to get a list of all of the files in the C drive, we cocould use the following XPath expression as our XQuery query:

document("FileSystem.xml")/filesystem/drive[@letter="C"]//file

Thedocument("FileSystem.xml")Part indicates the XML data store: an XML file namedFileSystem.xml. The output of this query, givenFileSystem.xmlFile above, wocould be:

<file>game1.sav</file> <file>game2.sav</file> <file>quake.exe</file> <file>README.txt</file> <file>EULA.txt</file> <file>win.exe</file>

The output of an XQuery statement is a collection of XML elements. In the above example, It's a collection<file>Elements. You can add static XML elements by just inserting them in the XQuery query. For example, in the above example, perhaps we want all of<file>Elements to appear within an XML root element titledMyFiles. This cocould be accomplished with the following XQuery expression:

<MyFiles>    { document("FileSystem.xml")/filesystem/drive[@letter="C"]//file } </MyFiles>

With this addition, the output wocould be:

<MyFiles> <file>game1.sav</file> <file>game2.sav</file> <file>quake.exe</file> <file>README.txt</file> <file>EULA.txt</file> <file>win.exe</file></MyFiles>

Note that in our query we used braces ({ ... }) Around und the XPath expression within<MyFiles>Element. The braces denote that the content within the braces is an XQuery expression, and not literal content. For example, had we omitted the braces and used the query:

<MyFiles>    document("FileSystem.xml")/filesystem/drive[@letter="C"]//file </MyFiles>

The output wocould have been:

<MyFiles>    document("FileSystem.xml")/filesystem/drive[@letter="C"]//file </MyFiles>

Flwr expressions
While simple XPath expressions are fine and good, the real power of XQuery shines through with flwr expressions. flwr standsFOr-LEt-WHere-REturn, and is pronounced "Flower ". the flwr expression is akin to SQL's SELECT query; it allows for XML data to be queried with conditional statements, and then returns a set of XML elements as a result.

Take a moment to think about a SQL select clause. The main ingredients there areSELECT,FROM, AndWHEREClses.FROMClause specifies the table (s) to query over. Then, for each row for the table (s) inFROMClause,WHEREClause is evaluated. Those rows that pass the evaluation have those fields that are specified inSELECTClause outputted. flwr statements are strikingly similar, as we'll see in a moment.

Flwr, as the acronym implies, has four parts, or clses, to it:

• For-forClause specifies the XML node list to iterate over, and is akin to the SELECT statement'sFROMClause. The list of XML nodes is specified via an XPATH expression. For example, if we wanted to iterate over all of<folder>Elements, we 'd use the XPath expressiondocument("FileSystem.xml")//folder.
• Where-whereClause contains an expression that evaluates to a Boolean, just likeWHEREClause in a SQL SELECT statement. Each XML node in the XML node list inforClause is evaluated bywhereClause expression; those that evaluate to true move on, those that don't are passed over.
• Return-returnClause specifies the content that is returned from the flrw expression.

You may have noticed that I have omittedletClause. The examples we'll be looking at in this article will not useletClause.

Now that we have briefly examined the three essential parts of the flwr expressions, let's see some examples! Here's a relatively straightforward example, showing how to get all<folder>Elements whosenameAttribute equals "Quake ":

for $myNode in document("FileSystem.xml")//folder where $myNode/@name="Quake" return $myNode

Notice thatforClause has the following form:

for variableName in nodeList

In XQuery, variable names are prefixed$(I. e .,$myNode).forClause enumerates the node list and for each node in the node list itBindsThe variable$myNodeTo the node. Then, inwhereAndreturnClses,$myNodeCan be used to reference the current node being evaluated. So,forClause iterates through all of<folder>Elements, and for each element,whereClause asks, "is this element'snameAttribute equal to quake? ", And if it is, thenreturnClause outputs<folder>Element.

ThereturnClause can be more involved. In fact,returnClause can return any XQuery expression. For example, we might want our output to look like the following:

<folder> <name>Quake</name> <files> <file>quake.exe</file><file>README.txt</file><file>EULA.txt</file> </files></folder>

We cocould accomplish this using the following XQuery expression:

for $myNode in document("FileSystem.xml")//folderwhere $myNode/@name="Quake"return <folder> <name>{string($myNode/@name)}</name> <files> {$myNode/file} </files> </folder>

Realize that flwr expressions are just as powerful as SQL select queries. flwrs are capable of joins, subqueries, and set-based operations, just like select queries.

Now that we're 've quickly looked at the XQuery syntax, let's turn our attention to using XQuery in. net! In part 2 we'll see how to get Microsoft'sXQueryClasses and how to start using them in an ASP. NET web application!

XQuery for. net
While the XQuery standard is not yet 100% complete, there are working implementations for a variety of platforms. microsoft provides a free XQuery engine. net, which is available at xqueryservices.com. in order to start using XQuery expressions in your asp. net web applications, you must first downloadxquery.msiFile at xqueryservices.com and start the installation process.

The installation process will create a new directory,C:/Program Files/XQuery Demo, Which contains, among other files and subdirectories, a file namedMicrosoft.Xml.XQuery.dll. To use XQuery expressions in an ASP. NET web application, simply copy this file to/binDirectory of your ASP. NET application. Next, you will want to importMicrosoft.Xml.XQueryNamespace in your code-behind class. Once you have completed MED these steps, you can start using XQuery expressions!

Xqueryservices.com down
Since at least July 27,200 3, xqueryservices.com has been down. (See this blog entry .) in any event, the word from Microsoft is that this site is having some "Issues" and shocould be back up soon. in the meantime, you can download the contents ofxquery.msiFile from the 4 guys server here.

Recall that all XQuery expressions are saved med over some XML document. the first step, then, in programmatically using XQuery expressions is to specify the XML document that will be involved in the search. to do so, we must createXQueryNavigatorCollectionInstance, and add to it the XML documents ents that we plan on using. (Note that we can perform XQuery expressions over more than one XML document, such as in semi Ming a join over two XML documents .) the following code demonstrates how to createXQueryNavigatorCollectionInstance and add to it an XML file:

Dim col as New XQueryNavigatorCollection()col.AddNavigator(Server.MapPath(filename), alias)

AliasIs a string alias that the XML document will be referred to in the XQuery expression. For example, if you had an XQuery expression like:

for $x in document(foo) return $x/bar

The aliasfooIs being used to reference the XML document. So, in your source code, you 'd want to use:

Dim col as New XQueryNavigatorCollection()col.AddNavigator(Server.MapPath(filename), "foo")

To actually perform the XQuery expression, we need to create an instance ofXQueryExpressionClass, passing in the XQuery expression we wish to execute in to the object's constructor. Following this, we callXQueryExpression'S object'sExecute()Method, passing inXQueryNavigatorCollectionInstance we created earlier. This will returnXQueryNavigatorInstance. We can call this object'sToXml()Method to get the raw XML back. The following Code demonstrates creating andXQueryExpressionObject, running the query, and getting back the raw XML:

Dim query as Stringquery = "for $x in document(""foo"")//bar " & _ "where $x/something = 4 " & _ "return $x/somethingElse"Dim expr as New XQueryExpression(query)Dim rawXML as String = (expr.Execute(col)).ToXml()

[View a live demo!]

That's all there is to it! The XQuery engine download from xqueryservices.com, using des sample code showing executing an XQuery expression in both C # and VB. NET. Furthermore, it contains a plethora of XQuery expression examples.

Conclusion
In this article we examined the basics of XQuery and looked at some sample XQuery syntax and example expressions. while the XQuery expressions examined in this article were fairly simple, do not let this fool you into thinking that XQuery can only perform simple queries.

In addition to examining the basic syntax of XQuery, we also looked at how to start using XQuery. net. microsoft provides a free XQuery engine at xqueryservices.com that you can use in your. net desktop applications or in ASP. net web applications. now is as good a time as any to start learning XQuery, as XQuery is bound to become more prominent as XML data stores continue their meteoric rise. furthermore, the next version of SQL Server will have inherent XQuery support.

To assist with learning XQuery, you might find the following articles helpful:

• XQuery, the query language of the future
• XQuery: an XML query language (PDF)
• Five Practical XQuery applications
• What is XQuery?
• An Introduction to XQuery

Happy programming!