Introduction
Its good practice to design the interface to a web service by
applying the same standards that would normally be applied to an XML Schema. It
should be versioned, extensible and be well structured allowing for good reuse.
Background
Within a Web Service all the messages are described using an
XML Schema (XSD), so it makes sense to design our interface in terms of an XML
Schema, and then embed this into our web service. Liquid XML Studio is an ideal
editor for this, as it’s free and easy to use.
The following example shows the messages we would like our simple
web service to exchange. Our simple Web Service will
provide information about a given product, when provided with a product code, we will also need to provide login details to the service.
The information returned describes the product, pricing, and stock levels.
So we would like to build a web service with a web method that takes a ProductDetailsRequestType and returns a ProductDetailsResponseType.
So that’s what we’re after, now lets look at how to achieve this.
Using the standard .Net Framework Serialisation.
We could create a class for each entity in the schema, give
them the appropriate properties and let the standard .Net framework [WebMethod]
serialisation do the rest. This is an acceptable approach for small projects,
but if you want to pass complex objects or objects from 3rd party
standards (HL7, FpML, ebXML, etc) this starts to become overwhelming.
Another approach I've seen used a lot is just to pass the XML data via a string parameter. This has a number of drawbacks, the XML will all be escaped so it expands (< becomes < etc). Also the web service description (WSDL) now tells you nothing useful about the web service - it takes in a string and returns a string. Not a great deal of help for implementers. Also your web service provides no validation, your web service method will get called regardless of the content of the string - it doesn't even have to be XML.
A better approach is to use an XmlElement as the in/out parameters to your web service. This solves some of the problems, the XML is not escaped, your method only gets called if the XML is valid (well formed), but implementers still have no idea what the web service data is supposed to look like, and you only get minimal validation performed on your XML.
It is however possible to apply the techniques to allow an XmlElement to be passed via a web service and have it fully described in the WSDL. But more about this later.
The XML Data Binding Approach
A really easy way to deal with XML within your application
is XML Data Binding. This involves the automatic creation of a class for every
entity in your schema. These classes are bound to the structure of the schema; so
using our example we would end up with an object called 'Credentials', with
properties called 'Username' and 'Password'. This removes any notion of dealing
with XML from the application, and means the developer just has to deal with
objects, properties and collections, all strongly typed, and enforcing the structure
described in the XML Schema.
These objects can be generated using a Data Binding tool, Liquid
XML Studio has an XML Data Binding tool which will create .Net code for 1.1,
2.0, 3.5 & Silverlight both in C# and VB.Net.
Once we have generated our Data Bound class library, we can
pull together our web service. The first thing to do is declare our web method.
namespace ProductEnquiryWebService
{
[WebService(Namespace = "http://www.liquid-technologies.com/ProductDetailsSample")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[ToolboxItem(false)]
public class Service1 : System.Web.Services.WebService
{
[WebMethod(Description = "Gets the details about a given product.")]
public ProductDetailsResponse RequestProductDetails(ProductDetailsRequest productRequest)
{
ProductDetailsResponse response = new ProductDetailsResponse();
response.ProductDetails.Name = "Widget";
response.ProductDetails.Description =
"The important widget that goes between the flange and the spindle";
response.ProductDetails.Price = 25.36;
response.ProductDetails.StockLevel = 15000;
return response;
}
}
}
Note the web service uses the targetNamespace from the XML
Schema we defined at the start of the example. The web method takes in and
returns the Data Bound classes generated by Liquid XML Studio.
If we left things here then the .Net framework would use its
default serializer on our ProductDetailsRequest and ProductDetailsResponse
objects. This works using reflection and would end up serializing a large
amount of unwanted properties. Instead we are going to let these objects
serialise themselves. In order to do this we must tell the framework about
them.
[XmlSchemaProvider("MySchema")]
partial class ProductDetailsRequest
{
// This is the method named by the XmlSchemaProviderAttribute applied to the type.
public static XmlQualifiedName MySchema(XmlSchemaSet xs)
{
// This method is called by the framework to get the schema for this type.
// We return an existing schema from disk.
if (xs.Schemas("http://www.liquid-technologies.com/ProductDetailsSample").Count == 0)
{
using (FileStream fs = new FileStream(@"ProductEnquiry.xsd", FileMode.Open))
{
XmlSchema s = XmlSchema.Read(fs, null);
xs.Add(s);
}
}
return new XmlQualifiedName("ProductDetailsQueryType",
"http://www.liquid-technologies.com/ProductDetailsSample");
}
}
In order for the framework to be able to correctly describe
the web service we first need to tell it about the schema. This is done by
adding the XmlSchemaProvider attribute to all classes that are in or out
parameters in the web service. The XmlSchemaProvider attribute tells the framework
that information about the schema can be obtained by calling the “MySchema”
method.
Now when the framework is trying to describe the web service it will
call the MySchema method. The framework expects our implementation of MySchema
to add all the schemas it needs to know about into the schema set collection, so
if you are working with elements from external schemas (HL7, FpML etc) then you
can add these to the schemas collection as well.
The return parameter from
MySchemas is the fully qualified name of the type within our XSD that describes
the parameter. Note this must be a complexType, the .Net framework will not
allow you to specify an element.
Customising the Serialization Performed by XmlElement
As discussed earlier, it is possible to pass XmlElements as parameters in your web service, and have them fully described in the WSDL.
In order to do this a class wrappering an XmlElement must be passed instead of the XmlElement,
this class must implement the IXmlSerializable interface, and have its
own XmlSchemaProvider attribute.
Because the 'MySchema' method is static, one wrapper class must be created for
each type of object being passed, as the return value tells the .Net framework which complexType in the schema this argument represents.
A further improvement on this would be to add validation to the element prior to serialisation, thus ensuring that the XML was actually valid before reading or writing it.
[XmlSchemaProvider("MySchema")]
public class MyXmlElement : IXmlSerializable
{
private XmlElement _element = null;
public MyXmlElement() {} // requires a parameterless constructor
public MyXmlElement(XmlElement xmlElemnt) { _element = xmlElemnt; }
// This is the method named by the XmlSchemaProviderAttribute applied to the type.
public static XmlQualifiedName MySchema(XmlSchemaSet xs)
{
// This method is called by the framework to get the schema for this type.
// We return an existing schema from disk.
if (xs.Schemas("http://www.liquid-technologies.com/ProductDetailsSample").Count == 0)
{
using (FileStream fs = new FileStream(@"ProductEnquiry.xsd", FileMode.Open))
{
XmlSchema s = XmlSchema.Read(fs, null);
xs.Add(s);
}
}
return new XmlQualifiedName("ProductDetailsRequestType", "http://www.liquid-technologies.com/ProductDetailsSample");
}
#region IXmlSerializable Members
public System.Xml.Schema.XmlSchema GetSchema()
{
throw new NotImplementedException();
}
public void ReadXml(System.Xml.XmlReader reader)
{
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(reader);
_element = xmlDoc.DocumentElement;
}
public void WriteXml(System.Xml.XmlWriter writer)
{
_element.WriteTo(writer);
}
#endregion
}
[WebMethod(Description = "Showing the technique with XmlElement.")]
public MyXmlElement Test()
{
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(@"SampleData.xml");
return new MyXmlElement(xmlDoc.DocumentElement);
}
Testing your Web Service
Because your web service contains complex parameters it is not
possible to call it via the simple Web Service Explorer.
So in order to call it you need to either build yourself a
test harness or use a test tool. Liquid XML Studio contains a handy tool for
calling web services. This will automatically generate the request envelope and
message, allowing you to modify can call it.
Sample Request
<?xml version="1.0" encoding="utf-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Body>
<tnsa:RequestProductDetails xmlns:tns="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:tnsa="http://www.liquid-technologies.com/ProductDetailsSample"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<tnsa:productRequest>
<tnsa:Credentials>
<tnsa:Username>Joe</tnsa:Username>
<tnsa:Password>password</tnsa:Password>
</tnsa:Credentials>
<tnsa:ProductSerialNo>0123456798</tnsa:ProductSerialNo>
</tnsa:productRequest>
</tnsa:RequestProductDetails>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
Sample Response
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<RequestProductDetailsResponse xmlns="http://www.liquid-technologies.com/ProductDetailsSample">
<RequestProductDetailsResult>
<!--Created by Liquid XML Data Binding Libraries (www.liquid-technologies.com) -->
<AA:ProductDetailsResponse xmlns:AA="http://www.liquid-technologies.com/ProductDetailsSample">
<AA:ProductDetails>
<AA:Name>Widget</AA:Name>
<AA:Description>The important widget that goes between the flange and the spindle</AA:Description>
<AA:Price>25.36</AA:Price>
<AA:StockLevel>15000</AA:StockLevel>
</AA:ProductDetails>
</AA:ProductDetailsResponse>
</RequestProductDetailsResult>
</RequestProductDetailsResponse>
</soap:Body>
</soap:Envelope>
Examining the WSDL
Because we have taken control of the serialisation process
the WSDL describing the web service contains the contents of the schemas that
you added in via the MySchema method.
Download the WSDL - 4.62 KB
It is also possible to explore this graphically using the
Web service Call Composer in Liquid XML Studio
Conclusion
If you are building web services that require a hierarchy of
objects to be sent or received, especially if those messages contain sections
from existing schemas (formal standards or internal data structures) or contain
complex constructs (inheritance, substitution groups etc), then you need more
control over the WSDL that defines your web service.
XML Data Binding not only simplifies
the reading and writing of your XML data, providing a framework of strongly
typed objects in which to work, but can provide information to the .Net Web Service
Framework that allows it to fully describe even the most complex data
structures accurately within the WSDL.
Using this approach makes it possible to include parts of complex standards within your web service and still have them properly described in the WSDL, enabling other tools to properly understand and use the interface.
Resources
History
Created : 12/1/2009
I recently had put a DTD parser together, there are a few open source ones out there but only for perl, java and C++, and our target platform is C#, so I had to start over. I initially thought the parser would be pretty straight forward, until I hit upon a little jem in the DTD standard that allows for text substitution.
This puts a whole new complexion on the problem, because the DTD standard can define substitutions (defined using the ENTITY tag), it can take a number of parses to determine a documents true meaning. This makes parser design much more difficult, slows validation and is error prone, see the following example.
Using the following example.
1: <?xml version='1.0'?>
2: <!DOCTYPE test [
3: <!ELEMENT test (#PCDATA) >
4: <!ENTITY % xx '%zz;'>
5: <!ENTITY % zz '<!ENTITY tricky "error-prone" >' >
6: %xx;
7: ]>
8: <test>This sample shows a &tricky; method.</test>
The first pass through, expands the entity %xx; defined in line 4, into its definition in line 6 giving (% expands to %)
1: <?xml version='1.0'?>
2: <!DOCTYPE test [
3: <!ELEMENT test (#PCDATA) >
4: <!ENTITY % xx '%zz;'>
5: <!ENTITY % zz '<!ENTITY tricky "error-prone" >' >
6: %zz;
7: ]>
8: <test>This sample shows a &tricky; method.</test>
The second pass through expands %zz; defined in line 5 into line 6 (< expands to <)
1: <?xml version='1.0'?>
2: <!DOCTYPE test [
3: <!ELEMENT test (#PCDATA) >
4: <!ENTITY % xx '%zz;'>
5: <!ENTITY % zz '<!ENTITY tricky "error-prone" >' >
6: <!ENTITY tricky "error-prone" >
7: ]>
8: <test>This sample shows a &tricky; method.</test>
And finally the &tricky; in the XML is expanded to "error-prone"
1: <?xml version='1.0'?>
2: <!DOCTYPE test [
3: <!ELEMENT test (#PCDATA) >
4: <!ENTITY % xx '%zz;'>
5: <!ENTITY % zz '<!ENTITY tricky "error-prone" >' >
6: <!ENTITY tricky "error-prone" >
7: ]>
8: <test>This sample shows a error-prone method.</test>
Useful tools
We've been doing some work with FpML (Financial Products Markup Language) which is an XML standard for describing trades between investment banks, swaps, futures etc. Its a complicated standard in the form of an XSD (XML Schema). So producing XML that is valid against the FpML schema is not entirely straight forward.
The best approach we found was to prototype the XML messages by hand before writing the code that would do this automatically from our front office system. This was reasonably straight forward where we had examples to copy, but suitable example only covered a few of our use cases, so we had to start designing some of the XML from scratch.
We started off trying to use notepad to create the XML, but this was a non-starter, tiny errors would take an age to track down and it's not what anyone would call a fully featured editor. We downloaded and started using Microsoft's XML Notepad, but it was awkward to use, and didn't seem to have any knowledge of the schema, so the validation was very limited.
Eventually one of the team found Liquid XML Editor, this is a free text based editor that is schema aware, and even shipped with the FpML standards so when we opened our XML file it automatically loaded the schema, and told us about our errors.
As well as validation, the editor also uses the schema as a source for the intellisense and documentation, which is pretty powerful given the complexities of the FpML standard. We ended up playing with a number of XML Editors, and this was the only one that could cope with the way the complexTypes are defined within the FpML standard.
The intellisense and validation helped a lot when trying to build up our XML prototypes, but what was still lacking was a feel for the structure of the FpML entities. We tried looking at the FpML schemas, but these were all but un-readable in code form. We eventually stumbled on a feature within the XML schema editor, that allows you to create HTML documentation from an XML Schema, we later discovered this documentation was already online see FpML online documentation. This gave us a good feel for the shape of our target XML document. We then used Liquid XML studio's XML editor to finish prototyping our XML documents.
Our final problem was the target system could not cope with the XML documents we produced. Initially we figured our XML was wrong, but after running our XML through a number of validation applications, and digging through the XSD standard we concluded the error was in the target system. After a lot of hassle with there support team, we eventually figured out a solution which involved adding in a few additional elements. This effectively made our XML non-FpML compliant, but it fixed the problem we set out to solve. We left a load of comments in the code and XML to explain the issues, but I've seen this kind of thing to many times where a standard is 'interpreted' by a number of 3rd parties and the resulting inconsistencies end up causing issues for years to come. Good tools, especially in the XML validation area should start to reduce this problem in future but its unlikely to resolve it altogether.
We are now looking at the feasibility of setting up an XML firewall on all our service clients. Basically anything that comes in must have an associated schema and it must validate against it. If it fails validation, then it doesn't get anywhere near our system. This seems like a great idea in principle, but I recon we'll end up bouncing so much traffic, we'll be forced to take it down…time will tell.
Our company has a pretty wide mix of systems Windows mostly VB & .Net and Solaris Java & C++.
Standards without standards.
Our company’s standard for interoperability is web services, however that’s more or less where standardisation stopped. The protocols and general design of the web services is down to the individual teams. On the whole the java guys produce XML schemas for there messages, then use this to generate web service stubs using various toolkits, the .Net guys tend to just create a load of objects and let .net serialise them as it sees fit. The Visual Basic code is legacy, and there are some interesting home made web interfaces thrown over some of this code.
Although all these teams work within the Web Service standards, the resulting interfaces can me more or less proprietary.
.Net & Web Services
The web service interfaces that result from .Net using the [WebMethod] attribute in .Net can lead to some quirky WSDL, as the .Net serialiser can add additional mark-up within the embedded XSD, in order to describe classes like Lists etc. This can make life very difficult for non .Net applications to consume.
The flip side of this is that when you try to reference a Web Service built around a complex XML schema (like those sometimes created within java applications), the .Net web method wrapper (the thing run behind the scenes when you add a web reference) can produce some interesting objects, or fail all together.
Java & Web Services
Its more difficult to generalise with java as there are many web service toolkits, but typically as the starting point for a web service is quite often an XML Schema, more thought goes into the messages, and they are typically of a higher standard. This can lead to problems when some of the more exotic devises within the XSD standard are used (Substitution Groups, odd forms of type restriction etc), which can trip up the proxy generation tools (both on the java tools and the .Net ones).
On the flip side, the java proxy generation tools are typically better than there .Net counterparts, and while you can end up with some ugly looking objects, they tend to cope with more complex schemas.
Conclusions when designing a web service.
If interoperability is something that would be valuable to your project now or in the future, then think about the interface. Ultimately this is expressed as an XSD embedded within your WSDL. If its structured correctly, from the outset your life will be much simpler.
Tips for examining your web services structure.
Visualising the Web Service
The XSD describing your Web Service is embedded within your WSDL. Looking at the XSD code in its raw from, can be pretty off putting, so its best to use a tool that will allow you to view it as diagram. We’ve been using Liquid XML studios XML schema viewer for this.
Visualising the request/response from the Web Service.
Ultimately it all comes down to an XML request and response. These are structured according to the XSD within the WSDL. You can easily build sample request, and see the web service output using Liquid XML Studios web service call composer. This allows you to call a web service without having to write an application. It constructs a sample request that you can change to add in more appropriate values, then shows the results.
I've just spent that last week building a translator, allowing us to convert a new customers catalogue data into a form our sales system can cope with. Luckily the source data was XML, so it was already in a form I could use, but there was no schema, and the structure was a bit flaky in places.
I started off trying to read through the data, but the structure was just to messy, in the end I figured out I could probably do most of the grunt work with XPath. This approach seemed promising, and I got good results initially, however my lack of XPath experience quickly started to let me down. Changing your XPath expression in code (C#) is possible, but typically means a re-compile - especially when you have anonymous methods, which I use a lot for sorting collections.
So I went in search of a good XPath tutorial, and I was somewhat disappointed with what I found. There are a few - but they are a bit lacking, the W3Schools XPath Article one was the most complete, and an XPath article on Google Knol had some nice examples. If anyone comes across a really good XPath tutorial let me know!
What I did find while I was looking was a free XPath Viewer, it highlights the selected nodes as you type your XPath query. This made the learning process so much quicker, and allowed me to debug a lot of my XPath queries before they got as far as the code. The Liquid XML Studio XPath viewer also added itself to Visual Studio 2008 which made my life even easier.
Liquid XML Studio XPath Viewer