Thursday 15 September 2011

How to create a SOAP request

So you know there's a web service deployed on some web server and you want to make a web service call. One way of achieving this is to create SOAP message, put it in a body of HTTP POST command and send it (via TCP/IP) to that web server. If you are using some of web service frameworks (e.g. Apache Axis), you don't need to know the structure of SOAP message; you'll just be adding its elements through dedicated framework functions. But if you don't want to rely on such frameworks, be prepared for creating and parsing SOAP messages by yourself! You can use some HTTP framework (e.g. Microsoft's WinHTTP) to create HTTP POST request and then insert your SOAP request in HTTP message body. Or, you can go one level down and create TCP socket, manually build SOAP and HTTP messages and send them over TCP connection...In this article I'll focus on SOAP messages only.

SOAP (Simple Object Access Protocol) is defined by World Wide Web Consortium (W3C) and the specification of its latest version can be found here.

Paragraph 5 ("SOAP Message Construct") states that SOAP message is represented with one element, named Envelope, from a namespace "http://www.w3.org/2003/05/soap-envelope". XML namespace can be referred via its prefix which is like its alias (abbreviation) and can be any string, soapenv in this case:

<soapenv:envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
...
</soapenv:envelope>

Prefix and local name of an element together make its qualified name (QName). XML parser uses the prefix to find the actual namespace URI.

Following examples are using different namespace prefixes but they all refer to the same URI where namespace is defined:

<soap:envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
...
</soap:envelope>

<soap-env:envelope xmlns:soap-env="http://www.w3.org/2003/05/soap-envelope">
...
</soap-env:envelope>

<env:envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
...
</env:envelope>

Apart from its local name and namespace, Envelope element contains:
  • other namespaces, from other schemas (optional)
  • namespace-qualified attributes (optional)
  • Header element (optional child)
  • Body element (mandatory child)

namespace-qualified means that node's name is made of namespace prefix and a local name.

One of the possible attributes determines which encoding should be used to deserialize this message. Its name must contain namespace part (envelope's namespace) and its local name - encodingStyle. Its value is URI of schema that defines encoding:

<soapenv:envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"
                  soapenv:encodingStyle="http://www.w3.org/2003/05/soap-encoding">
...
</soapenv:envelope>

SOAP Header element contains:
  • its qualified name made up of envelope's namespace and local name Header (mandatory)
  • namespace-qualified attributes (optional)
  • namespace-qualified SOAP Header Blocks - children elements (optional)
Each Header Block contains:
  • local name and namespace (mandatory)
  • attributes (optional); any of all of following attributes defined in soapenv namespace:
      • encodingStyle
      • role
      • mustUnderstand
      • relay
SOAP Body element contains:
  • its qualified name made up of envelope's namespace and local name Body (mandatory)
  • namespace-qualified attributes (including encodingStyle) (optional)
  • namespace-qualified children elements (optional)

Let us assume that web service has method add which accepts two arguments - two integers - and returns their sum. It would expect to receive XML element like this:

<add>
   <op1>3</op1>
   <op2>4</op2>
</add>

Client should send this element within Body part of SOAP message which could be, in its simplest form (containing only mandatory nodes), like this:

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
 <soapenv:Body>
  <add>
   <op1>3</op1>
   <op2>4</op2>
  </add>
 </soapenv:Body>
</soapenv:Envelope>

Web server receives HTTP POST message, extracts SOAP message from its body and forwards it to Web Service which analyses its headers (if any) and extracts XML from its body. It parses XML and extracts web method name and its arguments.

Obviously, both client and service must use the same function and argument names. This can be enforced if they both use XML documents that are associated to (and are validated against) the same XML schema. How to create one for our XML? If we know the valid structure of it (defined here; see W3C XML Schema page), we could do it manually (which is a kind of reverse engineering) but this is not a way to go: it's better using some tool which will generate it for us from a given XML. Microsoft Visual Studio (2008/2010) could be used as such tool: open your XML file in it and select XML->Create Schema from a main menu. It will create XML Schema document file, named after XML file and with extension xsd.

For wstest.xml:
<?xml version="1.0" encoding="utf-8"?>
<add>
   <op1>3</op1>
   <op2>4</op2>
</add>

wstest.xsd is output:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeformdefault="unqualified" elementformdefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="add">
    <xs:complextype>
      <xs:sequence>
        <xs:element name="op1" type="xs:unsignedByte"/>
        <xs:element name="op2" type="xs:unsignedByte"/>
      </xs:sequence>
    </xs:complextype>
  </xs:element>
</xs:schema>

In the example above we are not using namespaces which is a bad practice as element/attribute name clashes can occur. Let us write an XML with namespace-qualified names:
<?xml version="1.0" encoding="utf-8"?>
<ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1>3</ns1:op1>
   <ns1:op2>4</ns1:op2>
</ns1:add>

Its wstest.xsd:
<?xml version="1.0" encoding="utf-8"?>
<xs:schema xmlns:ns1="http://www.bk.com/webservices/wstest" attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://www.bk.com/webservices/wstest" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="add">
    <xs:complextype>
      <xs:sequence>
        <xs:element name="op1" type="xs:unsignedByte"/>
        <xs:element name="op2" type="xs:unsignedByte"/>
      </xs:sequence>
    </xs:complextype>
  </xs:element>
</xs:schema>

XML schema above defines namespace "http://www.bk.com/webservices/wstest" (look for targetNamespace attribute) and other metadata related to our XML: valid element names ("add", "op1", "op2"), element and value types. It is worth mentioning here that namespace names are just arbitrary strigs, not locations on the web that XML parsers use in order to retrieve any information. We usually use URIs though (in form of URNs or URLs) as that decreases the chance of two namespaces having the same name. XML parsers treat them just like simple strings.

SOAP request which regards this new namespace looks like this:
<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
 <soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1>3</ns1:op1>
   <ns1:op2>4</ns1:op2>
  </ns1:add>
 </soapenv:Body>
</soapenv:Envelope>

Having local names within namespaces still does not make this document valid. XML document is valid only if it meets requirements of the schema with which it has been associated.

XML Schema can be referenced in an XML document by using schemaLocation attribute in document's root element. This attribute is defined in "http://www.w3.org/2001/XMLSchema-instance" namespace (which is traditionally aliased with xsi name) so we need to include this namespace as well.

We can reference our schema in our SOAP request:
<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.bk.com/webservices/wstest wstest.xsd">
 <soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1>3</ns1:op1>
   <ns1:op2>4</ns1:op2>
  </ns1:add>
 </soapenv:Body>
</soapenv:Envelope>

There are two forms of referencing schema:

  • using relative URL and schema file name which says that the XSD file is in the same directory as the XML document: 
xsi:schemaLocation="http://www.bk.com/webservices/wstest wstest.xsd" (used in the example above)
               or
xsi:schemaLocation="wstest.xsd"

  • using absolute URL of XSD:
xsi:schemaLocation="http://www.bk.com/webservices/wstest/wstest.xsd"

If a schema processor finds schema reference in a document, it should try to retrieve the schemas at the locations indicated. When no information is provided at all the schema processor is free to try any method to find a schema. 

If our XML document uses simple types, we can use and reference W3C XML Schema in our document. It is bounded to "http://www.w3.org/2001/XMLSchema" namespace whose aliases are traditionally xs (used in this example) and xsd (often used for Microsoft schemas).

<?xml version="1.0"?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <soapenv:Body>
  <ns1:add xmlns:ns1="http://www.bk.com/webservices/wstest">
   <ns1:op1 xsi:type="xs:int">3</ns1:op1>
   <ns1:op2 xsi:type="xs:int">4</ns1:op2>
  </ns1:add>
 </soapenv:Body>
</soapenv:Envelope>

There are many tools that generate WSDL documents for given Web Services. WSDL document describes Web Service, lists its public methods and their arguments. It refers XML schema(s) and describes valid XML structure for web method calls. WSDL (and so XML Schemas) is made public so clients can create SOAP requests and validate them. Schema is used by web service to validate incoming SOAP request and then on the client side to validate response.

Links and references:

SOAP Version 1.2 Part 0: Primer
SOAP Version 1.2 Part 1: Messaging Framework
SOAP Version 1.2 Part 2: Adjuncts
SOAP 1.2 Part 3: One-Way MEP
XML Namespaces by Example
XML Tutorial
Creating Extensible Content Models
XML Schema Part 0: Primer
XML Schema Part 1: Structures
XML Schema Part 2: Datatypes
Using W3C XML Schema
XML Schema (O'Reilly eBook)
XML Schema (W3C)
XML schema (Wikipedia)
XML Schema (W3C)(Wikipedia)
XML Schema (XSD) validation tool? (Stack Overflow)
XML namespace (Wikipedia)
XML Namespaces (W3Schools)
XML Namespace Name: URN or URL?
Understanding XML Namespaces (MSDN Magazine)
XPath, XSLT, and other XML Specifications (MSDN Magazine)
XML Namespaces FAQ
The basics of using XML Schema to define elements
Tip: Send and receive SOAP messages with SAAJ
Apache SOAP type mapping, Part 1: Exploring Apache's serialization APIs
SOAP Tutorial (W3Schools)
XML Schema Tutorial (W3Schools)
SoapUser.com
WSDL Essentials

No comments: