SOAP Scripts in Mozilla

Last Modified Wed, 20 Mar 2002 01:01:58 GMT

Ray Whitmer

Abstract

Microsoft and others have advocated SOAP as a way to encode and exchange public data structures between agents on the web, which is now becoming a web standard at W3C (see the multiple submissions and working documents related to SOAP in the list of W3C current drafts ).

The browser client is the most universal web agent in existence, and Javascript is the standard, interoperable way of scripting browsers. Scriptable SOAP in browsers gives clients and servers more to say to each other through existing http-xml request services, providing scripts with persistence, database, and access to other web services not tied to the request and response cycles of the HTML-based user interface. Web data structures, exchanged in a platform-neutral way, should become as fundamental to web agents as web content is today. The key to this is a very natural binding to the data of Javascript so that the script can simply use the data instead of tediously encoding and extracting the data from the XML.

0 Table of Contents

1 SOAP Services

2 SOAP Blocks

2.1 Parameters
2.2 Header Blocks
2.3 Encodings

3 Using the Mozilla Low-Level SOAP API

3.1 Conventions Used Below in
3.2 Basic Operations of SOAP
3.3 Beyond Basic Operations
3.4 Header Operations
3.5 Non-RPC Operations
3.6 SOAPBlock and SOAPMessage Supertypes
3.7 More Operations
3.8 Using Schema Types
3.9 Customization of Encodings
3.10 Security Operations

4 Future Features

4.1 Access to SOAP as Proxies
4.2 Arbitrary Graphs of Data
4.3 SOAP With Attachments
4.4 New Transports and Local Services
4.5 Standards

5 Samples and Testing

6 Object Interfaces

1 SOAP Services

There are a number of sources for services available on the web being set up, such as the XMethods website which the Mozilla implementation has used for some tests and use cases. Apache also provides modules for SOAP that have been used to author services for purposes of testing the Mozilla implementation (and entertainment). Once it is set up, it is as simple as writing a service function in Javascript, Java, or any other of a number of supported languages and then writing a service description in XML to deploy the service. There are toolkits available from Microsoft and other webserver providers for authoring such services as well.

2 SOAP Blocks

2.1 Parameters

SOAP-based services exchange message envelopes which contain blocks of XML data roughly corresponding to the parameters of a service call. When an rpc-style message is exchanged, blocks representing the regular parameter blocks are placed inside an element which identifies the object and method being invoked, which is placed inside the body. In a non-RPC message, the blocks are placed directly inside the body instead of under the method element.

2.2 Header Blocks

If there are blocks which are optional or independently added or processed, these are carried in the header with an assigned role and marked if the recipient is required to understand them before trying to process the message.

2.3 Encodings

Interpretation of each block depends upon the encoding that was used, which is clearly specified in the message. If the standard SOAP encoding is used, then XML Schema types control the interpretation of the data within each block.

3 Using the Mozilla Low-Level SOAP API

To use the low-level API, the user creates a SOAPCall object, encodes the function call with a list of headers and regular parameters, and invokes the call, which returns a response which contains the results of the service call including a fault generated by the service which processed the message if it failed, output parameters, and/or header blocks. If the call is invoked asynchronously, then a function is supplied by the caller which receives the response when it arrives from the remote service.

Besides Javascript, the below-described operations should also generally work for other xpconnect-supported languages in Mozilla such as C++/XPCOM and Python, because language-independent cross-platform interfaces and architectures were used.

3.1 Conventions Used Below in Descriptions of Operations

Names or descriptions in angle brackets represent values or objects matching the name or description. These may be literal, constructed via "new", or variables. Occasionally the same syntax may represent script performing an operation. Quoted angle brackets indicate a character string of the described type.

So the following sample operation:

<Foo> = new Foo( <FooFact> );
<Foo>.bar = <any value or object> ;
<Baz> = <Foo>.find("<name> ");
<find out what you have>

might be replaced in a real Javascript program:

var myFoo = new Foo(new FooFact("always","Great","Baz"));

             myFoo.bar = 3.14159265;

             var myGreatBaz = myFoo.find("Great");

             if (checkBaz(myGreatBaz)) {

               alert(myGreatBaz.description);

             }

where "myFoo" is a variable holding an object of type Foo and myGreatBaz is a variable holding an object of type Baz.

3.2 Basic Operations of SOAP

The following basic SOAP operations is nearly all that most users of SOAP need to do.

Basic Operation	How to Do It
Create a parameter block, setting the Javascript value and name for rpc-style call.	<SOAPParameter> = new SOAPParameter(<any value or object> , "<name>"); // or <SOAPParameter> = new SOAPParameter(); <SOAPParameter> .value = <any value or object> ; <SOAPParameter> .name = "<name>";
Set parameters in a Javascript array.	<SOAPParameter array> = new Array(<SOAPParameter> <,...>); // or <SOAPParameter array> = new Array(); <SOAPParameter array> [0] = <SOAPParameter> ; <...>
Create and encode the parameters in a basic SOAP 1.1 rpc-style message.	<SOAPCall> = new SOAPCall(); <SOAPCall> .transportURI = "<http-based service URI> " <SOAPCall> .encode(0, "<method name> ", " <target object namespaceURI> ", 0, null, <SOAPParameter array> .length, <SOAPParameter array> );
Invoke call (send call message and receive response message).	<SOAPResponse> = <SOAPCall> .invoke(); <process the response -- see below> // or <SOAPCall> .asyncInvoke( <SOAPResponseListener> );
Handle completion of async SOAP call.	function <SOAPResponseListener name>(<SOAPResponse> , <SOAPCall>, <error>) { if (error != 0) { <action to be taken on failure to transport message> } <process the response -- see below> }
Get service's failure, if any.	<SOAPFault> = <SOAPResponse> .fault; if (<SOAPFault> != null) { <namespace URI string> = <SOAPFault> .faultNamespace; <name string> = <SOAPFault> .faultCode; <summary string> = <SOAPFault> .faultString; <actor URI string> = <SOAPFault> .actorURI; <action to be taken in case of fault> }
Get returned parameters from rpc-style response .	<SOAPParameter array> = <SOAPResponse> .getParameters(true, {});
Process Javascript values, etc. of returned parameters.	for (i = 0; i != <SOAPParameter array>.length; i++) { <SOAPParameter> = <SOAPParameter array> [i]; <value or object> = <SOAPParameter> .value; <name string> = <SOAPParameter> .name; <checking and processing of result> }

3.3 Beyond Basic Operations

The above operations are what every user of the lowlevel SOAP toolkit needs to invoke service requests and interpret the responses, as is easily seen by looking at some of the samples (see section 5 ). The bulk of the operations that follow will be used less frequently, but they need to be there for certain cases. The casual reader may wish to skip the remaining tables of operations in this section, or scan for features of interest.

3.4 Header Operations

The user can send or receive header blocks for information not carried in the body of the message. Sending and receiving header blocks is not very different from sending and receiving parameters as described above.

Header Operation	How to Do It
Create a Header Block.	<SOAPHeaderBlock> = new SOAPHeaderBlock(<any value or object> , "<name> ", " <namespaceURI> "); // or <SOAPHeaderBlock> = new SOAPHeaderBlock(); <SOAPHeaderBlock> .value = <any value or object> ; <SOAPHeaderBlock> .name = "<name> "; <SOAPHeaderBlock> .namespaceURI = "<namespaceURI> ";
Establish non-default role of a header block .	<SOAPHeaderBlock> .actorURI = "<actorURI> "; <SOAPHeaderBlock> .mustUnderstand = <true or false> ;
Set header blocks in a Javascript array.	<SOAPHeaderBlock array> = new Array(<SOAPHeaderBlock> <,...>); // or <SOAPHeaderBlock array> = new Array(); <SOAPHeaderBlock array> [0] = <SOAPHeaderBlock> ; <...>
Encode the headers in a SOAP 1.1 rpc-style message.	<SOAPCall> .encode(0, "<method name> ", " <target object namespaceURI> ", <SOAPHeaderBlock array> .length, <SOAPHeaderBlock array> , <SOAPParameter array> .length, <SOAPParameter array> );
Get returned headers.	<SOAPHeaderBlock array> = <SOAPResponse> .getHeaderBlocks(true, {});
Process Javascript values, etc. of returned headers.	for (i = 0; i != <SOAPHeaderBlock array>.length; i++) { <SOAPHeaderBlock> = <SOAPHeaderBlock array> [i]; <value or object> = <SOAPHeaderBlock> .value; <name string> = <SOAPHeaderBlock> .name; <namespace URI string> = <SOAPHeaderBlock> .namespaceURI; <actor URI string> = <SOAPHeaderBlock> .actorURI; <must understand boolean> = <SOAPHeaderBlock> .mustUnderstand; <checking and processing of result> }

3.5 Non-RPC Operations

For messages that are not intended to model RPC calls, there is no method name or target object URI, and the parameters generally have namespaceURIs. Otherwise, the basic operations are the same.

Non-RPC Operation	How to Do It
Setting the namespaceURI of a non-RPC parameter .	<SOAPParameter> = new SOAPHeaderBlock(<any value or object> , "<name> ", " <namespaceURI> "); // or <SOAPParameter> .namespaceURI = "<namespaceURI> ";
Encode a SOAP 1.1 non-rpc-style message.	<SOAPCall> .encode(0, "", "", <header block array> .length, <header block array> , <parameter array>.length, <parameter array> )
Get returned parameters from non-rpc-style response.	<SOAPParameter array> = <SOAPResponse> .getParameters(false, {});

3.6 SOAPBlock and SOAPMessage Supertypes

In the following operations, SOAPHeaderBlock and SOAPParameter may be referred to collectively as SOAPBlock objects.

Also, SOAPCall and SOAPResponse may be referred to collectively as SOAPMessage objects.

3.7 More Operations

The following table contains less-common operations.

Operation	How to Do It
Set or get an actionURI carried for the message in the HTTP header.	<SOAPMessage> .actionURI = "<action URI> "; // or <action URI string> = <SOAPMessage> .actionURI;
Directly set the DOM element to represent the block's encoded content, bypassing encoding of the value on the block .	<SOAPBlock> .element = <DOM Element> ;
Directly get the DOM element that represents the block's encoded content, bypassing decoding of the value on the block .	<DOM Element> = <SOAPBlock> .element;
Directly get the DOM element containing the SOAP envelope , header, or body of an encoded message.	<DOM Element> = <SOAPMessage> .envelope; // OR <DOM Element> = <SOAPMessage> .header; // or <DOM Element> = <SOAPMessage> .body;
Directly set the DOM document to represent the message's entire encoded content, bypassing encoding.	<SOAPMessage> .message = <DOM Document> ;
Directly get the DOM document of an encoded message, bypassing encoding.	<DOM Document> = <SOAPMessage> .message;
Get the method name and target object URI, if any, of the message.	<method name string> = <SOAPMessage> .methodName; <object URI string> = <SOAPMessage> .targetObjectURI;
Get the actual SOAP version of an encoded message -- 0 for SOAP 1.1 and 1 for SOAP 1.2.	<version integer> = <SOAPMessage> .version;
Encode a SOAP 1.2 message.	<SOAPCall> .encode(1, "<method name> ", " <target object namespaceURI> ", <SOAPHeaderBlock array> .length, <SOAPHeaderBlock array> , <SOAPParameter array> .length, <SOAPParameter array> );
Abort an in-progress async call -- this does not necessarily cause the message not to be processed, but the API stops listening for it to complete.	<SOAPCallCompletion> = <SOAPCall> .asyncInvoke( <SOAPResponseListener>); <...> <SOAPCallCompletion> .abort();
Get the encoding (style) used to encode or decode message. Not available on an unencoded call unless explicitly set -- use following operation instead.	<SOAPEncoding> = <SOAPMessage> .encoding;
Set the primary encoding style used to encode a message. A new SOAPEncoding objects (created by new) starts a different set of associated encodings, which are only reached (or created) in association by calling getAssociatedEncoding().	<SOAPEncoding> = new SOAPEncoding(); <SOAPEncoding> = <SOAPEncoding> .getAssociatedEncoding(" <style URI> ",<true to create> ); <customize encodings> <SOAPMessage> .encoding = <SOAPEncoding> ;
Specify the encoding style used to encode or decode specific blocks.	<SOAPEncoding> = <SOAPEncoding> .getAssociatedEncoding(" <style URI> ",<true to create> ); <SOAPBlock> .encoding = <SOAPEncoding> ;

3.8 Using Schema Types

The default SOAP encodings implement most XML built-in types, as well as the basic SOAP types. In the absence of a specified type, native values and objects will typically be correctly identified and mapped to a corresponding schema type. There may be no perfect correspondence between Javascript and XML Schema types, so they will be mapped to a close corresponding type.

Providing specific schema types can help the encoding produce the desired results. For example, multidimensional arrays must be decoded as nested arrays because Javascript only supports single-dimensional arrays. If no schema type is given that identifies the array as multidimensional, then a multidimensional array will be encoded as a nested array. An accurate schema type can also help when encoding or decoding of other complex objects such as SOAP structs.

Schema types may be attached to blocks before encoding or before accessing the value of a returned object to better control the encoding and decoding. All schema types which are used to control the encoding and decoding should come from the schema collection available through the encoding (associated encodings share the same collection).

Schema Operation	How to Do It
Get the schema collection of all associated encodings.	<SchemaCollection> = <SOAPEncoding> .schemaCollection;
Load additional schema types from XML Schema files into the schema collection.	<SchemaLoader> = <SchemaCollection> ; // and then <SchemaLoader> .load(" <schema file URI> "); // or <SchemaLoader> .loadAsync(" <schemaURI> ", <load completion function> );
Specify the XML Schema type to be used when encoding or decoding a block. Note: decoding a block occurs when you get its value, so after getting a block from an XML-encoded message such as a SOAPResponse, attaching type info affects evaluation of the value.	<SchemaType> = <SchemaCollection> .getType(" <name> ", " <namespaceURI> "); if (<schemaType != null) { <SOAPBlock> .schemaType = <SchemaType> ; }

3.9 Customization of Encodings

A specific encoding must have encoders and decoders to function. Encoding or decoding of data always begins with a default encoder or decoder, which then may lookup additional encoders or decoders by a string key as required. For either the 1.1 or 1.2 version of the default SOAP encoding, the default encoder and decoder use the schema type's namespaceURI and name, seperated by "#" to look up additional decoders for specific schema types. Additional encoders and decoders registered within the default encodings will automatically be invoked as an object identified as the corresponding type is processed. Other encodings can use any scheme for looking up additional encoders and decoders, or none at all if all the work is done by the default encoder and decoder for that encoding. Encodings which are registered with the system, such as the default SOAP 1.1 or 1.2 encodings, automatically come with encoders and decoders built-in, whereas new encodings have none. Custom encodings may also reuse existing encoders and decoders, but there is no guarantee which are present, since the mapping may vary when handling an infinite set of types with a finite set of encoders and decoders.

Also, there has been a proliferation of different schema URIs to describe the same types, which may often even be intermixed in usage, but expected to function properly. Most notably, the SOAP 1.1 specification used unofficial XML Schema URIs and SOAP encoding schema URIs not compatible with those which are in the W3C XML Schema and drafts for SOAP 1.2 specifications. It is not uncommon to send and receive messages using the URIs specified in the SOAP 1.1 specification, but described by WSDL using XML Schema that uses the official, correct URIs. To solve these problems, the encoding permits schema URIs to be aliased, both on input and on output, so that only the SOAP 1.2 and official XMLSchema types are used internally, while supporting the other URIs. Mappings of this type are built-in for encodings which are registered with the system, such as the default encodings of SOAP 1.1 and 1.2. The default URI mappings may be manipulated, for example, to output SOAP 1.1 but with the official XML Schema URIs, without having to rewrite any encoders or decoders.

Encoding Customization Operation	How to Do It
Create a custom encoder.	function <New DOM Element> = <SOAPEncoder name> ( <SOAPEncoding>, <value> , <namespaceURI> , <name>, <SchemaType> , <SOAPAttachments> , <Parent DOM Element> ) { <...> <DOM Element> = <Parent DOM Element> .ownerDocument.createElementNS(namespaceURI,name); <...> <Parent DOM Element> .appendChild(<DOM Element> ); return <DOM Element> ; } // and <SOAPEncoding> .defaultEncoder = <SOAPEncoder> ; // or <SOAPEncoding> .setEncoder(" <namespaceURI> # <name> ", <SOAPEncoder> );
Create a custom decoder.	function <New DOM Element> = <SOAPDecoder name> ( <SOAPEncoding>, <DOM Element> , <SchemaType> , <SOAPAttachments>) { <...> return <value or object> ; } // and <SOAPEncoding> .defaultDecoder = <SOAPDecoder> ; // or <SOAPEncoding> .setDecoder(" <namespaceURI> # <name> ", <SOAPDecoder> );
Map or unmap schema URI aliases	<SOAPEncoding> .mapSchemaURI(" <external URI> ", "<internal URI>", <true to alias output> ); // or <SOAPEncoding> .unmapSchemaURI(" <external URI> ");
Register modified or alternative encodings, making them automatically available to all SOAP scripts in the system	Install an appropriate registerable encoding in `components/<new encoding> .js`

3.10 Security Operations

In browsers, the risk of allowing an externally-loaded untrusted script to request information within a firewall and send it elsewhere has lead to very tight sandbox restrictions, permitting external browser scripts to only request xml data and services on the same domain from which the script was loaded. This same restriction applies by default to SOAP requests executed within the browser. This means that an externally-loaded script cannot, for example, call other external services unless they are in the same domain from which the page was loaded. Even if the page was loaded from the user's own hard disk, the script must ask for permission to make SOAP calls. A browser enhancement is planned to permit more-precise control of trust between scripts and specific available services.

Since SOAP permits headers to be added to messages that require interpretation by the recipient, this API can request a header to warn the recipient that it was sent by an untrusted script loaded from a specific sourceURI, and no good SOAP service will unintentionally disregard the warning. If the envelope is verified and the header is added, then the browser can allow the script less-restricted access to services outside of its source domain. Accepting this header permits SOAP services that really do want to be universally available to allow access without forcing the user to risk breach of the firewall protections and/or request user permission.

Security Operation	How to Do It
Mark the call with a verifySourceHeader so, if the service permits it, the browser can make the call with less privilege and risk.	<SOAPCall> .verifySourceHeader = true;
Request risky privileges within a local or signed script to make an unverified SOAP calls to other domains.	netscape.security.PrivilegeManager.enablePrivilege("UniversalBrowserRead")
Modify the security settings in the preferences file, allowing scripts from some domain to make risky SOAP calls to any other domain, which is disabled by default.	Add the setting in `default/pref/all.js` : pref("<some domain prefix> .SOAPCall.invoke","allAccess");
Modify the security settings in the preferences file to disallow even dross-domain calls made with verifySource header, which is generally permitted by default.	Remove or comment out the setting in `default/pref/all.js` : pref("capability.policy.default.SOAPCall.invokeVerifySourceHeader","allAccess");
Register alternative transport mechanisms, making available alternative transports to all scripts and perhaps creating alternative security models for protocols besides http(s). See the futures section for more info on possible transports.	Install an appropriate registerable encoding in `components/<new transport> .js`.

4 Future Features

4.1 Access to SOAP as Proxies

Although a SOAP call can generally be accomplished using this low-level API in a few dozen lines, WSDL is a standard that contains enough information to enable this to occur with no manual argument and type setup required. An implementation is under development that instantiates web service proxies complete with appropriate xpconnect interfaces by simply loading and using information out of a WSDL file that describes the service. The proxy's interface has dynamically-generated methods named appropriately to match the services described in the WSDL file which accept arguments of the appropriate data types and calls the appropriate low-level SOAP functions with the appropriate type information, making it even simpler not only to invoke services from Javascript code, but to associate appropriate schema types loaded from the WSDL file with the arguments. This higher level is not available in the first release. When it is available, invoking WSDL-described features gets even easier and more reliable.

4.2 Arbitrary Graphs of Data

The SOAP specification allows objects to be passed as arguments which may have originally referenced other objects that are not owned in a pure hierarchy. This is represented by using an href attribute. Due to the problems with leaking reference counts in COM objects with cyclic references, this has not been implemented yet. Also, the output of a cyclic-referencing set of objects has not been implemented. Outputting of objects that do not have cyclic references currently creates separate copies for each reference to an object, and with cycles output may never complete. On input, hrefs are currently ignored. In the future it may be possible to solve this and transmit and receive arbitrarily-referencing objects, but the solution is more complex than just using weak references.

4.3 SOAP With Attachments

Many clients and servers now support automatically transmitting large Mime with a SOAP message by encapsulating it in MIME, DIME, or other enveloping formats. This has been anticipated in the APIs, but the SOAPAttachments API is currently a place holder for this future feature which is not yet implemented.

4.4 New Transports and Local Services

Obvious new transports that would be useful include e-mail -- permitting a SOAP exchange to occur as an email exchange --, instant messenger for peer to peer, and a local manager with a controlled security model but without the size limitations, enabling SOAP to save and restore arbitrary Javascript application data on the client. These services require a framework, already being planned, for permitting the browser to host services as well as being a good client. There are obviously security issues to be solved to make these successful.

4.5 Standards

The interfaces to the objects of this API were designed to be as simple and universal as possible. We believe that we should submit a note describing them and sponsor a W3C proposal to standardize an API for invoking this type of service from web clients. In such an effort, changes would be inevitable and welcomed as the price for interoperability within web clients. Part of this API are incomplete, specifically the SOAPAttachments object, which will be defined to allow encoders and decoders to control uniqueness and referencing, both for resolving arbitrary graphs of data (as described above in "Arbitrary Graphs of Data") as well as for automatically resolving references to attached objects carried with the message in an external encapsulation (as described above in "SOAP With Attachments").

5 Samples and Testing

Tests and samples exist today can be found in the mozilla tree at mozilla/extensions/xmlextras/tests/soap*.html. We welcome the contribution of tests and samples by other parties. Please note that these files do not work in the linked location due to js files being shown as HTML cross-referenced files, so they must be copied from the current mozilla tree to the client's local harddrive or some web server before testing them.

To test the services, it is interesting to turn on MOZ_DEBUG=1, because even if you do not debug the program, this causes the exchanged SOAP messages to be displayed on the standard output if you run from a command shell where this is visible.

A SOAP service server has been set up at ray.dsl.xmission.com where additional services may be deployed to help test and demo the features for scripting SOAP in Mozilla. No home page describes the Apache Jakarta server on port 8080 which is the test facility, but this is currently used in two tests in the previously-mentioned directory: soapisprimenumber.html and soapunscramble.html. Adding services is as easy as writing functions in Javascript, Java, or other supported languages and describing the deployment in XML. Disclaimer: as this is merely an end-subscriber DSL service, if the server becomes too widely published, it may have to be shut down.

SOAP Bugs should be reported using bugzilla.mozilla.org, as a new bug on Browser, specifying the XML component, and assigned to rayw@netscape.com , which is the email address you can use to contact me directly if you have an issue that cannot be solved in the newsgroup netscape.public.mozilla.xml or by filing a bug.