XML-RPC Server Documentation

Overview

XML-RPC is a protocol which allows clients to make remote procedure calls on your server. Data is transformed into a standard XML format before being transferred between the client and server. This allows software using different OS's and programming languages to interact. See XML-RPC.com for more information.

Why do we need it in OpenACS?

Some XML-RPC protocols have become popular in the web world. The Blogger API and the Metaweblog API allow users to manage their blogs using tools of their choice and these have become widespread enough that users expect to find this functionality in any blogging software. For this reason, it's important to provide this minimum of functionality.

User Documentation

There are no user-facing pages. XML-RPC client software may require the user to know the URL that is accepting XML-RPC requests, which is admin-definable (default: http://example.com/RPC2/). XML-RPC savvy users can call the XML-RPC method system.listMethods to see which methods the server supports.

Admin Documentation

The server is installed by default at /RPC2/. Administrators can change this by unmounting the package and remounting it at the desired URL. The server can be disabled or enabled via the /admin pages.

The XML-RPC folks have defined a standard validation suite. These tests are implemented in the Automated Testing package, so admins can test their server against this suite (locally) by running all the automated tests. They can also go to http://validator.xmlrpc.com to test their site's validity (remotely) from there.

Developer Documentation

Adding XML-RPC support to your package

The first thing you need to do is write the methods that you want to be available via XML-RPC. They should be defined as ad_procs just as any other OpenACS procs except for 2 differences.

  1. They need to be able to accept arguments as sent to them from xmlrpc::decode

    In XML-RPC, every value has a datatype. Since TCL is a weakly-typed language, we could care less about the datatype (for the most part). So for scalar values (int, boolean, string, double, dateTime.iso8601, base64), xmlrpc::decode simply sends along the value to your proc. To recap, for scalar values, you need to do nothing special. For the 2 complex types (structs and arrays), the values are sent to your proc as TCL structures - XML-RPC structs are sent TCL arrays and XML-RPC arrays are sent as TCL lists. For example, if your proc expects a struct with 3 members (name, address and phone), then this is how the beginning of your proc will look. 

      array set user_info $struct
  set name $user_info(name)
  set address $user_info(address)
  set phone $user_info(phone)
    Or if your proc expects an array with n integers, which it then sums, then this is how your proc will look. 
      foreach num $array {
      incr sum $num
  }

  2. They need to be able to return data that xmlrpc::respond will be able to translate to XML.

    Scalar data should be returned as a 2 item list {-datatype value}. So if your proc returns an int, its last statement might be: 

      return [list -int $result]
    Returning complex data structures (struct, array) is a little more *ahem* complex. One of the confusing things is the terminology. As I noted above, XML-RPC arrays are equivalent to TCL lists and XML-RPC structs are equivalent to TCL arrays. The other confusing thing is that XML-RPC is strongly typed and TCL isn't, so when you're converting from TCL to XML-RPC, you need to add the datatype for each scalar value.
    • Returning an array of mixed type 
          return [list -array [list [list -int 36] [list -string "foo"]]]
    • Returning a struct (foo=22, bar=blah) 
          return [list -struct [list foo [list -int 22] bar [list -string blah]]]
    • Returning the above struct using a TCL array 
          set my_struct(foo) [list -int 22]
    set my_struct(bar) [list -string blah]
    return [list -struct [array get my_struct]]
    • Returning an array of structs 
          set user1(name) {-string "George Bush"} 
    set user1(id) {-int 41}
    set user2(name) {-string "Bill Clinton"} 
    set user2(id) {-int 42}
    return [list -array [list 
                             [list -struct [array get user1]] 
                             [list -struct [array get user2]]]]

Once your procs are defined in packagekey/tcl/foo-procs.tcl, register them in packagekey/tcl/foo-init.tcl. The *-init.tcl files are loaded after all the *-procs.tcl files have been loaded, so xmlrpc::register_proc will be available if the xmlrpc package is installed. Make sure you add the xmlrpc package as a dependency of your package if you register any XML-RPC procs. If you don't want your package to depend on xmlrpc, you can test for the existence of the xmlrpc_procs nsv before calling xmlrpc::register_proc

This registers 'system.listMethods' 

xmlrpc::register_proc system.listMethods

Implementation details

Here is the sequence of events in an XML-RPC call. See the documentation for each proc for more details.

  1. A POST request is made to your XML-RPC URL.
  2. The xmlrpc::get_content proc grabs the content of the POST request. This is a bit of a hack to cover the fact that there is no ns_conn content proc.
  3. xmlrpc::invoke is called to process the XML request.
  4. If the server is disabled, a fault is returned
  5. The XML is parsed for the methodName and arguments. xmlrpc::decode_value decodes the XML-RPC params into TCL variables.
  6. xmlrpc::invoke_method checks to be sure the method is registered and then attempts to call the OpenACS proc
  7. xmlrpc::invoke catches any errors from this attempt and creates a XML-RPC fault to return to the client if so. If there was no error, then xmlrpc::respond is called to format the result as a XML-RPC response.
  8. xmlrpc::construct does the heavy work of converting the TCL results back into valid XML-RPC params
  9. Finally, if no errors occur in this process, the result is returned to the client as text/xml

More details are provided in the ad_proc documentation for each proc.

XML-RPC client

This package also implements a simple XML-RPC client. Any package that needs to make XML-RPC calls can simply add a dependency to this package and then call xmlrpc::remote_call. As an example, the system.add method sums a variable number of ints. To call the system.add method on http://example.com/RPC2, do this: 

catch {xmlrpc::remote_call http://example.com/RPC2 system.add -int 4 -int 44 -int 23} result
set result ==> 71

It's important to always catch outgoing XML-RPC calls. If there's an error, it will be written to the catch variable (result in the example above). If there's no error, then the return value will be in result.

Implementation detail: The client needs to be able to POST requests to other servers. The util_httppost proc in acs-tcl/tcl/utilities-procs.tcl doesn't work because it doesn't let you specify the Content-Type, which needs to be text/xml, and it doesn't add Host headers, which are required if the server you're POSTing to is using virtual hosting. So, this package implements its own HTTP POST proc (which was stolen from lars-blogger's weblogs.com XML-RPC ping).

History of XML-RPC in OpenACS

The first implementation of XML-RPC for AOLServer was ns_xmlrpc, whose credits state:

Ns_xml conversion by Dave Bauer (dave at thedesignexperience.org) with help from Jerry Asher (jerry at theashergroup.com). This code is based on the original Tcl-RPC by Steve Ball with contributions by Aaron Swartz. The original Tcl-RPC uses TclXML and TclDOM to parse the XML. It works fine but since OpenACS-4 will use ns_xml I converted it.

I took this version and converted it into a OpenACS service package. All of the xml procs now use the XML abstraction procs inside acs-tcl (which currently use tDOM). All the procs are in a xmlrpc:: namespace and documentation has been added. I added support for some standard XML-RPC reserved procs (system.listMethods, system.methodHelp, system.multicall). I changed the semantics slightly in one area. XML-RPC arrays were being converted to TCL arrays, with the name of each item being an integer index. I thought it made more sense to make these TCL lists (since that is what a TCL list is anyways). It makes the code more consistent and makes it easier to understand how to deal with XML-RPC datatypes.

XML-RPC struct = TCL array.
XML-RPC array = TCL list.

ChangeLog

Vinod Kurup