IMS Sync driver design

Prev	Design

by Lars Pind
OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

TODO

We need examples of how the communication would be done from our clients.

The "GetDocument" communications service contract could be a generic system-wide service contract.

We might need a source/ID column in the users table to identify where they're imported from for doing updates, particularly if importing from multiple sources (or when some users are local.)

Execution Story

We will parse a document in the IMS Enterprise Specification format (example XML document), and translate it into calls to the batch user sync API.
The document will contain either the complete user listitemst (IMS: "snapshot"), or an incremental user listitemst (IMS: "Event Driven" -- contains only adds, edits, and deletes). You could for example do a complete transfer once a month, and incrementals every night. The invocation should decide which type is returned.

The design should favor interoperability, reliability and robustness.

<enterprise>

  <properties>
    <datasource>Dunelm Services Limited</datasource>
    <target>Telecommunications LMS</target>
    <type>DATABASE UPDATE</type>

    <datetime>2001-08-08</datetime>
  </properties>
  <person recstatus = "1">
    <comments>Add a new Person record.</comments>
    <sourcedid>

      <source>Dunelm Services Limited</source>
      <id>CK1</id>
    </sourcedid>
    <name>
      <fn>Clark Kent</fn>

      <sort>Kent, C</sort>
      <nickname>Superman</nickname>
    </name>
    <demographics>
      <gender>2</gender>

    </demographics>
    <adr>
      <extadd>The Daily Planet</extadd>
      <locality>Metropolis</locality>
      <country>USA</country>

    </adr>
  </person>
  <person recstatus = "2">
    <comments>Update a previously created record.</comments>
    <sourcedid>

      <source>Dunelm Services Limited</source>
      <id>CS1</id>
    </sourcedid>
    <name>
      <fn>Colin Smythe</fn>

      <sort>Smythe, C</sort>
      <nickname>Colin</nickname>
      <n>
        <family>Smythe</family>

        <given>Colin</given>
        <other>Manfred</other>
        <other>Wingarde</other>
        <prefix>Dr.</prefix>

        <suffix>C.Eng</suffix>
        <partname partnametype = "Initials">C.M.W.</partname>
      </n>
    </name>
    <demographics>

      <gender>2</gender>
      <bday>1958-02-18</bday>
      <disability>None.</disability>
    </demographics>

    <email>colin@dunelm.com</email>
    <url>http://www.dunelm.com</url>
    <tel teltype = "Mobile">4477932335019</tel>
    <adr>

      <extadd>Dunelm Services Limited</extadd>
      <street>34 Acorn Drive</street>
      <street>Stannington</street>
      <locality> Sheffield</locality>

      <region>S.Yorks</region>
      <pcode>S7 6WA</pcode>
      <country>UK</country>
    </adr>

    <photo imgtype = "gif">
      <extref>http://www.dunelm.com/staff/colin2.gif</extref>
    </photo>
    <institutionrole primaryrole = "No" institutionroletype = "Alumni"/>
    <datasource>dunelm:colinsmythe:1</datasource>

  </person>
  <person recstatus = "3">
    <comments>Delete this record.</comments>
    <sourcedid>
      <source>Dunelm Services Limited</source>

      <id>LL1</id>
    </sourcedid>
    <name>
      <fn>Lois Lane</fn>
      <sort>Lane, L</sort>

    </name>
  </person>
</enterprise>

Above would get translated into calls to the batch sync API as follows:

for { ... loop over persons in the document ... } {
        auth::batch::transaction \
            -job_id $job_id \
            -operation [ad_decode $recstatus 2 "update" 3 "delete" "insert"] \
            -authority_id $authority_id \
            -username { $userid if present, otherwise $sourcedid.id } \
            -first_names { $name.given if present, otherwise all except last part of $name.fn } \
            -last_name { $name.family if present, otherwise last part of $name.fn } \
            -email { $person.email ; we require this, even though the specification does not } \
            -url { $url, if present } \
            -portrait_url { $photo.imgtype/$photo.extref -- grab photo, store in DB }
    }
}

Mandatory fields which we can rely on are:

sourcedid: ID as defined by the source system. Used for username.
name.fn (formatted name). Used for first_names, last_name

Note that we require 'email' attribute, but the IMS Enterprise spec does not. Hence, unless we change our data model to allow users without an email address, we will have to throw an error.

Here's how we map IMS enterprise to OpenACS tables.

username:
1. <userid> ... </userid> if present.
2. <sourcedid><id> ... </id></sourcedid> otherwise
first_names:
1. <name><given> ...</given></name> if present.
2. <name><fn> ... ...</fn></name> otherwise
last_name:
1. <name><family> ...</family></name> if present.
2. <name><fn>... ...</fn></name> otherwise
email:
1. <email> ...</email> if present.
2. Blank/unchanged if not.
url:
1. <url> ...</url> if present.
2. Blank/unchanged if not.
portrait:
1. <photo imgtype="gif"><extref>...</extref></photo> if present: HTTP GET the photo, insert it into the system. (Do we do this, then, with all users when doing a snapshot update?)

Resources

Consolidation before the leap; IMS Enterprise 1.1: This article says that IMS Enterprise 1.1 (current version) does not address the communication model, which is critically missing for real seamless interoperability. IMS Enterprise 2.0 will address this, but Blackboard, who's influential in the IMS committee, is adopting OKI's programming interrfaces for this.
IMS and OKI, the wire and the socket