Package Developer's Guide to Workflow

By Lars Pind

Introduction

The workflow package manages a collaborative process surrounding some object.

Examples of the object could be a bug in a bug tracker, a ticket in a ticket tracker, a content item in a content management system, a user contribution in a moderated forum/comments/whatever application, a user's request for particpation in a course/event/whatever.

For example, when a new bug is submitted, someone's assigned to fix it, and whoever submitted it is assigned to verify the fix and close the bug. Once the bug's fixed, the submitter will get notified, and the bug will wait in the 'resolved' state until the submitter has verified and then closed the bug.

1. Define your default process. The idea typically is to allow your end users to modify the process to suit their needs, but you'll want eto provide a process which they can use as a starting point.
2. Identify, declare, and implement the callbacks that your application will need.
3. Write the code to set up the initial process, and to clone that process for each package instance.
4. Integrate workflow support into your application's API.
5. Integrate workflow support into your application's user interface.
6. Integrate workflow into your application's queries

Let's first look at some concepts before getting into the technicalities of how you actually do this. For a working example of building workflow-based applications, we recommend you take a look at bug-tracker.

Workflow Concepts

What's in a workflow

In its broadest, most conceptual sense, a workflow is defined as (and this does get a little hairy, so feel free to skip if you just want to start developing your application):

• A number of actions that can be executed (open, edit, comment, resolve, edit, publish, approve, reject, etc.) Whether or not an action is enabled, meaning it can be executed at this point in time, will depend on the current state of the workflow. Executing an action may change the state of the workflow.
• A definition of the possible states that the workflow can be in. This could simply be an enumeration of states, such as "Open", "Resolved", "Closed", in which case we're talking about a "Finite State Machine" (because there's a finite number of possible states). Other models, such as petri nets, have an infinite number of possible states. This makes things a lot more complicated, and you don't need to bother since this first implementation of workflow only supports finite state machines.
• People participate in a workflow through their role. For a bug tracker, the roles could be "Submitter", "Resolver", and "Tester". For a publication workflow, they'd be "Author", "Editor", and "Publisher". Actions will be to roles, which in turn will be linked to users or groups of users in a particular workflow case. The reason we link people to actions through the concept of roles is to allow you to specify that two or more actions should be done by the same party. In bug-tracker, for example, the same user who opened the bug should also close it.

Finite State Machines (FSMs)

As mentioned, workflow is designed to support workflows based on any model, however the only model currently implemented is the finite state machine.

The workflow API is designed so that whenever you're using features that are specific to finite state machines, the procedure name will include the letters "fsm" in the name, as in workflow::case::fsm::get. That way you'll know when you're hard-wiring a dependency on the particular workflow model.

It's considered good practice to only use non-fsm API calls, but in practice, it can be hard to create good user experiences without. So feel free.

Notation:

[Action] (State) {Role}

(State1) -> [Action1] -> (State2) -> [Action2] -> (State3)

Cases

So much for defining the workflow. When you start "running" your workflow, that's called a workflow case, or simply a case. A case is concerned with a particular object, it's always in a particular state, and it has specific people or groups assigned to its different roles.

In-flow and out-of-flow

When defining actions, we differentiate between in-flow and out-of-flow. In-flow refers to the normal idealized flow of the workflow, out-of-flow are the rest. Concretely what it means is that if you're assigned to an in-flow action, we'll bug you about it through notifications, and potentially get mad at you if you don't come and do something to get the workflow moving along. We don't do that with out-of-flow actions. So we'll send a notification that says "Please come back and resolve this bug", whereas we'll not notify everybody who are allowed to comment saying "Please come back and comment on this bug".

For bug-tracker, the normal flow (in-flow) is this:

(Open) -> [Resolve] -> (Resolved) -> [Close] -> (Closed)

Other actions not in the normal flow are [Edit] and [Comment], which are always enabled, but never change the state. And [Reopen] which throw you back to the (Open) state. And finally [Resolved] is in-flow when in the (Open) state, but out-of-flow when in the (Resolved) state, meaning that you can re-resolve a bug if you need to, but you're not required to.

In-flow and out-of-flow depends on the action, the state, and the user's role in the case. For example, it might be that users in the {Submitter} role are allowed to resolve their own bugs, but the [Resolve] action will still only be considered in-flow to people in the {Assignee} or {Resolver} role.

The Six Steps Conceptually

The recommended way a workflow is linked to an application is this: As part of developing your application, you define your default workflow, which will be used as a template for customization by the users of your applications. This workflow will be installed using the APM after-install callback, and will be linked to your application's package-key.

Then when a new instance of your application is created, your default workflow will be cloned, and the clone linked to the new instance, so that your users can customize the workflow for each instance of your application individually. The default copy installed along with your package is never actually used except for cloning when creating a new instance. This means that your users can customize this default workflow, and the modified version will serve as the boilerplate for all new package instances.

In order to integrate workflow with your application, you'll want to implement one or more of the callback service contracts. These can do things like determine default assignees based on certain data in your application, get information about your application's object for use when sending out notifications, or perform side-effects, such as actually changing the publication state of a content item when you execute the [Publish] action.

When integrating the workflow with your application's user experience, what workflow will give you is essentially the list of actions that the given user can perform on the given object at the given time. In bug-tracker, for example, bug-tracker takes care of displaying the form displaying and editing a bug, while workflow takes care of displaying the buttons that say [Comment], [Edit], [Resolve], [Reopen], [Close], etc., along the bottom of the form. Workflow also has a place to store which form elements should be opened for editing depending on the action being executed.

Your application should typically have an API for creating a new object, editing an object, etc. This application object API will need to be workflow-aware, so when a new object is created, a new workflow case will be started as well. And when the object's edited, that should generally only happen through a workflow action, so that needs to be taken into account as well.

The final step is integrating workflow into your application's queries when you want to filter an object listing/count based on the workflow state. This is the only place where you'll directly be dependent on the workflow data model.

Defining Your Process (FSM)

The current version of workflow only supports workflows based on a finite state machine (FSM). Support for other models, such as petri nets and directed graphs are possible, but not currently designed or implemented.

An FSM-based workflow consists of a set of states, actions, and roles.

You define a new workflow like this:

set spec {
    workflow-short-name {
        ...
        roles {
            role-short-name {
               ...
            }
            ...
        }
        states {
            state-short-name {
               ...
            }
            ...
        }
        actions {
            action-short-name {
               ...
            }
            ...
        }
    }
}

set workflow_id [workflow::fsm::new_from_spec -spec $spec]

All the items (workflow, roles, states, actions) have a short-name, which should be lowercase and use underbar (_) instead of spaces. These are mainly used to refer to the items in other parts of the spec.

The workflow short name can be used to refer to the workflow in your application, which is useful if you have several different workflows. The bug-tracker, for example, could have a workflow for bugs and another one for patches.

Finally, you can also refer states, roles, and actions in your application using short names, although this creates a dependency in your application on a particular form of the workflow, and there's currently no mechanism for ensuring that your workflow contains the states, roles, and actions you'd refer to. This is on the todo-list.

Workflow

These are the attributes you can specify for the workflow itself:

Internationalization Note:

When we make workflow internationalized for OpenACS 5.0, pretty names will contain message keys in the form "#message-key#". More about this in the package developer's guide to internationalization.

Roles

Attributes for roles:

States

A few typical examples of states:

These are the state attributes in the workflow specification:

Actions

Actions are what the workflow allows you to do to your object.

Terminology:

Enabled

The action is allowed to be executed in the workflow's current state.

Allowed

The given user is allowed to execute the action given his current relationship to the workflow case and the object.

Assigned

The same as allowed, but the action is in-flow for this user.

Available

The action is both enabled and allowed for this user.

Some actions will always be enabled. In bug-tracker, for example, we have [Comment] and [Edit] actions, which are always allowed, regardless of whether the bug is (Open), (Resolved), or (Closed).

Other actions, however, will only be enabled in certain states. In bug-tracker, for example, the [Close] action is only available in the (Resolved) state.

Another distinction is that some actions change the state, and others do not. [Comment] and [Edit], for example, do not. [Resolve], [Close], and [Reopen] do. For an FSM, when an action changes the state, you simply specify what the new state should be.

There's a special action called the initial action. This is implicitly executed when a new case is started for this workflow, and must always specify the "new_state" attribute to define which state new cases start out in.

Attributes for actions:

Putting A Workflow Together

When you put this all together, here's a real live example of what defining a workflow could look like:

ad_proc -private bug_tracker::bug::workflow_create {} {
    Create the 'bug' workflow for bug-tracker
} {
    set spec {
        bug {
            pretty_name "Bug"
            package_key "bug-tracker"
            object_type "bt_bug"
            callbacks { 
                bug-tracker.FormatLogTitle 
                bug-tracker.BugNotificationInfo
            }
            roles {
                submitter {
                    pretty_name "Submitter"
                    callbacks { 
                        workflow.Role_DefaultAssignees_CreationUser
                    }
                }
                assignee {
                    pretty_name "Assignee"
                    callbacks {
                        bug-tracker.ComponentMaintainer
                        bug-tracker.ProjectMaintainer
                        workflow.Role_PickList_CurrentAssignees
                        workflow.Role_AssigneeSubquery_RegisteredUsers
                    }
                }
            }
            states {
                open {
                    pretty_name "Open"
                    hide_fields { resolution fixed_in_version }
                }
                resolved {
                    pretty_name "Resolved"
                }
                closed {
                    pretty_name "Closed"
                }
            }
            actions {
                open {
                    pretty_name "Open"
                    pretty_past_tense "Opened"
                    new_state "open"
                    initial_action_p t
                }
                comment {
                    pretty_name "Comment"
                    pretty_past_tense "Commented"
                    allowed_roles { submitter assignee }
                    privileges { read write }
                    always_enabled_p t
                }
                edit {
                    pretty_name "Edit"
                    pretty_past_tense "Edited"
                    allowed_roles { submitter assignee }
                    privileges { write }
                    always_enabled_p t
                    edit_fields { 
                        component_id 
                        summary 
                        found_in_version
                        role_assignee
                        fix_for_version
                        resolution 
                        fixed_in_version 
                    }
                }
                reassign {
                    pretty_name "Reassign"
                    pretty_past_tense "Reassigned"
                    allowed_role { submitter assignee }
                    privileges { write }
                    enabled_states { resolved }
                    assigned_states { open }
                    edit_fields { role_assignee }
                }
                resolve {
                    pretty_name "Resolve"
                    pretty_past_tense "Resolved"
                    assigned_role "assignee"
                    enabled_states { resolved }
                    assigned_states { open }
                    new_state "resolved"
                    privileges { write }
                    edit_fields { resolution fixed_in_version }
                    callbacks { bug-tracker.CaptureResolutionCode }
                }
                close {
                    pretty_name "Close"
                    pretty_past_tense "Closed"
                    assigned_role "submitter"
                    assigned_states { resolved }
                    new_state "closed"
                    privileges { write }
                }
                reopen {
                    pretty_name "Reopen"
                    pretty_past_tense "Reopened"
                    allowed_roles { submitter }
                    enabled_states { resolved closed }
                    new_state "open"
                    privileges { write }
                }
            }
        }
    }

    set workflow_id [workflow::fsm::new_from_spec -spec $spec]
    
    return $workflow_id
}

Defining Callbacks

There are a number of different types of callbacks, each of which applies to different workflow items (workflows, roles, states, actions). They're all defined as service contracts.

In order to make use of them, your application will need to implement these service contracts, and register the implementation with the relevant workflow item through the 'callbacks' attribute in the spec above.

Here are the types of callbacks defined, how they're used, and the workflow items they apply to.

All the service contracts have 3 operations each. The first two are the same for all service contracts, and they really just act like static variables:

The third operation is the one that does the real work. Here are the inputs and outputs:

For the most up-to-date information about the service contracts, your safest bet is to refer to the user-visible pages of the acs-service-contract package itself, which will let you view your currently installed contracts and implementations.

Log Entry Data and Log Entry Titles

One noteworthy thing that side-effects can be used for, is to record information about a log entry for use later in displaying a more detailed log entry title, or can be used to e.g. tie a workflow log entry to a particular content repository revision, etc.

Using workflow::case::add_log_data, you can add arbitrary key/value pairs to a log entry. These can the be retrieved later using workflow::case::get_log_data_by_key, and workflow::case::get_log_data.

Installing and Instantiating (APM Tcl Callbacks)

Here are the workflow-related operations that you'll typically want your application to do from the APM Tcl Callbacks:

after-install

• Register service contract implementations
• Create default workflow (workflow::fsm::new_from_spec)

before-uninstall

• Delete default workflow (workflow::delete)
• Unregister service contract implementations

before-upgrade

• Add new service contract implementations
• Add new workflows
• Make changes to existing default workflows (if the installed version is not modified) (this isn't yet supported on the workflow API)

after-instantiate

• Clone default workflow to create a new workflow attached to the instance (workflow::fsm::clone)

before-uninstantiate

• Delete the workflow attached to the instance (workflow::delete)

To see what this could look like in practice, check out packages/bug-tracker/tcl/install-procs.tcl.

Integrating With Your Application's API

Newer applications will define a namespace for each of the objects it defines, which will contain procs like "get", "new", "delete", etc., to manipulate these objects.

Given such a setup, here are the procs that you want to integrate workflow into for your workflow-integrated objects. For a real-life example, see packages/bug-tracker/tcl/bug-procs.tcl and search for "workflow::".

get

In addition to your application's object data, you'll want to call workflow::case::get_id to get the case_id for your object, and then either workflow::case::get or workflow::case::fsm::get in order to get state information from workflow to include in the data set returned by your API proc.

new

When creating a new object, you should also start a new workflow case for that object using workflow::case::new.

edit

Editing an object should only happen through a workflow action, if you want to have a complete audit log (workflow case log). Thus, your edit proc should take the following arguments, in addition to the object_id and the array containing the object data:

      -action_id:required
      -comment:required
      -comment_format:required
      {-entry_id {}}
    

(entry_id is for double-click protection).

First, you should update your application object data as normal. Then you'll probably want to use workflow::case::get_id to find the case_id. If you have assignment integrated in your form, you'll want to call workflow::case::role::assign to pass these on to workflow, and finally you'll say workflow::case::action::execute to execute the action, including state changes, side-effects, and notifications.

Integrating With Your Application's User Interface

Usually, you'll want one page that lists all the objects in your package instance, and another that lets the user view/edit one object, called the object form page. This section is about the object form page, the next section is about the object listing page.

For a real-life example, look at packages/bug-tracker/www/bug.tcl. You may want to have that handy while reading through this.

We're hoping to make some streamlining of both ad_form and workflow to make this form page integration even easier at some point. But no promises.

Use workflow::case::get_id to get the case_id.

If you want buttons along the bottom of the form like bug-tracker, use the new 'action' feature of the form builder. What you do is pass a list of possible actions to the form builder as a list-of-lists with { label value }. These will be displayed as buttons at the bottom of the form.

When one of these buttons are clicked, the form will be in edit-mode, and you can use form get_action to get the value of the action chosen.

So up top, you'll want to ask the form if an action is in progress, and which one it is, by saying set action_id [form get_action form-id]. If no action is in progress this will return the empty string.

Then you should check that this action is currently available to this user by saying workflow::case::action::available_p.

To get the currently available actions so you can offer them to the user, use workflow::case::get_available_actions which will return the action_id's, then workflow::action::get to get the details about each of the actions.

If you're using ad_form, and you want only one assignee per role, and you want assignment integrated with your form, use workflow::case::role::add_assignee_widgets to add the widgets. It'll do an ad_form -extend, so they'll appear at the point at which you call this proc.

To set the editable fields as defined for the action, do this:

if { ![empty_string_p $action_id] } {
    foreach field [workflow::action::get_element -action_id $action_id -element edit_fields] { 
        element set_properties bug $field -mode edit 
    }
}

Similarly, on submit, you'll want to set the value of the editable fields.

To populate values of the assignee widgets, use workflow::case::role::set_assignee_values in your on_request block.

To add the case log to the comment field, use workflow::case::get_activity_html and feed it to the before_html property of a textarea.

Integrating With Your Application's Queries

Here's an example of how the bug-tracker integrates with workflow for information about the current state of bugs.

select b.bug_id,
       ...

       st.pretty_name as pretty_state,
       st.short_name as state_short_name,
       st.state_id,
       st.hide_fields,

       assignee.party_id as assignee_party_id,
       assignee.email as assignee_email,
       assignee.name as assignee_name

from   bt_bugs b,
       workflow_cases cas left outer join
       (select rpm.case_id,
               p.party_id,
               p.email,
               acs_object__name(p.party_id) as name
          from workflow_case_role_party_map rpm,
               parties p
         where rpm.role_id = :action_role
           and p.party_id = rpm.party_id
         ) assignee on (cas.case_id = assignee.case_id),
       workflow_case_fsm cfsm,
       workflow_fsm_states st 
where  cas.workflow_id = :workflow_id
and    cas.object_id = b.bug_id
and    cfsm.case_id = cas.case_id
and    st.state_id = cfsm.current_state
and    b.project_id = :package_id
order  by $order_by_clause

Note the outer join to get the assignee(s). The joins to get information about the current state is straight-forward.

Good Luck!

That's all I think you'll need to know to start developing workflow-enabled applications.

Let me know how it goes, or of something's missing, by posting on the OpenACS Forums.