By Lars Pind
I recently built a typical workflow-based bug-tracker, and decided against using the acs-workflow package that I myself built. That's not a good recommendation. We need to fix that.
The goal is to implement a workflow package that:
Gripes with the current acs-workflow:
Take bug-tracker as an example. The bug-tracker workflow and user interface can be defined as:
I've finally come to the realization that we'll be better off in the short to medium term with just a well-functioning implementation of a finite state machine based workflow module. In general, a workflow consists of a finite set of states, and a finite set of actions. Each action has a set of states in which it's enabled, or it can be always enabled in all states. And each action can cause the workflow case to move into a new state, or it can leave the state unaltered.
Note that the ability to have an action enabled in more than one state is a convenience, and not part of the mathematical model of finite state machines. Likeways with actions that don't change the state. But it's mighty convenient, as you've seen illustrated by the bug-tracker example above.
A workflow is a set of roles, actions, and states, and their relations.
A workflow is associated with an object, which would typically be one of the following:
There's also a short_name, so you can easily distinguish between multiple workflows for the same package, e.g., one for handling the bug, and another for approving creation of new versions or components in the bug-tracker.
A workflow is also associated with an object type. The reason for this is that assignments will frequently depend on attributes of the specific object for the case. In bug-tracker, for example, the default assignee for a bug will be the maintainer of the component in which the bug has been found. The bug-tracker will provide one or more assignment service contract implementations, which, given the bug_id will give you the component maintainer, or the project maintainer. These can be used to set up automatic assignment through a nice web-based user interface.
When you create a new workflow case for a specific object, we will check that this object descends from the object type for which the workflow is for. If your workflow is general enough to work for all object types, then you can simply associate it with the common ancestor of all objects, 'acs_object'.
When you create a new instance of the bug-tracker, we would make a copy of the default bug-tracker workflow for your particular package, so that you can make local changes to the workflow, to the assignments, etc.
A workflow can have side-effects, which fire when any action is triggered on that workflow. These fire after the specific actions. See more under action side-effects. These are declared as a standard "Action_SideEffect" service contract implementation.
Another service contract on the workflow level is the activity log entry title formatting contract. Using a side-effect callback, you can store additional key/value pairs in the activity log. You can use the title formatting service contract to pull these out, along with any other data you like, and use them to format the title of the log entry for display.
A workflow has a set of roles. For bug-tracker, this is Submitter, and Assignee. More complex bug-tracker workflows, could add Triager and Tester. For a typical pulication workflow, you'd have Author, Editor, and Publisher. Normally, you'd always include an 'Administrator' role.
Each role is associated with one or more actions in the workflow. The assignee is assigned to the 'Resolve' action, but also has permission to perform the Edit, Comment and Reassign actions. The submitter is assigned to the 'Close' action, but also has permission to 'Reopen', 'Edit', 'Comment', and possibly 'Reassign'.
The idea behind introducing roles is that you do not want to go through the bother of assigning each action individually, when normally they are grouped together.
Then, as the workflow case unfolds, people are given roles--you will be the submitter, you will be the assignee. Roles can get reassigned at any time.
The tricky part, however, is the rules saying who should be assigned by default, or who can be assigned to this role. First, let's look at how the default assignees can be determined.
These different options are supplied by programmers as implementations of a particular service contract (see below under service contracts).
In the definition of a workflow, you can select an ordered list of default assignment methods Each will be tried in the order you specify. The first to return a non-empty list of assignees is the one which will be used, and the rest won't get called. So for example you can say "first try component maintainer, and if non is specified, use the project maintainer".
The workflow package will supply a few standard implementations:
Default assignment is done in a lazy fashion, in that we don't try to find the default assignees until we need to. We need to the first time an action assigned to that role is enabled. This allows your default assignment to depend on things that happened in prior tasks.
Now, let's look at what happens when you want to reassign a role to someone else. the
A couple of default implementations will be supplied by the workflow package. For the pick-list:
For the search query:
In order to determine who are supposed to perform an action, and who are allowed to perform the action, we let you specify these three things for each action:
Actions can also have side-effects, which simply means that whenever an action is triggered, one or more specified service contract implementations will get executed. These side-effects are executed after all other updates, both to the case object, and to the workflow tables, have been completed.
This is specific to the FSM-model. A workflow has a finite set of states, for example "open", "resolved", and "closed". A case will always be in exactly one such state. When you perform an action, the workflow can be pushed into a new state.
There will be one initial state, which the workflow will start out in. This will be the first state according to the sort order from workflow_fsm_states
States have almost no information associated with them, they're simply used to govern which actions are available.
A case is the term for a workflow in action. A case always revolves around a specific object. and we currently only allow one case for one object. That is, you can only have one workflow in process for one object.
The case holds information about the current state, the current assignments, and an activity log over everything that happens on the case.
//--------------------// // Workflow level // //--------------------// create table workflows ( workflow_id integer ... primary key, references acs_objects short_name varchar ... pretty_name varchar ... object_id integer ... references acs_objects -- object_id points to either a package type, -- package instance, or single workflow case object_type varchar ... references acs_object_types -- which object type (or its subtypes) is this workflow designed for unique (object_id, short_name) ); create table workflow_callbacks ( workflow_id integer ... references workflows acs_sc_impl_id integer ... references acs_sc_impls sort_order integer ... constraint ... primary key (workflow_id, acs_sc_impl_id) ); create table workflow_roles ( role_id integer ... primary key workflow_id integer ... references workflows short_name varchar ... pretty_name varchar ... ); create table workflow_role_default_parties ( role_id integer ... references workflow_roles party_id integer ... references parties constraint ... primary key (role_id, party_id) ); create table workflow_role_allowed_parties ( role_id integer ... references workflow_roles party_id integer ... references parties constraint ... primary key (role_id, party_id) ); create table workflow_role_callbacks ( role_id integer ... references workflow_roles acs_sc_impl_id integer ... references acs_sc_impls -- this can be an implementation of any of the three assignment -- service contracts: DefaultAssignee, AssigneePickList, or -- AssigneeSubQuery sort_order integer ... constraint ... primary key (role_id, acs_sc_impl_id) ); create table workflow_actions ( action_id integer ... primary key workflow_id integer ... references workflows sort_order integer ... short_name varchar ... pretty_name varchar ... pretty_past_tense varchar ... assigned_role integer ... references workflow_roles ); create table workflow_action_allowed_roles ( action_id integer ... references workflow_actions role_id integer ... references workflow_roles ); create table workflow_action_privileges ( action_id integer ... references workflow_actions privilege varchar ... references acs_privileges ); create table workflow_action_callbacks ( action_id integer ... references workflow_actions acs_sc_impl_id integer ... references acs_sc_impls sort_order integer ... constraint ... primary key (action_id, acs_sc_impl_id) ); // Finite State Machine model // create table workflow_fsm_states ( state_id integer ... primary key workflow_id integer ... references workflows sort_order integer ... short_name varchar ... pretty_name varchar ... ); create table workflow_fsm_actions ( action_id integer ... primary key ... references workflow_actions new_state integer ... references workflow_fsm_states (can be null) ); create table workflow_fsm_action_enabled_in_states ( action_id integer ... references workflow_fsm_actions state_id integer ... references workflow_fsm_states ); create table workflow_fsm ( workflow_id integer ... primary key, references workflows initial_state integer ... references workflow_fsm_states ); //--------------------// // Case level // //--------------------// create table workflow_cases ( case_id integer ... primary key workflow_id integer ... references workflows object_id integer ... references acs_objects ... unique -- the object which this case is about, e.g. object_id of the bug ); create table workflow_case_log ( entry_id integer ... primary key case_id integer ... references workflow_cases action_id integer ... references workflow_actions user_id integer ... references users action_date timestamp not null default now(), comment text ... comment_format varchar ... ); create table workflow_case_log_data ( entry_id integer ... references workflow_case_log key varchar value varchar constraint ... primary key (entry_id, key) ); create table workflow_case_role_assigned_parties ( case_id integer ... references workflow_cases role_id integer ... references workflow_roles party_id integer ... references parties constraint ... primary key (case_id, role_id, party_id) ); // Finite State Machine model // create table workflow_case_fsm ( case_id integer ... references workflow_cases current_state integer ... references workflow_fsm_states );
workflow.Role_DefaultAssignees: GetObjectType -> string GetPrettyName -> string GetAssignees (case_id, object_id, role_id) -> { list of party_id }
workflow.Role_AssigneePickList GetObjectType -> string GetPrettyName -> string GetPickList (case_id, object_id, role_id) -> { list of party_id }
workflow.Role_AssigneeSubQuery GetObjectType -> string GetPrettyName -> string GetSubQueryName (case_id, object_id, role_id) -> { subquery_name { bind variable list } }
workflow.Action_SideEffect GetObjectType -> string GetPrettyName -> string DoSideEffect (case_id, object_id, action_id, entry_id) -> (none)
workflow.ActivityLog_FormatTitle GetObjectType -> string GetPrettyName -> string GetTitle (entry_id) -> title
The GetObjectType method is used for the service contract implementation to tell which object types it is valid for. For example, a DefaultAssignee implementation can look at a bug, find out which component it is found in, then look up the component definition to find the default maintainer. This implementation, though, is only valid for objects of type 'bt_bugs', or any descendants thereof. Thus, this is what the GetObjectType call would return for this implementation. If your implementation is valid for any ACS Object, then simply return 'acs_object', as this is the mother of all objects.
The GetPrettyName method will be run through a
localization filter, meaning that any occurrence of the
#message-key#
notation will be replaced with
a message catalog lookup for the current domain.
The AssigneeQuery service contract probably
needs a little explanation. You're supposed to supply a valid
subquery, which will select the columns party_id, name, email, and
screen_name (nulls are okay) of all the parties that a role can
possibly be assigned to. A simple version could simply be
"cc_users
". Another would be:
select u.user_id as party_id, u.first_names || ' ' || u.last_names as name, u.email, u.screen_name from cc_users u where (some condition)
This would then typically be used like this:
select distinct q.party_id, q.name || ' (' || u.email || ')' as name_and_email from (your subquery goes here) q where upper(coalesce(q.name, '') || q.email || ' ' || coalesce(q.screen_name, '')) like upper('%'||:value||'%') order by name_and_email
Now, one little caveat is that you have to return the query dispatcher query name, not the actual query. The query name will then get passed to db_map to produce the actual subquery.
Workflow will supply these service contract implementations by default:
You can sign up for notifications at several levels:
You should always receive at most one notification per activity. They're sent out in the order in which they're listed here, and if you get the first, you won't get the second, third or fourth; if you get the second, you won't get the third or fourth, etc.
A special case is that the first notification isn't optional. You don't have to manually go sign up for those notifications, and you can't turn them off entirely. You can still change the delivery method and the frequency, though.
In order to implement this, we need to make three fairly trivial enhancements to the notifications package.
notification::new
to the next.
So notification::new
needs to take a parameter like
-already_notified
and to not notify those again, and
likewise, to return the list of users notified by the given
notification.notification::new
would
take as a parameter the list of people who are assigned to some
role on this particular case. Only people who are subscribed
and on that list will get notified. I can't think of a
good name for such a parameter, perhaps
-positive_list
.-force:boolean
parameter
which works in conjunction with the positive list, so that people
on the positive list who aren't subscribers get a default
email/instant subscription automatically. They can then go back and
change their delivery method and frequency later.You can define it using a Tcl interface:
set workflow_id [workflow::new \ -short_name "bug" -pretty_name "Bug" \ -object_id [package::object_id "bug-tracker"] \ -object_type "bt_bug" \ -callbacks { bug-tracker.FormatLogTitle } ##### # # Roles # ##### workflow::role::add $workflow_id \ -short_name "submitter" \ -pretty_name "Submitter" \ -callbacks { workflow.CreationUser } workflow::role::add $workflow_id \ -short_name "assignee" \ -pretty_name "Assignee" \ -callbacks { bug-tracker.ComponentMaintainer bug-tracker.ProjectMaintainer } ##### # # States # ##### workflow::fsm::state::add $workflow_id \ -short_name "open" \ -pretty_name "Open" \ workflow::fsm::state::add $workflow_id \ -short_name "resolved" \ -pretty_name "Resolved" workflow::fsm::state::add $workflow_id \ -short_name "closed" \ -pretty_name "Closed" ##### # # Actions # ##### workflow::fsm::action::add $workflow_id \ -short_name "comment" \ -pretty_name "Comment" \ -pretty_past_tense "Commented" \ -allowed_roles { submitter assignee } \ -privileges { feedback } workflow::fsm::action::add $workflow_id \ -short_name "edit" \ -pretty_name "Edit" \ -pretty_past_tense "Edited" \ -allowed_roles { submitter assignee } \ -privileges { write } workflow::fsm::action::add $workflow_id \ -short_name "resolve" \ -pretty_name "Resolve" \ -pretty_past_tense "Resolved" \ -assigned_roles { assignee } \ -enabled_states { open resolved } \ -new_state "resolved" \ -privileges { write } \ -callbacks { bug-tracker.CaptureResolutionCode } workflow::fsm::action::add $workflow_id \ -short_name "close" \ -pretty_name "Close" \ -pretty_past_tense "Closed" \ -assigned_roles { submitter } \ -enabled_states { resolved } \ -new_state "closed" \ -privileges { write } workflow::fsm::action::add $workflow_id \ -short_name "reopen" \ -pretty_name "Reopen" \ -pretty_past_tense "Closed" \ -allowed_roles { submitter } \ -enabled_states { resolved closed } \ -new_state "open" \ -privileges { write }
Alternatively, we could have an ad_form/ad_page_contract style spec as well:
set workflow { roles { submitter { pretty_name "Submitter" callbacks { workflow.CreationUser } } assignee { pretty_name "Assignee" callbacks { bug-tracker.ComponentMaintainer bug-tracker.ProjectMaintainer } } } states { open { pretty_name "Open" } resolved { pretty_name "Resolved" } closed { pretty_name "Closed" } } actions { comment { pretty_name "Comment" pretty_past_tense "Commented" allowed_roles { submitter assignee } privileges { feedback } } edit { pretty_name "Edit" pretty_past_tense "Edited" allowed_roles { submitter assignee } privileges { write } } resolve { pretty_name "Resolve" pretty_past_tense "Resolved" assigned_roles { assignee } enabled_states { open resolved } new_state "resolved" privileges { write } callbacks { bug-tracker.CaptureResolutionCode } } close { pretty_name "Close" pretty_past_tense "Closed" assigned_roles { submitter } enabled_states { resolved } new_state "closed" privileges { write } } reopen { pretty_name "Reopen" pretty_past_tense "Closed" allowed_roles { submitter } enabled_states { resolved closed } new_state "open" privileges { write } } } } set workflow_id [workflow::new \ -short_name "bug" \ -pretty_name "Bug" \ -object_id [package::object_id "bug-tracker"] \ -object_type "bt_bug" \ -callbacks { bug-tracker.FormatLogTitle } \ -workflow $workflow]
set bug_id [bug_tracker::bug::new ...] workflow::case::new \ -workflow_id [workflow::get_id -object_id [ad_conn package_id] -short_name "bug"] \ -object_id $bug_id
The intended user interface for a workflow-based application is similar to the bug-tracker. The form is shown in display-only mode, with buttons corresponding to actions along the bottom (e.g. Comment, Edit, Resolve, Close).
Find the case_id from object_id and workflow short_name.
Find out which roles the current user has wrt the current object.
The actions currently enabled in this state.
The enabled actions which the current user has permission to perform.
Which fields should we edit, depending on the current action. NOTE! We probably won't be able to support this in the first version.
Which fields should we hide, depending on the state. NOTE! We probably won't be able to support this in the first version.
Is this action enabled and allowed for this user?
The new state which the case will have after this action has been performed (if action doesn't change state, returns the current state again.
Perform the action, updating the workflow state, etc. This should be called from inside a db_transaction where the case object has just been updated.
Here's what the form page would look like:
ad_page_contract { ... } { bug_id:integer,notnull } # Setup return_url, user_id, etc. ... # Current action, blank for display mode set action [form get_action bug] # Check permissions workflow::case::require_permission -object_id $bug_id -action $action # Create the form form create bug \ -mode display \ -actions [workflow::case::get_actions -object_id $object_id -action $action] \ -cancel_url $return_url element create ... # Valid submission: Update if { [form is_valid bug] } { bug_tracker::bug::edit \ -bug_id $bug_id \ ... ad_returnredirect $return_url ad_script_abort } # Non-valid submission: Either request or error form if { ![form is_valid bug] } { bug_tracker::bug::get -bug_id $bug_id -array bug set bug(status) [workflow::action::new_state -object_id $object_id -action $action] # Hide elements that should be hidden foreach element [workflow::state::get_hidden_fields -object_id $object_id] { element set_properties bug $element -widget hidden } # Set element values ... # - if [form is_request] then set all # - otherwise only set elements in display-mode # Page title, context bar, filters, etc. ... }
Nice-to-haves that aren't entirely pie-in-the-sky include:
I've looked into pluggable models before, and it's not too complicated. The trick is that you have four areas where the generic workflow framework/engine will interface with the plugin model:
These are the interaction points between a generic workflow engine, and its specific model implementations.
Should we discard workflow and rewrite, or should we try to incrementally improve what's there?
In general, you should be weary of rewriting if:
Neither of these are the case here. We don't have any significant users of workflow, and we have access to the same people (person) who did the original implementation to implement it again.
Besides, the planned changes are so big that there would be no code left untouched.
Hence, we've concluded that a rewrite is in fact the most productive strategy.