Simuation Design - Workflow Extensions

Timers

Requirements

Use cases:

• A student has one week to send a document to another role. If he/she fails to do so, a default action executes.
• An OpenACS OCT member has one week to vote on a TIP. If he/she does not vote within that week, a default "Abstain" action is executed.

The timer will always be of the form "This action will automatically execute x amount of time after it becomes enabled". If it is later un-enabled (disabled) because another action (e.g. a vote action in the second use casae above) was executed, then the timer will be reset. If the action later becomes enabled, the timer will start anew.

Design

We currently do not have any information on which actions are enabled, and when they're enabled. We will probably need a table, perhaps one just for timed actions, in which a row is created when a timed action is enabled, and the row is deleted again when the state changes.

Extending workflow_actions

create table workflow_actions(
    ...
    -- The number of seconds after having become enabled the action
    -- will automatically execute
    timeout                 interval
    ...
);

DESIGN NOTE: The 'interval' datatype is not supported in Oracle.

The Enabled Actions Table

create table workflow_case_enabled_actions(
    case_id                 integer
                            constraint wf_case_enbl_act_case_id_nn
                            not null
                            constraint wf_case_enbl_act_case_id_fk
                            references workflow_cases(case_id)
                            on delete cascade,
    action_id               integer
                            constraint wf_case_enbl_act_action_id_nn
                            not null
                            constraint wf_case_enbl_act_action_id_fk
                            references workflow_actions(action_id)
                            on delete cascade,
    -- the timestamp when this action will fires
    execution_time          timestamptz
                            constraint wf_case_enbl_act_timeout_nn
                            not null,
    constraint workflow_case_enabled_actions_pk
    primary key (case_id, action_id)
);

The Logic

After executing an action, workflow::case::action::execute will:

1. Delete all actions from worklfow_case_enabled_actions which are no longer enabled.
2. If the timeout is zero, execute immediately.
3. Insert a row for all enabled actions with timeouts which are not already in workflow_case_enabled_actions, with fire_timestamp = current_timestamp + workflow_actions.timeout_seconds .

NOTE: We need to keep running, so if another automatic action becomes enabled after this action fires, they'll fire as well.

The Sweeper

The sweeper will find rows in workflow_case_enabled_actions with fire_timetsamp < current_timestamp, ordered by fire_timstamp, and execute them.

It should do a query to find the action to fire first, then release the db-handle and execute it. Then do a fresh query to find the next, etc. That way we will handle the situation correctly where the first action firing causes the second action to no longer be enabled.

The Optimization

Every time the sweeper runs, at least one DB query will be made, even if there are no timed actions to be executed.

Possible optimizations:

• We keep an NSV with the timestamp (in [clock seconds] format) and (case_id, action_id) of the first action to fire. That way, the sweeper need not hit the DB at all most of the time. When a new timed action is inserted, we compare with the NSV, and update if the new action fires before the old action. When the timed action referred to in the NSV is either deleted because it gets un-enabled, or executed, we'll clear the NSV, causing the next hit to the sweeper to execute the query to find the (case_id, action_id, fire_timestamp) of the first action to fire. Finally, we would need an NSV value to represent the fact that there are no rows in this table, so we don't keep executing the query in that case.

Hierarchical Workflows

Requirements

Use cases:

• Leiden: We have several occurances of the simple AskInfo-GiveInfo question/response pair. Defining simulation templates would be simplified if that was a reusable component.
• TIP Voting: There's a master workflow case for the TIP itself. When voting, there'll be a sub-workflow case for each TIP member to vote on the issue, with timeouts so if they don't vote within a week, their vote is automatically 'Abstained'.

Design

• Actions will no longer be atomic. An action can be "in progress" for a long time, while the child workflow(s) completes.
• We will introduce an uber-state of a case, which can be 'active', 'completed', 'canceled', or 'suspended'.
• When the action gets enabled, a callback will create child cases linked to this particular enabled action.
• Whenever a child case changes its case_state, a callback on the parent action is invoked, which examines the state of all of its child cases and determines whether the parent action is complete and ready to fire or not. If the parent action is completed, all any 'active' child cases will be made 'canceled'.
• If the action should ever get un-enabled, a callback will cancel all remaining 'active' child cases.
• If the action becomes enabled again, we will create new child cases.
• A case which is a child of another case cannot leave the 'completed' or 'canceled' state, unless its parent enabled action is still enabled.

Data Model

create table workflow_actions(
  ...
  child_workflow            integer
                            constraint wf_action_child_wf_fk
                            references workflows(workflow_id),
  ...
);

create table workflow_fsm_states(
  ...
  -- does this state imply that the case is completed?
  complete_p              boolean,
  ...
);

create table workflow_action_fsm_output_map(
  action_id               integer
                          not null
                          references workflow_actions(action_id)
                          on delete cascade,
  acs_sc_impl_id          integer
                          not null
                          references acs_sc_impls(impl_id)
                          on delete cascade,
  output_value            varchar(4000),
  new_state               integer
                          references workflow_fsm_states
);

create table workflow_action_child_role_map(
  parent_action_id          integer
                            constraint wf_act_chid_rl_map_prnt_act_fk
                            references workflow_actions(action_id),
  parent_role               integer
                            constraint wf_act_chid_rl_map_prnt_rl_fk
                            references workflow_roles(role_id),
  child_role                integer
                            constraint wf_act_chid_rl_map_chld_rl_fk
                            references workflow_roles(role_id),
  mapping_type              char(40)
                            constraint wf_act_chid_rl_map_type_ck
                            check (mapping_type in 
                                ('per_role','per_member','per_user'))
);


create table workflow_cases(
  ...
  state                      char(40)
                             constraint workflow_cases_state_ck
                             check (state in ('active', 'completed',
                             'closed', 'canceled', 'suspended'))
                             default 'active',
  suspended_until            timestamptz,
  parent_enabled_action_id   integer
                             constraint workflow_cases_parent_fk
                             references workflow_case_enabled_actions(enabled_action_id)
);

create table workflow_case_enabled_actions(
    enabled_action_id       integer
                            constraint wf_case_enbl_act_case_id_pk
                            primary key,
    case_id                 integer
                            constraint wf_case_enbl_act_case_id_nn
                            not null
                            constraint wf_case_enbl_act_case_id_fk
                            references workflow_cases(case_id)
                            on delete cascade,
    action_id               integer
                            constraint wf_case_enbl_act_action_id_nn
                            not null
                            constraint wf_case_enbl_act_action_id_fk
                            references workflow_actions(action_id)
                            on delete cascade,
    enabled_state           char(40)
                            constraint wf_case_enbl_act_state_ck
                            check (enabled_state in ('enabled','completed','canceled','refused')),
    -- the timestamp when this action automatically fires
    fire_timestamp          timestamp
                            constraint wf_case_enbl_act_timeout_nn
                            not null,
    constraint wf_case_ena_act_case_act_un
    primary key (case_id, action_id)
);

Callback Types

• Action.CanEnableP -> (CanEnabledP): Gets called when an action is about to be enabled, and can be used to prevent the action from actually being enabled. This will only get called once per case state change, so if the callback refuses to let the action become enabled, it will not be asked again until the next time the case's state changes. If the callback refuses, the enabled_state of the row in workflow_case_enabled_actions will be set to 'refused'.
• Action.OnEnable -> (output): Gets called when an action is enabled. Output can be used to determine the new state of the case (see below), in particular for an in-progress state.
• Action.OnUnEnable: Gets called when an action that used to be enabled is no longer enabled. Is not called when the action fired and thus caused it to no longer be enabled.
• Action.OnChildCaseStateChange -> (output, CompleteP): Called when a child changes its case state (active/completed/canceled/suspended). Returns whether the parent action has now completed. Output can be used to determine the new state of the case (see below).
• Action.OnFire -> (output): Executed when the action fires. Output can be used to determine the new state of the case (see below).
• Action.SideEffect: Unchanged from current implementation. The difference between this and OnFire is that we can have multiple side-effects, but they cannot determine the new state of the case.

Callback Output

The callbacks returning 'output' above must enumerate all the values they can possible output (similar construct to GetObjectType operation on other current workflow service contracts), and the callback itself must return one of those possible values.

The workflow engine will then allow the workflow designer to map these possible output values of the callback to new states, in the case of an FSM, or similar relevant state changes for other models.

Enabled Action Logic

Executed when an action which was previously not enabled becomes enabled.

1. If the action has a timeout of 0, then execute the action and quit.
2. Insert a row into workflow_case_enabled_actions.
3. If the action has non-null timeout > 0, then the row will have a execution_time of current_timestamp + timeout.
4. If the action has non-null child_workflow, create child cases. For each role which has a mapping_type of 'per_member' or 'per_user', create one case per member/user of that role. If more roles have per_member/per_user setting, then the cartesian product of child cases are created (DESIGN QUESTION: Would this ever be relevant?)
5. If there is any ActionEnabled callback, execute that (only the first, if multiple exists), and use the workflow_fsm_output_map to determine which new state to bump the workflow to, if any.

Un-Enabled Action Logic

Executed when an action which was previously enabled is no longer enabled, because the workflow's state was changed by some other action.

1. If the action has any child cases, these will be marked canceled.

Child Case State Changed Logic

We execute the OnChildCaseStateChange callback, if any. This gets to determine whether the parent action is now complete and should fire.

We provide a default implementation, which simply checks if the child cases are in the 'complete' state, and if so, fires.

NOTE: What do we do if any of the child cases are canceled? Consider the complete and move on with the parent workflow? Cancel the parent workflow?

NOTE: Should we provide this as internal workflow logic or as a default callback implementation? If we leave this as a service contract with a default implementation, then applications can customize. But would that ever be relevant? Maybe this callback is never needed.

On Fire Logic

When the action finally fires.

If there's any OnFire callback defined, we execute this.

If the callback has output values defined, we use the mappings in workflow_action_fsm_output_map to determine which state to move to.

After firing, we execute the SideEffect callbacks and send off notifications.

DESIGN QUESTION: How do we handle notifications for child cases? We should consider the child case part of the parent in terms of notifications, so when a child action executes, we notify those who have requested notifications on the parent. And when the last child case completes, which will also complete the parent action, we should avoid sending out duplicate notifications. How?

Case State

Cases can be active, complete, suspended, or canceled.

They start out as active. For FSMs, when they hit a state with complete_p = t, the case is moved to 'complete'.

Users can choose to cancel or suspend a case. When suspending, they can type in a date, on which the case will spring back to 'active' life.

When a parent worfklow completes an action with a sub-workflow, the child cases that are 'completed' are marked 'closed', and the child cases that are 'active' are marked 'canceled'.

The difference between 'completed' and 'closed' is that completed does not prevent the workflow from continuing (e.g. bug-tracker 'closed' state doesn't mean that it cannot be reopened), whereas a closed case cannot be reactivarted (terminology confusion alert!).

Appendix: Resolution Code

• I found another dynamic-workflow product (TrackStudio) on the web, and they have resolution be a first-class citizen. If it helps we could do something similar, which would make it possible to custom-define the resolution codes and simplify something like bug-tracker somewhat. Resolution is just a way of saying "these states are all the same wrt the workflow, but they have another significance". Bug-tracker "duplicate" resolution code could be solved by just having a generic mechanism for relating bugs, and ignore any UI connection with resolution code "duplicate". Also, resolution code "postponed" could be eliminated, and replaced with the case-state of "suspended" above. We could also add an "suspended_until" date, to keep track of when the case should resurface (if at a fixed date).

Appendix: TIP Voting Process

TIP Master Workflow
  Model = FSM
  Roles
    Submitter
    Voter
  States
    Proposed
    Voting
    Withdrawn
    Approved
    Rejected
  Actions
    Propose
      Initial action
      New state = Proposed
    Vote
      Enabled in state = Proposed
      Role = Voter
      Sub-workflow = Individual Vote
        In progress state = Voting
        Sub-role Voter = pparent role Voter
          One sub-case per user in the Voter role
      New state = Approved | Rejected
      Logic = 
        0 Rejects + > 0 Approvals = Approved
        2/3rds Approvals => Approved
        Otherwise => Rejected
    Withdraw
      Enabled in state = Proposed, Voting
      Role = Submitter
      New state = Withdrawn

TIP Individual Vote Workflow
  Model = FSM
  Roles
    Voter
  States
    Open
    Approved
    Rejected
    Abstained
  Actions
    Open
      Initial action
      New state = Open
    Approve
      Enabled in state = Open
      Role = Voter
      New state = Approved
    Reject
      Enabled in state = Open
      Role = Voter
      New state = Rejected
    Abstain
      Enabled in state = Open
      Role = Voter
      New state = Abstained
    No Vote
      Enabled in state = Open
      Timeout = 7 days
      New state = Abstained

Appendix: Leiden's Example Workflow

Leiden Master Workflow
  Model = Dependency
  Roles
    Client 1
    Lawyer 1
    Partner 1
    Secretary 1
    Client 2
    Lawyer 2
    Partner 2
    Secretary 2
    Library
  Actions
    L1 Info Request From C2
    L1 Info Request From L2
      Dependent on = L1 Info Request From C2
    L1 Info Request From Library
    
    L1 Draft With S1
      Dependent on = L1 Info Request From L2, L1 Info Request From Library
    L1 Send Document
      Dependent on = L1 Draft With S1

    P1 Intervenes With L1

    L2 Info Request From C1
    L2 Info Request From L1
      Dependent on = L2 Info Request From C1
    L2 Info Request From Library
    
    L2 Draft With S2
      Dependent on = L2 Info Request From L1, L2 Info Request From Library
    L2 Send Document
      Dependent on = L2 Draft With S2

    P2 Intervenes With L2

    Done
      Dependent on = L1 Send Document, L2 Send Document

AskInfo-GiveInfo Loop Workflow
  Roles
    Informer
    Recipient
  States
    Asked
    Given
  Actions
    Ask Info
      Initial action
      New State = Asked
    Give Info
      Enabled in state = Asked
      Role = Informer
      New state = Given