1 Architectural Considerations in Systems Supporting Dynamic Workflow Modification

Architectural Considerations in Systems SupportingDynamic Workflow ModificationShazia W. Sadiq*Maria E. OrlowskaComputer Science and Electrical EngineeringThe University of QueenslandQLD 4072 Australiaemail:{shazia.maria}@.auAbstractOne of the major limitations of current workflow technology is the lack offlexibility necessary to support the dynamic nature of most business processestoday. We present dynamic modification of workflows as a means of providingthe technological support necessary for adapting workflows to changes inprocess requirements. In this paper we propose software architecture for aworkflow system that supports dynamic modification. We identify requiredadditional components, increased functionality of existing components, andinterfaces between the components and agents external to the workflow system.The proposed architecture is based upon a three-phase modificationmethodology, which we also briefly describe.1Scope of Workflow ChangesDue to the continually accelerating pace of technological advancements, changing requirements and regulations, and introduction of new methods, business process models are being constantly reviewed, improved and adapted to the changing environment. Change can arise due to three main reasons. First is process i mprovement, which involves performing the same business process with increased efficiency [Dav93]. Secondly, process i nnovation, which involves performing the business process in a radically different way. Business process reengineering would generally fall in this class of change, since it is "a new conceptualization of the business process" [Ham90]. And finally, process a daptation, which involves adapting the process to unforeseen change.Before taking the discussion further, we briefly introduce basic concepts to clarify terminology. The workflow model or process model is a definition of the tasks, ordering, data, resources, and other aspects of the process. Most, if not all, workflow models are represented as graphs [CCP+95], [RD97], [SO97], [AHH94] where nodes in the graph represent process activities or tasks, and edges depict the flow or ordering of the tasks involved in the process. For example, we can define an admission workflow that handles admission applications in a university. Ideally the workflow model is intended to completely achieve process goals with maximum efficiency. Workflow instance is a particular occurrence of the process, for example, a particular application for admission represents an instance of the admission workflow. Different instances of the same workflow may perform a different subset of workflow tasks, i.e. they may follow different paths * The work reported in this paper has been funded in part by the Cooperative Research Centres Program through the Department of the Prime Minister and Cabinet of the Commonwealth Government of Australia.1in the workflow graph. An instance class is a set of instances that can be represented by the same sub-graph. An instance that represents a departure from the process model is an exception.A first step towards understanding workflow modifications is to determine the various effects that business process change can have on the workflow. When business processes change, because of some event internal or external to the organization, the modifications are generally planned, revised, approved and specified by high level managers or consultants, and then gradually propagated to operational level. We see the role of a workflow administrator (WFA), not different from a database administrator, who acts as a mediator between management’s proposals and strategies, and the propagation of these proposals to the operational level. WFA thus has to be capable of translating the (changed) process specifications into workflow models, and making decisions regarding handling of active workflow instances. We view the effects of business process change on workflows, as Modification Policies, which can be adopted by the WFA, and identify five modification policies for workflows.FlushIn flush situations all current instances are allowed to complete according to the old process model, but new instances are planned to follow new model. New instances may be put on hold, until all current instances have completed. However, the two specifications could also be allowed exist simultaneously, and be treated as two different models. For example, immigration policies could be changed by new government regulations, effecting all applicants who apply after a certain date. However, ongoing applicants remain unaffected, i.e., their applications would be processed according to old rules, causing the two schemas to co-exist in the transition period. AbortActive workflow instances may be aborted when the process model is changed. Abort is most commonly used for adaptation of individual instances, for example canceling a reservation. However, it may also be a result of a radical change in the organization, for example, the management of a purchasing office may be changed because of bad planning and procedures practiced previously. To overcome the crisis, the new management may cancel all current purchasing orders, reallocate the budget, and introduce a new purchasing procedure. Cancellation of the purchasing orders would cause current instances of the workflow to abort, and then restart according to new procedures. This approach may incur losses to the organization, for example, the organization may be penalized for order cancellation in the form of fines, reputation etc. In some cases the losses may be unacceptable, for example in a manufacturing environment, an abort means that components assembled so far are either wasted, or have to be disassembled. In most cases, abort will require undoing, or compensating for the work accomplished so far.MigrateThe change effects all current instances but it has to be introduced without allowing current instances to abort or flush. Current instances would normally be in different stages of process execution. The main problem arises when an instance is at a stage where tasks already accomplished have affected the process in such a way that subsequent tasks are unable to proceed in accordance with the new specification. Thus migration may involve undo or compensation of completed tasks, in order to bring the instance in compliance with the new specification. The worst case is when the complete process has to be rolled back to the start, that is all work is lost or undone. This special case is equivalent to Abort. Taking the example of immigration again, the new government may require all applicants, current and new, to sit for an English language test before the application is finally approved. Thus an additional task is introduced which has to be executed at an appropriate time for all current instances. When it is actually executed, will be dependent on the instance.AdaptAdapt includes cases of errors and exceptions, where the process model may not change permanently, but some instances have to be treated differently because of some exceptional and2unforeseen circumstances. For example, in a university admission process there could be an applicant with a background in information systems and computer science, who is applying for a doctorate in management. Reviewers in the department of management sciences may refer the application for review by faculty of computer science, to determine the potential of the applicant. Such ad-hoc changes in otherwise repetitive and predictable workflows are bound to occur once in a while.BuildBuilding of a new process is also a class of process change. The difference is that the starting point is not a detailed pre-existing model, but an elementary description, which captures only the basics, or even an empty process. A typical example can be where process activities are identified, but the order of execution is mostly unknown. The advantage of including build as a class of process change, is that it allows the inclusion of processes which cannot be fully predefined, into the domain of process change. Thus essentially the same mechanism can cater for dynamic definition (build) as well as dynamic modification (migrate, adapt etc.)The differences in the policies are highlighted in the following table:ModificationPolicyModelChanged?Affected Instances(Out of Current)Compliant Instances(Out of Affected)Flush Y None-Abort Y/N Some NoneMigrate Y All SomeAdapt N Some SomeBuild Y All AllThe common denominator in all of the above policies, with the possible exception of flush, is that they effect active instances of the given process model. Thus they dictate the scope of workflow modification and constitute dynamic modification, in contrast to static modification, which is merely a change in the workflow model, i.e. no currently active instances are involved.2Modification MethodologyWe propose a simple yet effective framework to handle the modification policies identified in previous section. The framework is based on a three-phase modification methodology that consists of defining, conforming to and effectuating the modification. We will briefly describe the three phases in the following sections. The formal specifications of the underlying graphical workflow model and detailed procedures for the methodology can be found in [SO98].Defining the ModificationThe modification process begins by defining the modification, which constitutes specifying the modification policy, specifying the affected instances in case of abort or adapt, and specifying the changes to be made to the workflow model.We define M to be the Modification on the Workflow W, such that M: W k→W k+1 where W k is Workflow Model Version k. M consists of a sequence of prescribed operations which when performed upon W k, will give W k+1.In designing the operations, we observed an interesting trade-off between flexibility of specification and support for verification. Operations that are designed to ensure correctness compromise flexibility of specification. However, flexible operations do not necessarily compromise correctness. Recent research in this area has mostly focused on designing operations that guarantee the (structural) correctness of the new model [CCP+96], [RD97]. Our contention is that modifying a WF should be as flexible as building a new WF. Where, subgraphs of the3workflow graph can be changed, without having to perform each operation one by one, or in any specific order. Since no instances are involved during the modification of the model, condition for correctness after each operation unnecessarily restricts the modifier. This degree of flexibility is possible if modification operations are encapsulated into transactions, and only begin and end of the transaction guarantees WF correctness. The ideal situation is where control over the granularity of the modification transaction is given to the ‘modifier’, and system verification of the model is imposed only after completely specifying the modifications to the workflow model. Conforming to the ModificationAfter defining the modification, the next step is to bring the affected instances of W k in conformity with the specifications of W k+1. Instances must be grouped with respect to progress or stage, if automation is to be provided, else the process will reduce to individual handling of every affected instance. We have proposed a three level grouping scheme. At the first level, instances are grouped on the basis of Compliance. At the second level, the stage of the non-compliant instances is determined. It would be unrealistic to assume that there would never be a situation during the modification of active instances where external intervention will not be required. It is conceivable that instances may be at such a stage that trying to adjust the instances will be too difficult, or equivalent to abort. These instances may have to be handled externally. At second level, instances are grouped on this basis, i.e. whether compliance can be achieved within system, or externally. At the third level, the remaining non-compliant instances are grouped with respect to their class (same subgraph).We have further proposed the concept of C ompliance Graphs for affected instances. The Compliance Graph CG i for instance i initialized under W k, defines a bridge between W k and W k+1. An instance i, follows a unique path which consists partially of W k, the compliance graph, and partially of W k+1. The construction algorithm and properties of CG i are detailed in [SO98]. Briefly, the algorithm identifies actions or compensations necessary to achieve compliance for instance i, and a suitable ‘plug’ point in W k+1. Compliance graphs thus provide revised schedules, which chart out the plan till completion for affected instances. Thus except for the (hopefully small) group of instances identified in level 2 grouping, the workflow management system can continue to provide full automated support for the changed business process.This phase is no doubt the most critical and challenging. The strength of the methodology we are presenting, lies in its ability to uniformly support all classes of workflow modification. It is not limited to handling workflow schema change, or adaptation of individual instances. Provision for the different classes of workflow modification is essentially made in this phase, where the construction of the compliance graph is driven by the specified modification policy. Effectuating the ModificationThe last phase in the modification procedure is that of effectuating the modification. This relates to the handling of workflow execution during the transition period. The transition period is signified by instances that started with the old model but are still executing. Instances may follow old model (e.g. in flush), or new model (for instances initialized after modification has been defined), or revised schedules based on compliance graphs. Instance execution, which may have been put on hold, will restart. When all instances following old model or revised schedules have completed, the new model becomes the current workflow model, and the modification process is complete.3Architecture for Systems Supporting Dynamic Workflow ModificationThe Workflow Management Coalition [WfMC95] gives a generic workflow product structure, intended to provide a general implementation model of a workflow system. Currently available4workflow products follow varieties of this structure [Forte97], [IBM98]. We propose an architecture, given in Fig. 1, which conforms to the generic model. The components shaded in gray indicate additions or extensions to the generic structure, which are designed to provide dynamic modification capabilities.Process RevisionsFig. 1. ArchitectureDefinition ToolThe definition tool is used to capture the process logic, and model it through a given language. Several commercial and research workflow modeling languages exist. How the workflow model is defined, and the extent of process semantics it captures, bears significantly on the modification process. The workflow model should be capable of capturing different aspects of the business process including structure, data, resources, transactional and temporal properties. The model constructed through the definition tool generates a process definition.5Workflow Engine(s)The workflow engine is the heart of the workflow enactment service. It interprets the process definition, creates and controls instances of the process, and communicates with workflow applications and clients.Work list Handler and User InterfaceThis component is the primary mode of communication between workflow clients and the workflow engine. It may also be responsible for the invocation of associated tools and applications. In a most simplistic view, and to emphasize on the relevant aspects only, we identify communication from the work list handler to workflow clients as ‘To do lists’, and from workflow clients to the work list handler, as two responses: ‘Done’ and ‘Cannot do’. Where response of the later type indicates a semantic failure and the possible initiation of a modification process, since the activity in question cannot be successfully completed due to exceptions which may have risen in that workflow (instance or process). Thus the work list handler communicates process schedules from the workflow engine to workflow clients, and communicates events from clients to the engine.Workflow Administrator (WFA)The need for supervisory operations is obvious. We discuss the role of the workflow administrator in the context of the modification process. As explained before, we see the WFA as the central point in the implementation of modification policies, and as such has to be capable of creating revised process definitions and making decisions regarding handling of active workflow instances. Thus the WFA, or a person with similar responsibility and capability, would work closely with the additional software components that we will discuss presently.Workflow RepositoryThe workflow repository is the central store of workflow relevant data. It includes workflow control data, process definitions, and instance logs, which are used and built respectively by the workflow engine as the workflow progresses. Besides the central repository, the workflow engine will make use of related databases for organizational model and application or enterprise data. We anticipate extensions and changes in the workflow repository to support dynamic modification. The repository will also hold data regarding modification sessions, for example, modification policy, affected instances, and session date. Revised schedules based on compliance graphs, for instances affected by the modification will also be stored in a database within the workflow repository which we call Modification Roll.Administration and Monitoring ToolsAn important extension to these tools (Interface 5of the WfMC reference model) in the context of dynamic modification is the identification of affected instances, which in general, cannot be a manual process. For certain cases of adapt and abort, the modification may be confined to a very small group of instances or even a single instance, and thus the affected instances could be specified manually through instance ids. However, in general, a mechanism needs to be incorporated wherein a selection criteria based on decision parameters of instances is specified, and all instances meeting that criteria are determined and considered candidates for the modification.These tools will also provide extended interface to the WFA. Events triggering modification will be communicated to the WFA through this module. These events may come from workflow clients, generally indicating exceptions that may require workflow adaptation, or abort. External events may also trigger modification, as in process improvement or innovation resulting in flush or migrate. The WFA will also utilize this interface to communicate with workflow clients and other components in the system in relation to the modification.6Perhaps the most critical message during the process is to put the system on hold. The need for putting affected instances on hold is necessitated by the fact that workflows are ongoing processes, the configuration of the current instances is continuously changing as the instances progress through the workflow. ‘Catching’ the instance at the right time is imperative for the successful deployment of the modification. We interpret putting an instance on hold as not scheduling any further tasks, recalling scheduled tasks, and suspending, and in most cases aborting active tasks. For modification policies affecting all current instances, for example migrate, this translates to a system shut down. The timing, recipients and implications of this message are fundamental management issues in the process of dynamic workflow modification. We suggest putting affected instances on hold immediately after initiating a modification session. This avoids to some extent, that during the time when affected instances are identified, and when they are put on hold, their configuration does not change significantly.Modification AnalyzerWe introduce an additional software component called modification analyzer which will be used by the WFA. We see the scope for incorporating some intelligence into this component, as it can support the WFA in conducting an analysis, and making informed decisions, with regard to compliance levels and compensation effort required for a given modification at a given time. This information can go a long way in the successful deployment of the process revisions. Verification EngineWe advocate the use of a verification engine with the definition tool. As explained above, our opinion is that modifying a WF should be as flexible as building a new WF. At the time of process (re)definition, no instances are involved, and therefore the condition for correctness after each operation will impose unnecessary restrictions. We encapsulate the modification into transactions, and only begin and end of the transaction guarantees WF correctness. We allow subgraphs of the (graphical) workflow model to be changed within one modification transaction, and correctness of the model is verified by a verification engine at user specified intervals. The correctness properties and verification algorithms for the underlying workflow model can be found in [SO97], [SO99]. These have been implemented in a prototype FlowMake. FlowMake is a graphical modeling tool supported by a verification engine that verifies the workflow graph by identifying structural conflicts. In the context of dynamic modification, the importance of a verification tool increases substantially. It will be used, not just for model verification, but also for verifying temporary changes affecting only some instances.Version TrackerSince a given process model may undergo multiple revisions over a period of time, there is an administrative need to keep track of the modification sessions and the workflow versions generated as a result. These versions will portray the evolution of the workflow process over time, and also provide a recorded history for inquiries on instances completed by previous workflow versions. It may be relevant to point out here that at a given time, a given process has only one current workflow model.Compliance ModuleThe compliance module is the key software component for the system supporting dynamic modification. It intakes the modification policy, set of affected instances, instance logs, and process definition, and generates compliance graphs which will dictate the revised schedules for the given set of affected instances. This component thus constitutes automated support for dynamic modification. Arguably the compliance graphs may be subjected to further (manual) revisions by the WFA, but that would demand a great deal of expertise and insight on the part of the WFA to be able to make changes without violating any constraints, especially for complex processes. It is critical that the algorithm for compliance graph construction takes into consideration a number of key factors that will generate the minimum error free graph. We have7identified several factors in this regard, which include defining the compliance criteria, managing active tasks, and capturing transactional properties of workflow tasks.4Conducting a Modification SessionIn this section we will illustrate the modification process by conducting a hypothetical modification session through the proposed architecture. The purpose is to demonstrate the sequence of events in the modification process, interaction of software components, and the workability of the proposed architecture.1.The modification process begins when events, internal or external to the organization, causethe business process or process instances to change.2.The WFA, or designated person(s), receive the message•when instance exception is identified by a ‘cannot do’ response from a workflow client •by external entities when process change, affecting all or some instances, is initiated. 3.The WFA understands the change, and initiates a modification session through the extendedadministration and monitoring tools interface. Based on the circumstances and nature of process changes, the WFA will determine the modification policy, and where required, identify the affected instances either through instance ids, or through a selection criterion.Essential parameters for the session will be modification policy, identification of the affected instances, and session logistic details like number, date, etc. This information is sent to the workflow repository.4.The workflow clients are notified of upcoming modification, and affected instances are puton hold. As a result, further tasks are not scheduled, scheduled tasks are recalled, and active tasks are put on hold.5.The WFA then activates the definition tool and makes required modifications.6.On completing the modification, the Verification Engine will attempt to verify correctness ofthe revised model. The changes will not be committed until the revised model is free from correctness violations.7.On committing the modification, the Version Tracker will be activated. The new model willbe archived into the version store, along with its version number, date, authorization, affected instances (if specified) and other relevant data.8.The new model is then sent to the workflow repository replacing the current processdescription if the modification is permanent, or creating a dual process description for temporary situations.9.The compliance module is than activated. From the workflow repository, the moduleretrieves requisite data regarding modification policy, instance logs, and process definition.Affected instances are first grouped according to the grouping scheme explained previously, and compliance graphs are then generated for the instance groups.10.All affected instances and their respective revised schedules based on compliance graphs areplaced in the Modification Roll. The modification roll may build into a huge database for example, in migrate policies with thousands of instances, but would eventually disappear since it is a transient data store.11.The hold on the instances is released.12.Workflow activities begin to be scheduled according to revised schedules.13.As instances listed in the modification roll reach the ‘plug’ point, which is a task in the newworkflow model marking the end of the compliance graph, the instance record is removed from the roll. This essentially means that the instance can now continue to be scheduled in accordance with the new model, that is, it is now fully compliant with the new process.14.When the modification roll is emptied, it indicates that all non-compliant affected instanceshave reached compliance.15.For cases of temporary modification, upon completion of all affected instances, the dualprocess description is removed from the workflow repository.16.The modification process is completed.85ContributionsIn this paper, we primarily lay the foundation for the software architecture of a workflow implementation model that aims at supporting dynamic modification. We defined the scope of workflow modification through 5 modification policies of Flush, Abort, Migrate, Adapt and Build, and presented a modification methodology that effectively handles all identified modification policies under essentially the same framework. The proposed architecture targets the modification methodology. We have explained the need and functionality of the identified components and interfaces and also illustrated the modification process by a complete pass through the system. Thus bringing to light some key considerations with regards to the structural design of such a workflow system, independent of any modeling language or product. In summary we propose an extended workflow repository, to suit the process of dynamic modification; extensions to the administrative and monitoring tools, complementing the existing interface 5 of the WfMC reference model; a modification analyzer, an intelligent component which supports the WFA in handling the modification; a verification engine, working closely with the definition tool to provide flexibility during definition of the modification; a version tracker, to monitor and record workflow evolution; and a compliance module, to provide automated support for the process of dynamic modification.References[AHH94]Aalst W.M.P. van der, Hee K.M. van, Houben G.J. (1994) Modeling and Analyzing Workflow using a Petri-net based approach, Proceedings of Second Workshop onComputer-supported Cooperative Work, Petri nets related formalisms, Pages 31-50,1994. Editors G. De Michelis, C. Ellis and G. Memmi, P.[CCP+95]Casati F., Ceri S., Pernici B., Pozzi G. (1995) Conceptual Modeling of Workflows.Proceedings of 14th Object Oriented and Entity-Relationship Approach internationalConference, Gold Coast, Australia, Springer Verlag Lecture Notes in ComputerScience 341-35.[CCP+96]Casati F., Ceri S., Pernici B., Pozzi G. (1996) Workflow Evolution. In Proceedings of the 15th International Conference on Conceptual Modeling, ER’96, Cottbus,Germany. Springer Verlag, Lecture Notes in Computer Science.[Dav93]Davenport Thomas (1993) Process Innovation – Reengineering work through Information Technology. Harvard Business School Press.[Forte97]Forte Conductor (1997) Forte Conductor Process Development Guide, Release 1.0, 1997.[Ham90]Hammer Michael (1990) Reengineering Work: Don’t Automate, Obliterate. Harvard Business Review, July/August.[IBM98]IBM MQSeries Workflow (1998) Concepts and Architecture, Version 3.1, July 1998 [RD97]Reichert Manfred, Dadam Peter (1997) ADEPTflex - Supporting Dynamic Changes of Workflow without loosing control. Journal of Intelligent Information Systems(JIIS), Special Issue on Workflow and Process Management, 19979。