I have had a number of discussions with Jean-Jacques Dubray about the nature of work, and particularly whether unpredictable work exists. Jean-Jacques is a luminary in the field, so I figured he has probably a well considered opinion on the subject, and I really wanted to understand what he meant. So I invited him to discuss this.
Lets start with the basic question: “Do Unpredictable Processes Exist”, what would your answer be? Why?
To be clear, the answer is “no” processes are not “unpredictable”, they appear “unpredictable” because you are not relating the activities of the processes to the lifecycles of the underlying business objects. Business entities can have a very complex lifecycle, even made of composite states (the entity is in more than one state at a time). Activities are performed to transition business entities from one state to another. As I mention processes works like a “map”, you want to go from point A to point B, there are many “activities” (i.e. segments of roads on a map) that you can perform to go from point A to point B. At any given point during the journey, you have only a small number of segments you can take. This is the same for a process. At any given time, the business entities are in a state and there are only a certain number of activities that can be “enabled”.
Generally, you cannot easily express a “map” (i.e. all the possible activities and the sequence in which they can happen) for a given process. BPMN is useful, it can help document the “happy” path, the most commonly taken path,… but rarely it can reflect all the possibilities that can arise if you associate activities to transitions between states of business entities. So, if you never look at the business entity lifecycles, it is normal that activities sometimes happen in an unpredictable manner. Unfortunately, states are not “unpredictable”. You can define states as the process goes (for instance this is often what happens in “project management”, as the project advances, the PM defines the states he/she wants to reach and the activities to get there). He/she may never be consciously defining the states, the activities might be enough, but in the end activities advances the state(s) of the project.
So I completely disagree with the statement “processes are sometimes unpredictable”. The entire universe has well defined states. Nothing at all, be it a physical entity or an information entity, does not have well defined states. Only artists work of the path of processes and states (and even then some art has well defined states: dried, bakes, varnished… that the artist has to compose with). Even if you say/believe the statement is true, please specify what is unpredictable about a process: outcomes, goals, activities…?
The only reason things seem “unpredictable” is simply because:
a) multiple states are enabled at the same time (for many entities and for the same entity)
b) (often) several activities can transition from one state to another
For an observer unaware of the states and their transition, he/she cannot find any logic until all the paths have been explored, until then, he/she will discover new paths from time to time.
I am still somewhat confused by your response. When you say that the entire universe has well defined states, I might think you are saying that there is always a cause for every effect. But I don’t see the connection to business processes. A business process is written down in order to guide people in the performance of their work. It describes what should be done, in the hopes that it actually will be done. Clearly, if I am omniscient, then there are no unpredictable processes — but that is a tautology. But we are not omniscient; can a person always write down a meaningful work process that describes all the possible future activities necessary to achieve the goal?
You seem to be saying that the work process exists, but the people may not know the process. If the process is not knowable, that is the same, in my mind, as unpredictable. You say that you can not easily express a map for a given process. I think it is because the exact work process can not be known. That is exactly what I mean by saying it is unpredictable. A person in not able to predict it. Am I misunderstanding your position?
No, what I am saying is that the result of an activity is to transition resources (of the process) from one state to another. One a resource (or a set of resource) is in a given state (or set of states), there are only a certain number of activities that are possible. There is nothing you can do, it is inherent to the resources. The problem is that BPM gurus have always ignored the resource level to focus on the activity level. It is does not mean that there are no relationships between the two. You have to take into account that the BPM field has been wrong, you cannot assume omnisciently that you are always right. Thinking that activities can be defined and sequenced without taking into account the states of the resources is pure fantasy. I hope we can agree on that.
I think, yes, there is a big misunderstanding. Again, the day “BPM” people will finally understand the relationships between resource lifecycles and activity flows we will have made a huge step forward. Until then, I don’t see the point of discussing it further. The burden of proof is on your side because clearly the types of activities that can be performed are a function of the state of the resources, yet none of the Business Process Definition express this relationship in anyway. It is left to the designer of the process definition to figure it out.
We probably should write a few definitions, because I get the feeling that we are simply missing each other.
- What is the meaning of “unpredictable”?
- What is a “business process”?
- What is an “unpredictable business process”?
We need to have agreement on these, or the rest of this conversation is of no use.
To me, a “business process model” is a construct meant to represent the agreement between a set of people about what should happen. If you don’t see it this way, do you have a different definition?
For me a process is the set of activities that are performed on the resources of the enterprise to achieve a particular goal. Examples of goals include: make a widget, sell a widget… As a consequence of that definition, the set of activities become an agreement, not the other way.
Unpredictability means that the set of activities and their sequencing rules cannot be fully known. That is clearly untrue. As you can see we can predict today the weather with quite a remarkable accuracy. Despite the fact thst we don’t track the movement of every butterfly. This is achieved by a better understanding of the causes ( measurements) and their effects. There is absolutely nothing unpredictable about business processes.
As I suspected, we are at cross purposes due to differing definitions. Lets not argue about which definition is right, but talk instead about predictability of each case.
JJ-Process is: “the set of activities that are performed on the resources of the enterprise to achieve a particular goal.”
K-Process is: “the set of activities that are performed by people to achieve a business goal”. I have purposefully tried to match the same form as your definition.
JJ-unpredictable: Unpredictable means that the set of activities and their sequencing rules cannot be fully known.
K-unpredictable: Unpredictable means that the set of activities and their sequencing rules are not known by the person starting the process at the time that the business process is started.
Not so far apart, but I am making it clear that it is of no use to me to predict thing in the past, I need to predict future activities, and I have go ahead and launch into work whether those activities are known or not. Furthermore, it does no do any good if there is a person who can predict the activities, but they are unavailable.
Let consider then, the case of a hospital accepting a patient for care who has been in a car accident as a Type-1 process. The accident has caused a blood clot in the brain, ultimately requiring surgery. But there is no way that the staff accepting the patient to know this.
You seem to be holding the position that: the patient is a body that is in the state of having a clot in the brain, even though it is not apparent. That clot will ultimately be detected by the staff, and that will lead appropriately to surgery on the brain. Therefor, the process is predictable, because it all depends on the state of the body. Is that right?
My position is that the accepting staff can not predict the process at the time of admission because they have no way of knowing about the clot. It is not until the time of the diagnosis of the clot that the prediction can be made that there will be surgery. The process “unfolds” as new information is found. The discovery of the clot causes the surgery to be added to the process.
Does this seem like a reasonable summary of our positions?
In your patient example, it does not matter whether you know the state of the patient a priori or not. As the physician uncovers the state the patient is in, (let’s assume he has perfect knowledge of the activities he can perform for a given state), he makes decisions about what activity to perform. Business Process Engine have the responsibility to identify and keep track of the state of resources and offer to the participants a set of corresponding activities. In that case, you don’t even need to define “the process”. All you care are the states, the transitions and the set of activities that allow you to transition from one state to another. You can document “processes” as being the “most common path”, the “most efficient path”, the “cheapest path”, … just like in a set of roads you may take different paths to go from A to B. Now you could form an “agreement” the participants have to perform the activities that matches the cheapest path. However, it is not preemptive to the fact that the activities are bound to the states and transitions of the resources.
Even though our definitions are unpredictable different we (seem to) agree that there is not a situation where “the set of activities and their sequencing rules cannot be fully known”. We always assume that the set of activities is fully known, what may not be know is the state(s) some of the resources manipulated by the process. Obviously, there are cases with the states, and activities are unknown ahead of time (after all we are inventing new surgery procedures all the time) but hopefully this is out of scope for BPM (at least for now).
You anticipated the clearest example of unpredictability: when a new surgical procedure is invented. The doctor has an “Ah Ha!” moment, and decides to do something that has never been done before. This situation will most likely happen after the patient is admitted (the invention would not have happened without the patient being there) it is therefor clear, I believe, in this case, that is it impossible to predict this procedure at the time that the patient is admitted. This is not out of scope at all – but a real life situation that doctors and hospitals must deal with on the regular basis. I will however broaden my claim to include cases where the knowledge was known outside the organization, but was a completely new procedure to that hospital, and would not be anticipated by them. Finally there are cases where there is no firm agreement on the proper procedure, and so the actual process will depend upon something as arbitrary as who happens to be on duty at the time.
Setting. Just to be clear, I am expecting that the BPM approach is that the patient arrives at the hospital, and that “admitting” the patient mean that a “process” is started for them. BPM is process centric, so a process definition is the organizing principle around which all care will be provided. In order to start a process with BPM, a process definition must exist. That process definition is specified by process specialists: people with special skills and are trained in the art of composing a process definition. In reality there will be many people with varying skill who will collaborate to make the final process definition. It does not matter how the process definition is put together, the only thing that matters for my argument is that (1) it is NOT created on the fly by whoever happens to be standing there, instead (2) it must be created in advance using skill unavailable to the average bystander.
No, this is incorrect. It is like saying, to start a journey, a path must be chosen. All you need is a map, maybe even a compass would do.
I agree a process model is a like a map, and there are choices along the way. I don’t mean to imply that a process model restricts you exactly one course. There are choices along the way. But when I say some processes are not predictable, I mean that the map is not able to be known in advance. The doctor that invents a new procedure is putting a new thing on the map that was not there before because nobody knew about it, and nobody could have predicted that the doctor
The map for a particular hospital might be something like:
1. Have Dr. Jones inspect the patient and determine if there is (a) bleeding, (b) broken bones, (c) head trauma, (d) internal organ trauma, ….
2. if there is bleeding, then have Dr Wilson look at the bleeding and determine (a) is it life threatening? (b) Is it in a place that can be easily bandaged? (c) is there any indication of hemophilia? (d) ….
3. if it life threatening bleeding, then move the patient to ICU, assign more staff, etc.
This will branch a lot, and there are thousands of possible tests as well as thousands of possible treatments. By predictable process, I mean that there is a predefined (branching) sequence of tests and specific treatments. But, I claim that the process model (the map, or state model in your case) for the patient will have to be modified after it is started in a significant number of cases.
Some of the difference in our positions seems to be the difference between “knowable” and “known”. You say: “there is not a situation where the set of activities and their sequencing rules cannot be fully known”. Do you mean theoretically known, or actually known? All we need to do, is to make the state model of a patient: and then we would not need any doctors, or really any nurses either. A complete state model for the patient would tell anyone exactly what to do in any situation. Why doesn’t such a state model exist? Two reasons.
(1) The first is that it is a huge task to gather all that knowledge in one place and to externalize it. A doctor through experience will learn that a certain kind of red splotch in a X rash, while a slightly different one is a Y rash. It is almost impossible to externalize this kind of knowledge. A specialist in a particular disease will be able to look at a patient and draw some conclusions immediately, but it is almost impossible to externalize even just the rules about when to do what.
(2) The second is about agreement: If you were to attempt to codify the procedures, you would end up in arguments about what the right procedure is. I am sure you have had the experience where one doctor says do one thing, and another says do the opposite. A single state-model would require agreement at a very detailed level — and that simply does not exist. Different doctors have different theories but that is not because they are incompetent: each doctor is applying knowledge and learning more about their specialization, in the context where they work. A doctor in Cincinnati may find that a particular treatment works there, but a doctor in Atlanta may find something different. There is no central authority to settle the dispute: and arbitrarily picking one doctor’s approach may be an inappropriate simplification. The reason that there are not strict rules on what doctors do, is that a doctor really does need this kind of discretion to DECIDE the course of treatment, potentially for each different case.
Instead, I see doctors in a continuous state of learning: trying out new treatments, seeing which ones work, attempting to understand the underlying rules. And it is happening in parallel across 10,000 doctors all at the same time. It is constantly changing and evolving, with the introduction of new drugs, new procedures, and new findings about what does and does not work. The course of treatment will depend upon the doctor as well as what the doctor happens to know at that time.
For this reason, I say it is impossible to actually write that state model for the patient. Not only would it be prohibitively expensive to make that model, but by the time you made it, it would be out of date. This is knowledge work, and knowledge is always changing. Thus I say the process is unpredictable.
Mostly, I am worried about the “practical” problems with predicting the process (the people involved don’t know, and it is simply too expensive to collect the knowledge) but the agreement problem even eliminates the theoretical possibility of an accurate state model.
Did I understand your resource state-model correctly?
Keith, when I drive, I don’t stop on the side of the road and start digging a new path. Some people do that. The beauty of the lifecycle model is that you can add states an transitions at any time, anywhere, you can add a resource in an assembly at any time, it all works. There is no “start” there is no “end”, only states and transitions. So again, you can keep ignoring the obvious, but I don’t see a single reason in our entire discussion to ignore the resource lifecyles: agreements don’t work, business process orchestration doesn’t work, BPM as it has been defined for 15 years doesn’t work. There is nothing clearer today. Don’t you think it is time to explore “routes that are not on the map”? It is long overdue if you ask me.
You say “The beauty of the lifecycle model is that you can add states an transitions at any time”. This actually is precisely the point I am trying to make. My approach has always allowed people to add states / transitions / activities at any time. I am wondering if you somehow assume that a “process” can not be adapted in this way, and this is then leading you to believe that it need not be. If work processes as you say are always entirely predictable, if a process is fixed ahead of time and never needs to change, then why is the ability to change it such a beauty?
You also say “Worse, most “applications” that are written are written with hard coded processes.” This too is exactly my point: BPM has promoted the idea that processes should be fixed, and I am trying to argue that the reason hard coded processes don’t work, is because the underlying work is not predictable. Many times you have expounded the virtue of being able to add states and transitions on the fly, but have never taken the step to say that the reason you need to add such states and steps is because it is not possible to define all the states and transitions ahead of the situation. Predictability is a requirement for hard coded state-and-transition models to work.
It is worth exploring routes that are not on the map. But to stay on subject, this discussion is about “why aren’t those routes on the map?” The map contains all the predicted routes. The routes on the map are not predicted. There are types of work for which it is not possible to put all the routes on the map. That kind of work is “unpredictable”, and maps need to be malleable in order to support it.
I collected this from a much larger email stream. I never did get him to agree that there unpredictable work existed. He kept launching into a discussion on how a resource/state model was superior to a process model. Actually, he has some very good points on this, and I have no reason to believe that it is not superior. I hope to explore this in detail some day. But to keep this discussion bounded, I simply wanted to discuss why he felt that unpredictable work does not exist, and (the converse) why he felt that all work was predictable. I am afraid I remain unconvinced that all work is predictable, so we must simply agree to disagree upon this point.