So much buzz about a new emerging category of process technology.  Analysts and vendors alike are talking about it, using a variety of different names: Case Management, Unstructured Processes, Human Coordination Technology, Human Interaction Management, Smart Case Management, Dynamic Process, etc.  I helped lead a Thought Leader Summit meeting on this topic in November in England where key technologists came to debate and attempt to define this trend.

Why this is happening now?  I can identify four broad trends in the history of process technology.  We can plot these trends on a dimension of “unpredictability” over time.  This trend chart is representing the public perception of a particular term or category with regards to the ability to support a particular kind of work.  When a category comes into existence, there are many opinions, and it is supposed that it can cover many different kinds of work.  Different analysts assume different things about the category.  Evidence for disparity in understanding can be seen in discussions like “is Workflow a part of BPM, or is BPM a part of Workflow?” which appear early in the defining of a category. Many vendor products are associated with the category even though those products are not identical, and sometimes have non-overlapping functionality.   As time goes on, there emerges a consensus on what the category is, and what it is useful for.  As new categories appear, the old categories a squeezed into a more restricted/well-defined meaning.

This chart is my impression of how four key trends started without a clear consensus on what the category was useful for, and how over time a consensus emerged mapping the category to a particular kind of work use, and leaving other uses to other categories.  A category that is understood to best support very predictable work patterns will trend toward the bottom of the chart, while something that best supports very unpredictable work patterns will be higher up.

Email – this category actually stands for a large number of different messaging technologies: email, instant message, texting, SMS, social software (Facebook, Linked-in) and more recently micro blogging (Twitter).  In the late 70’s and 80’s, when “office automation” was big,  email was projected to directly support all kinds of work.  Later trends clarified this as being useful mainly for unpredictable tasks, and still today is the primary way that knowledge workers get things done.

Workflow – this category appeared in the late 80’s and through the 90’s.  There were many workflow products, with widely varying capabilities, and it was generally assumed that workflow was useful for a broad range of work.  Some of those products were, and still are, useful for a broad range of work patterns, but the term workflow has come to mean a kind of routing of documents through a set of people.  I put this routing in the middle of the predictability range, because while the set of tasks is fairly well known, the people to do the tasks is constantly changing in any normal organization, and there is a lot of flexibility needed due to the unpredictable changes in the organization.

STP – this category of Straight Through Processing includes a number of different integration technologies, including Enterprise Application Integration (EAI), CORBA integration, SOA Web Services, and other technologies that can be used for system to system processes.  The need to integration a collection of applications appeared in the 90’s when networking first became ubiquitous and was initially thought to be useful for all kinds of processes.  STP is now generally understood to be useful for very predictable processes that can be entirely automated.

BPM – the most controversial category because this there is still widespread disagreement on what the term means.  The initial claim in 2001 was that BPM was the superset of all process technologies, fully encompassing all of workflow and STP as well.  In 2002 the term became associated with BPEL, which is also widely misunderstood.  Eventually a consensus emerged that BPEL is useful for web service orchestration, and for the most part the term “BPM” has come to mean “what BPEL does”.  BPEL4People is an attempt to stretch the coverage upward into less predictable realms, but in general BPM as a category has converged to mean something that is more flexible than STP, and not quite as loose as document routing style workflow.

The point of this diagram is not to quibble about the exact relationship of categories, but to notice that there has appeared a gap in process technology coverage which was not previously apparent.  Knowledge work is neither entirely unpredictable nor predictable enough to use workflow/BPM.  A knowledge worker does not follow any fixed plan, but rather makes up the plan as the work is being done.  Examples of knowledge work include court cases, crime investigations, medical cases, social work and many management and executive functions.  In all of these examples, there are short processes (procedures) which are repeatable, but it take real human intelligence to decide what procedure to bring to bear at what time.  The amount of variation is so great that it is not simply a fixed process with a lot of branches and loops, but in fact knowledge work is creative and often involves a combination of things that have never been done before.

This gap is being encroached from both directions.  We are seeing social software coming from the completely ad-hoc realm and gaining some aspects of predictable work patterns, or at least helping to provide a structure to an otherwise unstructured set of email.  At the same time workflow and BPM products are adding “Dynamic BPM” in order to help support less predictable kinds of work.  Analysts are starting to talk about this new category of process technology, and this subject will doubtlessly be a major topic of conversation in 2010.

This technology gap will require more than a slight modification/extension of existing technology.  It is insufficient to simply make BPMN modeling available on the web to knowledge workers.  Email made more powerful with scripting will not suffice either.  It will probably involve “planning” capabilities, like Microsoft Project, but more oriented toward the immediate evolution of cases (i.e. Collaborative Planning).  A lot of people are calling this a variant of Case Management, but some vendors sell “Case Management” solutions that are little more than a few medical industry specific templates.  To support knowledge workers, this new category needs to have a direct adaptability, it is less programmed, and more customized, to particular cases.

A result of the summit meeting was to form a Linked-in Group called “Adaptive Case Management” which is name we are using until a better one appears.  If interested, you are welcome to join the group to discuss various issues around this new evolving process technology gap and how to fill it.