4 Process Trends & 1 Gap

So much buzz about a new emerging category of process technology.  Analysts and vendors alike are talking about it, using a variety of different names: Case Management, Unstructured Processes, Human Coordination Technology, Human Interaction Management, Smart Case Management, Dynamic Process, etc.  I helped lead a Thought Leader Summit meeting on this topic in November in England where key technologists came to debate and attempt to define this trend.

Why this is happening now?  I can identify four broad trends in the history of process technology.  We can plot these trends on a dimension of “unpredictability” over time.  This trend chart is representing the public perception of a particular term or category with regards to the ability to support a particular kind of work.  When a category comes into existence, there are many opinions, and it is supposed that it can cover many different kinds of work.  Different analysts assume different things about the category.  Evidence for disparity in understanding can be seen in discussions like “is Workflow a part of BPM, or is BPM a part of Workflow?” which appear early in the defining of a category. Many vendor products are associated with the category even though those products are not identical, and sometimes have non-overlapping functionality.   As time goes on, there emerges a consensus on what the category is, and what it is useful for.  As new categories appear, the old categories a squeezed into a more restricted/well-defined meaning.

This chart is my impression of how four key trends started without a clear consensus on what the category was useful for, and how over time a consensus emerged mapping the category to a particular kind of work use, and leaving other uses to other categories.  A category that is understood to best support very predictable work patterns will trend toward the bottom of the chart, while something that best supports very unpredictable work patterns will be higher up.

Email – this category actually stands for a large number of different messaging technologies: email, instant message, texting, SMS, social software (Facebook, Linked-in) and more recently micro blogging (Twitter).  In the late 70’s and 80’s, when “office automation” was big,  email was projected to directly support all kinds of work.  Later trends clarified this as being useful mainly for unpredictable tasks, and still today is the primary way that knowledge workers get things done.

Workflow – this category appeared in the late 80’s and through the 90’s.  There were many workflow products, with widely varying capabilities, and it was generally assumed that workflow was useful for a broad range of work.  Some of those products were, and still are, useful for a broad range of work patterns, but the term workflow has come to mean a kind of routing of documents through a set of people.  I put this routing in the middle of the predictability range, because while the set of tasks is fairly well known, the people to do the tasks is constantly changing in any normal organization, and there is a lot of flexibility needed due to the unpredictable changes in the organization.

STP – this category of Straight Through Processing includes a number of different integration technologies, including Enterprise Application Integration (EAI), CORBA integration, SOA Web Services, and other technologies that can be used for system to system processes.  The need to integration a collection of applications appeared in the 90’s when networking first became ubiquitous and was initially thought to be useful for all kinds of processes.  STP is now generally understood to be useful for very predictable processes that can be entirely automated.

BPM – the most controversial category because this there is still widespread disagreement on what the term means.  The initial claim in 2001 was that BPM was the superset of all process technologies, fully encompassing all of workflow and STP as well.  In 2002 the term became associated with BPEL, which is also widely misunderstood.  Eventually a consensus emerged that BPEL is useful for web service orchestration, and for the most part the term “BPM” has come to mean “what BPEL does”.  BPEL4People is an attempt to stretch the coverage upward into less predictable realms, but in general BPM as a category has converged to mean something that is more flexible than STP, and not quite as loose as document routing style workflow.

The point of this diagram is not to quibble about the exact relationship of categories, but to notice that there has appeared a gap in process technology coverage which was not previously apparent.  Knowledge work is neither entirely unpredictable nor predictable enough to use workflow/BPM.  A knowledge worker does not follow any fixed plan, but rather makes up the plan as the work is being done.  Examples of knowledge work include court cases, crime investigations, medical cases, social work and many management and executive functions.  In all of these examples, there are short processes (procedures) which are repeatable, but it take real human intelligence to decide what procedure to bring to bear at what time.  The amount of variation is so great that it is not simply a fixed process with a lot of branches and loops, but in fact knowledge work is creative and often involves a combination of things that have never been done before.

This gap is being encroached from both directions.  We are seeing social software coming from the completely ad-hoc realm and gaining some aspects of predictable work patterns, or at least helping to provide a structure to an otherwise unstructured set of email.  At the same time workflow and BPM products are adding “Dynamic BPM” in order to help support less predictable kinds of work.  Analysts are starting to talk about this new category of process technology, and this subject will doubtlessly be a major topic of conversation in 2010.

This technology gap will require more than a slight modification/extension of existing technology.  It is insufficient to simply make BPMN modeling available on the web to knowledge workers.  Email made more powerful with scripting will not suffice either.  It will probably involve “planning” capabilities, like Microsoft Project, but more oriented toward the immediate evolution of cases (i.e. Collaborative Planning).  A lot of people are calling this a variant of Case Management, but some vendors sell “Case Management” solutions that are little more than a few medical industry specific templates.  To support knowledge workers, this new category needs to have a direct adaptability, it is less programmed, and more customized, to particular cases.

A result of the summit meeting was to form a Linked-in Group called “Adaptive Case Management” which is name we are using until a better one appears.  If interested, you are welcome to join the group to discuss various issues around this new evolving process technology gap and how to fill it.

10 thoughts on “4 Process Trends & 1 Gap

  1. Keith,
    Hi. Very interesting post. I think another reason that the subject of human processs is coming up now, is that there is a lot more focus on process as a result of the focus on BPM. As a result of becoming more familiar with most BPM suites – organizations are finding that many processes can’t be handled at all – and even most structured real world end-to-end processes also have an unstructured component.

    I wouldn’t be so fast to write off enhanced email. I think technologies that merge email, documents and processes (as we do in ActionBase,as Google Wave hints at) could be a good enough ACM in many cases (pardon the pun).

  2. Hi Keith,

    Interesting. I also think that Google Wave will undoubtedly play a very important role in what you are calling Adaptive Case Management.

    Also, I would probably recommend breaking out another category – ECM – and putting that on your chart, and renaming Email to “Messaging.” I think that might add a bit more clarity to your trends chart.

  3. I am a big fan of Google Wave, and I think that wave will help all categories across the entire spread. I see it as a new way to collaborate on content, and since content is always communication, there is a strong relation to process. But I see it helping in very formal processes as well as very unpredictable processses.

    Same sort of thing with ECM. It is useful in very predictable processes, as well as very unpredictable processes. So I can’t really put it in a particular spot on this dimension.

    I only included trends that dealt directly with representations of activities — work — and one category that has no representation of work: email.

    Email, as you suggest, could be called messaging. It should include telephone calls as well as physical mail. What I mean is to include all means of communications where the decision to initiate is entirely manual: some thinking person has to decide “I am going to write an email now….” They often do this because according to a process encoded in the habits and practices they “have to” do it.

    Jacob, I do think Action Base is an early mover in this gap, but not because it is based on email. Remember that some workflow is based on email. Rather because it is offering some aspects of “adaptability” when representing process.

    Maybe the email category should be called “Manual”.

  4. Keith,
    I agree that the fact ActionBase is based on email is the Sizzle and not the Steak. The real value is in its ability to manage and track unstructured processes. That being said I think doing so in email does bring to the forefront some the basic differences between standard BPM and Adaptive Case Management:
    1. Focus on the end user experience not IT. BPM leaves the end user experience up to the IT department – not always the best choice.
    2. Put the users in charge, for the most part they need to be in control of the process. For knowledge work I believe there are very few instances when things must be done in a certain way – knowledge workers have a lot of freedom in how to get things done. When they decide to stray from the standard, they can and the new process should be tracked. You can’t force them to do things in a specific way.
    3. Don’t disregard the way people get thier job done today. Business processes existed before BPEL and I have never seen an instance of business process innovation as a result of BPEL (optmization yes, innovation no). I think there is benefit in an evolutionary approach (let the user stay in their standard tooling and enhance that tooling) vs. a revolutionary approach (throw the old away and start again). I think revolutionary approaches win only when they really change the status quo for the better. I don’t think BPM has achieved that.

  5. Pingback: links for 2009-12-04 « steinarcarlsen

  6. Hi Keith,

    Nice summary on process trends and technologies. However, I missed one use case, which is actually inherited from both workflow and BPM, applications development.

    Process technologies main focus should be applications development and maintenance improvement. Process automation is great but customers requirements are most of the time related to how to improve or implement applications. Process technologies can bring a great value in the way applications are developed and maintained in companies.

    best regards,

  7. Where do Knowledge Management tools like Wikis fit in your model? Are they a part of ECM?

    I’ve found Wikis useful for putting structure around information previously shared via ad-hoc messaging, but not yet stable enough for automation.

    Also, is there also a temporal progression for any given process that starts out at ad-hoc messaging when in its infancy, then acquires more structure as knowledge, methods and best practices precipitate out until all or part of the process becomes mature and stable and can be automated? If so, wouldn’t the choice of tool or methodology follow that progression?

  8. Mainly this focuses on communications tech. To the extent that Wikis, blogs, and ECM are communication, they would go in the same category with Email because they have no explicit representation of time / process.

    There are some things which start out without a process, and gain process as you work with it, and that is the technology that goes in the gap. Calling this Adaptive Case Management, because they start with a representation of the “case” without process, but gain a process (through adaptation) as people work with it. There are a few example of this today, but not enough that I would call it a trend yet. This will be the area of growth in the coming year.

    The “unpredictable/predictable” dimension some people like to call “structured/unstructured”. I don’t like this, because as you point out, a wiki can give you the ability to “structure” the information. But without an explicit representation of process (time) I would not position a wiki lower than the top of the chart. However, you might easily imagine a wiki that allows incremental, collaborative construction of a process … and that would belong further down into the predictable realm.

  9. Pingback: The Five Things I Learned at the "Adaptive Case Management" Conference | ActionBase Blog - Thoughts on Collaboration Process Management Unstructured Compliance and Audit

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s