Message senders do so with all the best intentions, but there is a better way to share documents, if only they would take the time to learn how.  Instead, they unwittingly propagate a pattern that wastes wastes time from everyone.  It is really not that hard to put the document in a document management system, and then mail a link to it.  Why is uptake so slow?  To help them along on the road to wisdom, here is the message that I would like to email back in response:

Dear Sender,

I writing in response to that document you just sent me by email. I wanted to let you know how much I appreciate it, which is not at all. I am sure that the document you sent might be useful to me someday, but for that to happen, I need to store it someplace where I will find it again when I need it. That means I have to read enough of the document in order to figure exactly how it should be classified. Even then, it can be ambiguous: do I put this under a particular customer name, and sometimes there are multiple, or under a salesperson’s name, which also might be multiple, or under the product name, which also might be multiple, or maybe a project name, or under the date I received it, or the date it was written, or possibly even the date on the title page of the document.

I think you can see that this only takes me a minute or two to decide where to save the document, but consider this: I receive dozens, if not hundreds, of documents every day, and this ends up taking a good part of an hour every day.

Wasting just my time being might be acceptable, but usually these documents sent by email are copied to tens or hundreds of other people, and every one of them goes through the same process. This means that across the company of 100 people several weeks of time is wasted every day, just to find the right place to store such documents.

That waste might be acceptable, except for how pointless it is. In most cases, the documents I receive are not the absolute final versions, and it is quite likely that before I get the chance to actually use that document, a newer version will be sent out. I will then waste the same amount of time finding a place to save that, but because it can be ambiguous where to save the document, and because priorities change over time, I am just as likely to save the new version in a different place. Then I will have two versions of the document floating around. Thus when I go to find the document, I am quite likely to find the wrong version, and not get the latest information, and that might be worse that not saving the document at all.

I am sure that you meant well, and imagine how awkward it is for me to complain about the document you are sharing with me. You are surely motivated by a desire to keep everyone informed, but probably don’t realize that this means of sharing causes a tremendous amount of work for everyone else.

May I suggest a better way: put the document in a repository or document management system that is accessible to everyone. Spend a few minutes tagging the document with the appropriate key words so that others can find it with a search. The message you were writing to me: instead put that in the description of the document. Then email me a link to the document letting me know it is there.

This way, I am informed about the new information, and can read it immediately if I want to. I can also be assured that it will be accessible in the future, if I need it, and I will be able to find it by searching at the normal document management system. If new versions of the document are produced, then put those new versions in the same place, on top of the older version, so when in the future I need this document, I will always get the latest one.

Yes, I know this is a little more work than just mailing the document as an attachment. But a couple of minutes of your time could save hours across the entire organization. If everyone does this, we all benefit. There will be fewer documents flowing through the email, we will all be spending less time saving and organizing documents, and we will be building a strategic library of documents that can be searched and found WHEN YOU NEED THEM.

As they say, the road to hell is paved with good intentions, and your intention to share important information with colleagues was honorable, it is precisely this behavior that makes us weaker and less effective. Please be more considerate in the future.

Yours Truly,  Recipient

That is it.  It would be so simple, and such organizations would be more effective.  I fear that the incentives are all wrong.  If I put a document in a DMS, I never quite know if everyone else can access it from there.  I never quite know if the link is going to work for them the same it does for me.  And, if this document is sensitive, I can never be sure that the access control is correct so that the right people can see it, and only the right people.

True, it does take a little more effort to put the document someplace.  It is soooooo easy to just attach it to the message I am sending.  Fewer systems to coordinate: everyone who gets the message gets the attachment (unless removed by anti-virous software).  Why should the sender worry about the time wasted by the recipient, anyway?

I am doomed to be an email attachment librarian for the foreseeable future.

John: To help our readers gain some perspective on your background, could you share some thoughts on what you do when you are not writing?

Keith: I am really a software architect: I help design system and that requires that standard processes and procedures be put into place within the entire team of developers. To do that, I do a lot of thinking about how to design rules that people can work with. In the early 1990′s I got involved in a number of standards efforts: OMG, Case Communique, and Workflow Management Coalition. This was because I saw the need for design rules across the industry, and these organizations were the best ways to accomplish such things.

John: What inspired you to write your first book?

Keith: It was a long time coming. In the early 90′s I got my first papers published in conferences because Fujitsu had no real mechanism to publicize ideas, and conference proceedings seemed the best way to get in contact with others who care about the topic. From there I moved to tutorials and writing the materials for that. Then a German professor contacted me to let me know he was using my tutorial notes in his college class on Business Process Reengineering and Workflow. I self published four books from tutorial notes and from other documents that I had developed over the years for guidelines on software development techniques. In the mid 2000′s I participated in a chain of technology tutorials with the WfMC. Robert Shapiro and I recorded one of our sessions and turned that into a book on BPM standards. I spend so much time trying to get the concepts clear to me, that I really was compelled to write the stuff down and make it available in a reproducible form.

What was the motivation behind your research and writing on the subject of knowledge work and Adaptive Case Management (ACM)?

Keith: My goal has always been to find a way to support office workers. I thikn I was always considering the creative work the knowledge workers do, although I was not using that term then. If you read my papers in the 1990′s you will find discussion of how each team, and potentially each worker, needs to find the right fit to their own needs. The idea behind “Collaborative Planning” was that getting together to decide what to do was an important part of actually doing things. This is a BIG difference philosophically from those who believe that all work can be automated, and that you can eliminate humans from work entirely. The Workflow Management Coalition always included the idea that some work can be automated, but some work inherently must be done by humans, that is enshrined in the Workflow Reference Architecture produced in 1995 and still relevant today. I, and many others in the BPM field, were struggling to support humans along side the automatable data processing. It was in 2009 that it became clear that the definition of BPM had “collapsed” in the public eye to mean only automatable work, and specifically orchestrating data flow between servers. People felt that BPM was a part of SOA. Many of us were frustrated by that, because there is so much that can not be automated and required human intelligence. The key concept was that people were doing work that could not be predicted in advance, and so predictability became distinguishing aspect of what was and was not knowledge work. We had a meeting in Maidenhead England where the WfMC invited many of the top thinkers on this topic. Eleven of the 12 authors of Mastering the Unpredictable were at that meeting. We felt that the subject was so important, and so urgent, that we decided to share the effort and write it all down. Five months later Mastering the Unpredictable was released to the public.

Are there any sections (or elements) in “Mastering the Unpredictable” that you are particularly passionate about?

That is a little like asking which of your children you like the best! The book progressively discloses ideas, and so the first chapter covers the concepts at a high level, and chapters 4 and 5 get into a lot of important details. Then the rest of the book reflects on use cases and how the technology fits into different fields. We know so much more know about the field, but still I think the book stands on its own for giving a cogent picture of what is possible.

Are there any particular challenges that come into play with ACM? Can you recommend any best practices?

The biggest challenge is getting IT departments to realize that they can’t automate the creative parts of the organization. They have a “Jetson’s Mentality” thinking that all work will ultimately be reduced to a single button press. At the same time, the people who lead creative organizations, and know that it can not be automated, tend to shun al technology because of the way that the IT department tried to reduce all work to automation. Creativity comes so naturally to people that we don’t realize what we are doing and how we do it. So we find that we have to sell not a technology, but a management philosophy. Many manager believe that there is actually exactly one way to do things, even though their teams are constantly changing form and method. Because they are not faced with the reality that work processes are continually in flux, they tend to believe that there must be one single best way to do something. Getting past the “Newtonian Illusion” that an organization is base on fundamentally simple rule is the biggest challenges to deployment of ACM.

Do you have any advice on how to incorporate social media into managing the unpredictable?

I am always careful to distinguish “social media” from “social technology” – they are similar technology, but distinct uses and benefits. Social media is typically used to broadcast to — and from — masses of people. Social technology however is a set of capabilities that take social network relationships into account. In many ways ACM *is* social technology because it makes use of explicit representation of relationships between people to support the work on a particular case.

Do you have any case studies where you’ve combined social media with ACM?

I would say that ACM is about supporting case managers to accomplish projects and get things done. One of the things they may want to do is to engage the public using social media and their use of social media will be not significantly different from those who don’t use ACM. One of the use cases covered in the book is “New Product Development” and clearly when you release a product, you are going to want to leverage social media in the normal way. So they go together but I don’t see any specific dependence between the two subjects.

In hindsight, are there any features of your published works that you would change or build upon?

There have been way to many discussion of the comparison of ACM to BPM. This is because we accurately predicted that those who see the world as something to be automated, would perceive ACM as being the same as BPM, and we wanted to preempt the discussion and try to make it the distinction clear from the start. However, most of this discussion is in vein because those who try to automate all work tend to think only of automatable work cases are important, and dismiss the unpredictable work as either nonexistent or unimportant. If I were to do it over, I would probably simply ignore the BPM comparison, simply focus on those organization that depend upon innovation and creativity, and show how ACM can support that work, without any comparison with BPM. It seems that the knowledge work support, and routine work support, are distinct problem that can be handled by different support organizations.

What are your current/future projects?

Organization are realizing the value of supporting knowledge workers, particulary that knowledge work is the most important element of competition in the coming years. As routine work is automated, what is left is work that requires a human intelligence, and better you can leverage human intelligence the more competitive you will be. I am helping companies to select the right technology and configure it for use in supporting knowledge workers. I am trying to work with a group of people to define more clearly what is and is not essential as part of an ACM package, and if things work well I would like to develop interoperability standards to allow various ACM system to cooperate with each other.

Can you share some of the feedback that you’ve recieved for “Mastering the Unpredictable”? Have people in the field embraced the concepts outlined in the book?

There has been a big and positive response among those who have to deal with creative knowledge work. Forrester have published a rating of case management products (they call it Dynamic Case Management). IBM calls it “Advanced Case Management” and people from TJ Watson Research Center have published paper recently on similar topics. A proposal have been submitted for holding the first International workshop for Adaptive Case Management in Sept 2012. Even with this response, we are still just seeing the beginning of movement. Most IT Departments are still focused on automating routine work, and for good reason: there is still a lot of low-hanging fruit or routine processes that really need automating, and such automation is saving organizations tremendous amounts of needless human activity. But over the next 10 years those routine processes will be mostly automated. What will IT departments do when all the routine processes are automated? They will implement ACM system to facilitate knowledge workers, and such support will will make the difference between a winner and loser in the marketplace. It will become a key strategic factor for a company.

Any recommendations on how to get IT departments on board with ACM deployment?

The most important thing is to understand the principle of “unpredictability”. I use the example of a search and rescue team to demonstrate the kind of decisions-on-the-fly that are involved. A search and rescue team prepares in advance, but not by making specific pre-defined actions, but instead to practice “patterns” of working together that can be reused. Think of this like different “plays” in American football: the team practices how to interact using different plays, but the game is never scripted, and even a give play has variability that is decided at the time of the play, and by the players as the play works out. The launch of a new movie makes use of patterns of working together (booking first run theaters, billboard advertisements, promotional products at fast food restaurants, etc) but no two movies are ever launched in exactly the same way, because every launch needs to take into account many factors of current events, culture, fads, trends, as well as competition that is being launched at nearly the same time. IT departments need to acknowledge that in certain types of work, there is no single process, but instead the process followed is different every single time.

Another example I often use is Dr. House the television show because it demonstrates in nearly every episode that the information necessary to predict the correct treatment is not available up front. The patient is dying, and something has to be done, but nobody knows what. After treatment starts, strange responses to the treatment give additional clues about what is wrong, causing the doctor to decide to change the treatment as it progresses. This idea of insufficient information up front, and evolving the plan as the treatment is being given is an essential aspect of Adaptive Case Management. Many of the patients have combinations of problems. Statistical analysis shows that even combining only two illnesses can give you hundreds of millions of combinations, implying that every such patient that comes to a hospital is potentially unique. The idea that there are a small number of standard treatments that can be prescribed up front must be abandoned.

Once the IT department understands that things are not predictable up front, once they have let go of the Newtonian idea that there is a simple rule and the bottom of all behavior, then it is fairly easy to get them to see that ACM provides a way to put an intelligent human at the center of the work. Instead of a factory that is automated to eliminate all human involvement, the IT system becomes a kind of “bionic limb” that allows people to access information faster, sift and sort it more effectively, communicate to others more effectively, and to coordinate the tasks of other people more powerfully. The IT systems become extensions of the decision makers, not the replacement of them.

For many, when they realize this, they also realize that they have always known that executives would never be replaced by IT systems. For many, this is a realization that IT systems can support more kinds of work than they ever thought possible before. Instead of being an approach that competes with existing ways of designing and implementing applications, this become a way to extend IT systems to jobs that have never been able to be effectively supported before. It is a new opportunity for IT departments.

What is your vision of the long range future?

If you indulge me in a bit of wild speculation, we might glimpse at the far future. In 10 to 20 years we will see the transformation of business such that all workers are executives. What I mean by this is that work will consist of making decisions: someone will have to decide how much money and resources to invest in a particular initiative, but the actions resulting from that decision will be largely automated. I don’t mean to imply that we all will live like some futuristic Henry VIII. There will be hundreds of millions of other executives to deal with, so the chief decisions will be about strategies to get others on board with your initiatives — or more frequently on whether to get on board with other initiatives. Once the decision is made to do something with all the right people involved, all the rest will be essentially automated and will proceed without any “work”. Decide that you want the latest electronic gadget and it will be manufactured for you in Siberia and automatically shipped to your doorstep and possible even set up automatically for you. Decide that angioplasty is right treatment for this patient, and all the rest of the preparation, handling, exchange of money, and supply of materials is done automatically, and possibly even the surgery itself (although I expect innovative surgery will be one of the last areas of automation because surgeons need to be very adaptive as they work). Decide that a particular product line is unprofitable, but that there appears to be an uptick in demand for another product line, and this can be automatically communicated to all the right people, triggering a cascade of decision making through the organization, but much of what we consider today as “work” will be fully automated. It is not a panacea: decision making can be more stressful and more exhausting than what we call work today. Furthermore, the effect of poor decisions will be amplified the same way that good decision are, and so the pressure to make the right decision increases. Progressive companies are recognizing this trend today, and preparing by focusing on getting good decision makers, and on IT systems that support decision making such as ACM.

Again, thank you so much for your time on this.  It was a pleasure chatting with you, and I think our readers will really appreciate this opportunity to further connect with the concepts that you discuss in your writing.

My pleasure as well.

You memories are not perfect, but the interpreter can help to fill in details which are plausible.  Evidence of this is seen from how people who have suffered trauma that eliminates some of their real memories, will confabulate things to fill in the gaps, in a way to make a consistent narrative.  We do this all the time, and if you memories are complete enough, and your understanding of the context good enough, this confabulation works in your favor.

This tendency to create a plausible narrative effects process design in two ways.  The first is that when people are interviewed about a given process they have been taking part of, the interpreter part of the brain will make up things to fill in the gaps in the process story that they might not have been aware of.  Indeed, while working in the process, people will have a narrative of how the process proceeds outside of their direct interaction that may be completely inaccurate.  That narrative, however, help to support their own part of it.  Memory works by having a consistent story, not necessarily an accurate story.  They may hold false beliefs about a process, but as long as they do their part correctly there is no harm.  It may be hard, however, for a process researcher to distinguish the parts that they actually know, from the parts that they filled in to make a consistent narrative.

The second reason it gets in the way is that when the process changes, the narrative can change along with it, and in doing so the original process is forgotten.  Organizations are constantly changing around us, but as we learn of a change we incorporate it into our narratives, and forget that there was a change.  Because the memory has a filled in narrative, it is hard for people to remember the exceptional cases.  People will often insist that they have been running a process the same way every time, until you remind them of a particular exception they handled.  Then the remark is usually “oh yes, that did happen.”

I have written about “Process Confabulation” before as a danger inherent interviewing people to find out the existing process.  I found Michael Gazzaniga’s video interesting because it explains how this is caused by a basic element in the way memory works.

Official Title: The First International Workshop on Adaptive Case Management and other non-workflow approaches to BPM (ACM 2012) in conjunction with BPM 2012

Goal: While practitioners are trying to overcome the restrictions of workflow thinking, the research on the topic is somewhat lagging. The goal of this workshop is to bring together researchers and practitioners to discuss theoretical and practical problems and solutions in the area of non-workflow based approaches to BPM in general, and ACM (as a leading movement) in particular. This workshop is aimed to promote new, non-traditional ways of modelling and controlling business processes, the ones that suit better the dynamic environment in which contemporary enterprises and public organizations function.

Important Dates:

• Submission deadline: 1 June 2012
• Notification due 2 July 2012
• Camera-ready submission deadline: 30 July 2012
• Workshop: 3 September 2012

Submissions Categories:

• Position papers raising relevant questions in the workshop area, identifying problems and providing a glimpse of solution for a given problem. Representing a basis for discussion, a position paper does not necessarily need to include solutions to its stated problems. Position papers must not exceed 4 pages.
• Idea papers exploring the history, successes, and challenges for various non-workflow approaches to BPM and outlining research roadmaps for the future. Contrary to short position papers, idea papers should provide the in-depth analysis of a problem, review its existing solutions, demonstrate insufficiency of these solutions and suggest new (yet unevaluated but well argued) solutions. Idea papers must not exceed 12 pages.
• Experience reports presenting challenges encountered in practice, their related case studies, success and failure stories. An experience report should clearly describe the working context, and focus on the problem and on the lessons learned. Experience reports should be complete and allow for rigorous testing of research theories methods and tools. Experience reports must be limited to 5-12 pages.
• Research papers reporting original results in the area addressed by the workshop. A research paper should clearly describe the problem tackled, explore the relevant state of the art, describe the proposed solution and provide a preliminary validation of this solution. Research papers must not exceed 12 pages.

The real motivation for holding a workshop came from Irina Rychkova of the University Paris and Ilia Bider of Ibisoft in Stockholm who both got me involved as well.  Those who know me well may find it ironic that first academic workshop is being held in conjunction with the academic BPM conference.  Indeed, I criticized the BPM 2010 conference for its complete lack of case management topics.  While many proponents of BPM seem entrenched in Taylorist ideas that behind every job is a simple fixed process, there are many others who are searching simply for ways to make workers more efficient, regardless of whether the process can be predicted or not.  The forming of this workshop is evidence of that, and association with the well respected BPM conference series, it is likely that this workshop will include many well considered rigorous papers on Adaptive Case Management research.

I have been to Tallinn only once before, when I took the ferry across the Baltic from Helsinki where I was working on the TeamWARE Flow product in the early 1990′s.   That visit was marvelous, and I can only expect that Estonia much have changed remarkably in the long time since it has been out from under the shadow of the Soviet Union.  I am looking forward to it.

This is the third of three ACM events already planned for 2012, the other two are the ACM Awards, and the ACM Live Virtual Event in June.

I don’t yet see the detailed agenda yet of the event — that announcement is still yet to come.  It is not my intention to scoop the announcement, but what I do know is that that we are planning to announce the winners of the 2012 International ACM Awards at this event.

Last year’s event featured one-on-one interviews with many top speakers on Adaptive Case Management as well as a keynotes by Jim Champy and Tom K himself.   Also, we announced the winners of the 2011 ACM Awards there as well.  Because the entire two day event is virtual, it is easy to attend from anywhere. And all the talks and presentations were video recorded and so they can still be viewed now, on demand.  I felt the event was well organized and represented a solid way for people to become much more knowledgeable about Adaptive Case Management.

I will update this entry when I have more specifics about this event.

Last year the WfMC sponsored the ACM awards and it was a huge success.  From the submissions, the panel of world renown judges selected the best 10 examples of the use of case management.  These were then featured in the First ACM Virtual Summit, and then in September the case studies became available to everyone in the form of the first ACM case book called “Taming the Unpredictable.”  The feedback has been incredibly positive.  It is very vaulable to have a book that describes in detail how people have actually used case management techniques to accomplish business goals.

This year we want to make it bigger and better.

• We are going to spend more effort up front making sure that everyone knows about it, and attract a larger set of submissions, and hopefully select more finalists from that.
• The judging criteria will be more refined after what we learned last year.
• We already know to plan for the virtual summit which will encourage more people to participate.
• We have an example of a successful book which will attract interest in submissions.

This format, of accepting case studies, judging, selecting, then presenting, and publishing in a book is a accepted way to gather the best examples into a place for delivering to an audience.  It is clear to me that the more that the public knows about this approach, the better they will be able to make appropriate use of it.

Most of the information you need is at the Adaptive Case Management Awards site.  The schedule is as follows:

• Any time between now and Feb 28 register an intent by submitting an abstract.  There is no risk, and those who do this early will get some feedback and guidance on the abstract.
• Before March 15 officially register.
• Before April 20, submit the finished case study for judging.
• June 6, winners will be announced at the ACMLive Virtual Summit
• September, the associated book will be launched

Mark these dates on your calendar, you don’t want to miss out just because you missed the deadline.  I have a list of questions an answers below:

Must the case use a bona fide ACM product?  No.  ACM is an approach to supporting knowledge workers, not a type of product.  Many of the submissions last year were on custom home grown systems.  Some other cases were hosted on systems designed specifically for ACM, while still others were build on systems that had to be, shall we say, adapted into supporting case work.  Instead of looking at the system, you should ask the question of whether the case study is about supporting knowledge workers, and if so that would most likely be appropriate.

What really constitutes knowledge work?  A good reseource for this is my chapter from the 2010 BPM and Workflow Handbook on Business Intelligence which I link here for convenience (pdf):  Knowledge Work and Unpredictable Processes.  The publisher has made other books available to registrants at a discount.

What about Enterprise 2.0, or Social BPM Cases?  There is a big overlap between E20, social business, and enterprise social with adaptive case management.  See the judging criteria to evaluate whether your E20 case might qualify.  The case must show how knowledge workers are supported while getting work done.

How about Just Plain Case Management?  Be aware of the importance of adaptability.  Read my post on Understanding what “Adaptive” means.  For example, telephones clearly support knowledge workers, but they are not adaptive in any significant way.  Judges will be looking for some ability to mold the system to the individual knowledge worker over time.

Do you need more judges?  I am hoping to expand the pool of judges this year, and if you are have made a contribution to ACM discussions in the past year, please send a message to .

Stay tuned, I still have to cover the following posts:

• The ACM Live Virtual Summit in June 2012
• The 1st International Workshop on Adaptive Case Management (ACM 2012) in conjunction with BPM 2012 in September

Traditional Process Life Cycle

The traditional process life cycle for BPM has been set out in a very definite form.  We start by designing a process — maybe laying out the boxes on a flow diagram.  We interview people, and ask them what they think the process is. Or maybe we use a tool to allow business people to collaborate directly on describing the process.  Implementation then starts, and that can either be a model preserving strategy where the process diagram is interpreted directly, or it is somehow transformed to an executable form.  The completed application is tested in the standard manner, and finally deployed into actual production use.

After deploying, we can switch to a number of different tools to monitor and measure the success of the process, like process analytics, history, or even just simply asking the users where the process is working, and where it does not. We use that insight to improve the process, and after testing, the improved application is deployed to production.

This is not just the BPM life cycle; everyone in the IT department knows this is right and proper way to make an application or solution of any type.  BPM offers some special capabilities through more powerful tools, but the life cycle is the same.

Flipping the Life Cycle stands this Approach on its Head.

Step 1: deploy the system into production with real users.  There is no need to develop an application or a solution, they simply start using it.  The system itself is useful without any modification, the same way that telephone or email are useful without building an application.

Step 2: After it has been in use for a while, use process mining to see what the process has been.  Process mining gives you an aggregate picture across an organization of any size, picking out the most common processes, and the exceptions.   You have access to metrics telling you how long workers spent in any given step.  It can show you what percentage of the time the work proceeded done one path, or down another.  From this you understand how to improve the work of the organization.

Flipping the life cycle is an extension to the basic Adaptive Case Management (ACM) pattern.

Why Flip the Life Cycle?

Flipping the life cycle is a useful technique in a number of situations:

• The process is too vast to automate.  Consider a patient today interacting with a healthcare supplier.  There are literally thousands of reasons that a person might need to interact with a doctor today.  It is simply not possible to ask everyone to stop getting healthcare until we have had time to automate every possible interaction.  While parts of this interaction is being ever more automated, but  show must go on, and it will be a long long time before that job is complete.
• The work is too complex to automate.  You think that it is impossible for a process to be too complex?  You think that a programming language can handle any degree of complexity?  Tell that to someone who is negotiating the merger of two companies.  The complexity is great, and even the number of factors have not been enumerated.  Case managers are doing these job today, and they will not be automated any time soon.
• The work is too unpredictable.  Anyone who has followed my writings has seen plenty of examples of processes that are done only once, and then thrown away.  The board of directors that asks a company to shift the focus of a product line in a new direction, or to consolidate two different departments into one.  It simply is not economical to automate processes that are different every time.
• The work requires a person who has specific knowledge to that specific particular situation.  These are what Jacob Ukelson calls a knowledge process, and they do not have the same characteristics as routine business processes.

Flipping the process life cycle avoids the up-front expense and lock-in to a suboptimal path while at the same time gives you many of the advantages of being able to measure the process performance, and find ways to improve the work.

What Difference Does It Make?

With Adaptive Case Management (ACM), we often say there is no life cycle to mean that there is no separation between design and doing.  There is no distinction between the development environment and the run time environment.  Planning is part of doing.  In fact, you don’t really design a process, you really just plan your work.  Planning is somewhat like process design, but it is very much less like programming, and a lot more like communicating.

Planning for knowledge work has to be done by knowledge worker themselves, without specialized skill in designing processes.  The doctor himself plans the treatment.  The detective herself plans the investigation.  These people are specialists in their field, but have no special skills in process design.  ACM must not require specialized process skills.

Note that the traditional process life cycle enables different technical specialists to be involved at different times: at the very beginning high level requirements are drawn up by the business owner; then business workers help define the process; then the process specialist (a.k.a. process analyst) figures how best to express this as the process model; then developers and testers may be involved in crafting the application, forms, data persistence; finally an administrator will deploy and manage the final application.  The fact that you have specialists for process support means that you can have much more specialized tools, and that drives a kind of arms race of features for more powerful process support.

I have nothing against powerful tools for developing applications, but they quickly become so specialized that a typical doctor, policeman, lawyer, judge, or nurse can no longer use them effectively.  The highly specialized tools distance the knowledge worker from the planning activity.

Many knowledge workers will plan and complete their own work. They are evaluated by the normal means: satisfied clients and financial measures.  When warning signs appear, managers of large organizations will want to know more about what is going wrong.  Post facto process mining gives those managers many of the benefits of a rigorous process: they have visibility to what has been working well, and they have a visual representation of why particular processes did not go well.

The traditional life cycle introduces a delay between the design of the process, and the actual use of the process.  This is not a technical delay, as most BPM vendors will show you one-button deployment that can put a process on-line in seconds.  Instead the delay is caused by more mundane reasons having to do with approvals, sign-offs, testing, debugging, and simply manpower available to do the improvement.  By flipping the order of these, and you can actually compare the work with a process that was designed after the work was done.

If you think about this last idea it is really amazing:  If your process lasts years, you can monitor progress, and discover practices from ones going well, and then apply those practices to other cases — even when those techniques were not known when the work was started.

Background

Process mining is the scanning historical records of activity in order to determine the process that had been used.  A process is simply a sequence of actions.  Process mining uses recorded evidence of many such sequences to give you a picture of the typical path and exceptional paths as well.

Research on this field started in the mid 1990′s.  This means it is still a relatively young field, it has come surprisingly far in a decade and a half, and can be used successfully in business situations today.

Fujitsu launched process mining as a service offering three years ago, called Interstage Automated Process Discovery.  I have been involved in the launch and evangelism of this new approach for understanding and optimizing business.  See this 9 minute demo and this podcast on Integration Developer News.  My most recent webinar last month to ASQ was on how process mining can be used to support Lean and Six Sigma methodologies.

Why a Manifesto?

I have personal experience with using the approach, and the results are dramatic.  A simple, two week consulting engagement can give a medium to large organization a number of tips that will certainly save that organization millions of dollars.

I recently attended a presentation to a fortune 50 company where some Fujitsu consultants had spent a few days analyzing some of their order fulfillment history records.  Focusing on operations in two countries, we were able to pin point individual products which had handled incorrectly.  We were able to show how sales progressed differently in the two countries.  We could break this out by specific product line, and could tell them, for example, how long different models were typical delayed in customs or other processing.  I remember particularly the enthusiastic response of one IT executive saying “This is witchcraft!“

It is not unusual for people who hear about the approach without experiencing it, to believe that it is too good to be true.   They have a hard time believing that it really works, and it is pretty amazing.  Other members of the IEEE Task Force have run into the same thing: there is not enough known about process mining, and it seems too good to be believed.

This drove the need for a Process Mining Manifesto to clearly define process mining and to help people understand the real promise of this new technology.  It was written largely by Wil van der Aalst and his colleagues at the Technical University of Eindhoven, but also with representation from a broad range of contributors across industry and academia.  It is now being translated into a dozen languages.

If you are not already familiar with process mining, the manifesto is probably the best way to come up to speed quickly.  If you have a high level understanding, the manifesto will give you a solid base to understand in detail when you might want to make use of this approach

Relevance to ACM

Process mining is especially important to Adaptive Case Management (ACM).  When it is impossible to predict the process ahead of time, knowledge workers need to move forward without a predefined process.  Process mining promises a way to retrospectively see what the pattern of work turned out to be, even though it was planned as the work was being one.

Process mining can give you metrics about the process which can be used to evaluate how well the work is being done.  Even though the process is not automated, you can get many of the benefits normally associated with BPM.

It may be possible to avoid completely the large expense of designing the process up front.  This would allow social business software to be used for planning and managing work directly, and still have the insight to improve the business processes over time.

See my next post about “Flipping the Process Life Cycle” to see how process mining might be incorporated into everyday management of knowledge workers.

Links

• Mining Activity Streams – from Oct 2010 link with social technology
• Process Mining Update -  from Oct 2010 summary of BPM 2010
• Webinar on Automated Process Discovery – Oct 2011

Discussion on what is required for ACM is often held by people with a lot of experience supporting work with BPM.  The discussion often falls into what I call the “process-analysis-trap”: analyze the work, find common sub-patterns, simplify, automate patterns, work to unify and consolidate automation.  This comes directly from the idea that there must be one right way to do something, and we just need to find it.  Then automate it because that would make it easier.  This all makes perfect sense depending upon what your definition of the “user” is.  In particular, it has nothing to do with knowledge workers.

It is helpful to think of a knowledge worker as an artist.  An Andy Goldsworthy who takes a bunch of sticks from a field and makes a wonderful pattern from them.  A Christo who unexpectedly wraps things up.  Painters like Dali, Picasso, Michelangelo, and Leonardo Da Vinci.  All different, all creative.  (Not all artists are famous like these, but in a public blog post I need to use famous names that we are all likely to have some knowledge of.)

I will use these the example of these artists to show that the process-analysis-trap leads us to entirely the wrong conclusion about how to support artists.  My goal is to show that the process-analysis-trap also leads us to the wrong conclusion on how to support knowledge workers.

Art-Kits 1

How would a bunch of process wonks define tools for artists?  We would define “art kits” that had paints,  paint brushes, paper, paint-by-number instructions, etc.  Only consider this: a real artist does not use a “paint-by-number art kit”.  Wouldn’t it be great if my paint-by-number could produce a work of art like Van Gogh’s Sunflowers.  I can arrogantly say that it never will.

That does not mean that art kits don’t exist, and there are certainly people who buy them.  There are a lot of people who have the job to put paint on canvas, and who are not artists.  For those people (whether hobby or profession) the paint-by-number approach is unquestionably easier to use. Ease of use is not necessarily the goal of the artist.

It is important not to get mixed up between the artist, and the person who paints for a living.  Similarly, we must not get mixed up between the knowledge worker, and the person who simply handles cases for a living — there are a lot of both out there.  If we attempt to draw conclusions without being sensitive to whether the person is really a knowledge worker, we will get invalid results. This is why in these discussions I obsess about “unpredictable” behavior.

Randomness

Even the most creative artist does not start each day with a completely random action.  Do artists follow a pattern, a methodology?  Of course they do, that is the essence of their learning to become an artist in a particular school of art.  The method is what they bring to the table.  The method is what they learn from others and from doing the job. They practice the method.  The exercise themselves.

It is the same with knowledge workers. Their approach to a problem is their approach which has been learned from history and from others.

Enforcing a particular method would unacceptably constrain what a knowledge worker does.  ACM should not enforce any particular method.  The knowledge worker pours their own method into the system. Through use, the ACM system is adapted to the work, but each knowledge worker can define their own method, or decide to use any method from anyone else.

Art-Kits 2

Here is an irony: when I say that a true artist will never use a paint by number kit, I don’t really mean to say this.  The kit, after all, has paint and brushes, and it is possible for a true artist to make a real piece of art from such a kit … even when basically following the numbers.  Does this contradict my earlier statement about the Sunflowers?  Yes it does.  That is the process-analysis-trap we fall into by assuming that there is one true process.  In a certain sense, a true artist can use an art-kit just as well as anything else.

It is the same with knowledge workers.  A knowledge worker can use pre-defined processes, working with them or around them as needed, using email and telephone if necessary.  Pointing to a particular use of a pre-defined processes by knowledge workers is like pointing to a paint-by-number kit used by an artist.  It is not the construct that matters, but what the people do with it that it key.  We can say, however, that an art kit that prevented the artist from painting outside the lines would be too restrictive for an artist to use.  The key for ACM is the ability to ignore the rules when necessary, to redefine the process as needed, and to alter all aspects of the predefined structures.

Artist Supplies

Must artists start with their bare hands?  No.  There are art supplies in standardized forms.  For example, you can buy canvas that is already stretched over a frame in standardized sizes.  This both constrains the artist, and yet is a common enough pattern that it is useful.   The important thing to note is that the canvas stretched over a frame did not come from any smart analysis of what an artist might need, but instead as a result of artists over many years stretching their own canvas: the pattern emerged.

Once pre-built canvasses become available, they cease to be part of the job that the artist does.

As knowledge workers/case managers work, they will develop patterns, and those patterns will be codified into applications (process or otherwise).  When those become codified, they cease to be part of the work of the knowledge work: the knowledge work is just the stuff that is not automated, not formalized.

What did we learn?

The knowledge worker/case manager needs an environment to get work done, and to express any method they want to bring.  There are a set of capabilities that support re-use, but it is all completely under the control of the case manager.

Later, as patterns emerge, an IT department may take on the job of formalizing a part of the work, and producing a BPM application.  When the BPM application becomes available the knowledge worker may start to use it, but it is no longer knowledge work.  In a very real sense, the work has been taken out of the ACM system, and put into a BPM system. It is this last conclusion that makes me feel that ACM and BPM really are completely non-overlapping sets of capabilities.

It is not a different in the technology, but a difference in how you use it.  If the technology is optimized for self-adaptation by a knowledge worker with typical skills for that domain, then it us useful for ACM style work.  If the benefits require a specialist in any kind of skill that is not common in the domain, then this removes it from consideration as ACM.  Artists can use a surprising variety of things to make art.  Similarly, knowledge workers work in a wide variety of environment.  Today, 99% of the knowledge work is being done with email and documents.  Let stop falling into the process-analysis-trap, and focus instead on how to help creative knowledge workers do things their own way.

Don Tapscott – Macrowikinomics: Rethinking the enterprise for the age of networked intelligence.

In 1991 he wrote a book “paradigm shift” where enterprise is shifting from hierarchical to network.  It seems like we are discussing the same thing today, while we are a lot more networked, we still have a ways to go.

Business processes are no longer structural, but instead molecular reconfigurable.  We live in the age of information liquidity.   The focus has changed: now relationships have become a form of capital.   One of the biggest focuses should be on how we create a positive customer experience.

Four key drivers: technology, economic, the net generation (digital natives are bathed in bits), social revolution (the rise of social networking, becoming social production).   Profoundly changing the deep structure. What does it tell us that the best selling business book was written by Scott Adams?

New operating system for the enterprise. We need to be constructing industrial strength social networks. We are seeing a new generation project tools — that enable people to work together in teams.  Mentioned collaborative decision management: what he calls support for deliberation.

In reference to the financial crisis of Sept 2008 he quoted Bob Dylan: “Because something is happening here; But you don’t know what it is.”   Who would imagine a few years ago that the big theme for business books would be “how to save capitalism“.

Our schools have the very best model of learning that the technology of 17th century can provide.

A trillion dollars of toxic assets are on the balance sheets of the big banks. Some call them the zombie banks. 21% youth unemployment rate.  80% of new jobs come from companies 5 years old or less.  One solution: put all the information about the toxic assets on line, open it up. Get a bunch of financial experts to review it, make a model, figure out the value, and get rid of them. Get them off the books, and then move ahead.

How about healthcare and research? It is designed around an industrial age model. I am a clinition, you are a patient, don’t ask anyone else for help.  Made a reference to “PatientsLikeMe” which is a social site for people with rare diseases to be able to help themselves.  There are a lot of these taking off like wildfire, having a big effect.

A large part of the drug industry will be hitting in the next couple years the pharma patent cliff. Need to move collaboration out to the internet and fix the industry. Place research and clinical trials into the public. The whole model of the industry is broken.

The music industry is collapsing. It inherited a bloated distribution model from a business model decades old based on shipping around things made of atoms.  The Internet eliminates the need for shipping atoms, but they refuse to adapt to this reality. The business model forces the industry to bets only on home-runs. They just need to change music from being a product to a service. Put all the music into a commons, and become a service to distribute and access this. Everyone in this room will pay $3/month for every song ever written streamed to their devices. The industry that brought you Elvis and the Beatles is now suing and hated by children.

Education and higher learning is broken.  Need to change the entire model of how we teach and learn.  Climate change is an issue as well that needs to be address.

Perhaps one model is the “open cities initiative“.  They opened up an internet site to list things that need to be done, and allowed people (politicians?) to say yes or no whether they would do that.  The citizens of Bogota produced 2K proposals for the mayor.

How can we best use collective activity for social change? Syria has hundreds of snipers killing people. Kids are using cell phones to triangulate where the snipers are and report them to others. London had their riots. The occupy movement spreading around the world. You can say what you want, but there is a deep deep sentiment that the system is broken. We need to make some very fundamental changes.  Leadership is everybody’s opportunity. It is you. You self selected to attend this event.

Can we learn from nature? Played a wonderful video of “murmuration” a behavior of starlings. It helps protects the birds and warms them up for the night. There is no one leader – they follow the crowd as a single big collaboration. Can we learn from this?  We are not in an information age, but an age of cranial intelligence.  Showed pictures of the Arab Spring, and stated that the kids in Egypt knew their collective power and used it.

Rachel Happe – Are You Getting Ahead, or Are You the Red Queen

The title comes from a Louis Carol quote where the red queen says that you have to run very fast just to stand still, and to get somewhere you need to run at least twice as fast.

People are the weakest link in our organizations.  In the past we have reorganized the company around, and focused on, transactions.  this made sense when comupters and equipment was very expensive and rare.  But now infrastructure (servers, networks, storage space) is cheap and plentiful.  People are now the scarce resource.  Need to optimize the performance of humans in a way we never have had to do before.  Labor used to be cheap (and still is some places) but here at home labor needs to be enhanced and supported.

The benefit of the early adopters of social are fading.  Problem is we can’t absorb all this technology.  Culture does not change quickly enough. So we need to be careful and thoughtful about what technology we select.

Culture is the competitive differentiator in the future.

She talked primarily about forming relationships with customers.   These soft things have hard returns for our businesses, but hard to measure.

• Customer loyalty is the savings of having to pay for a new customer acquisition.
• Forgiveness is very valuable.
• Relationships give you time .. something we feel we don’t have enough of.  A customer that trusts you will wait for the time to develop the right thing.
• Peer support in external communities, same with mentoring inside communities.
• Humans are good at recognizing others needs before those individual recognize themselves:  we can only specifically ask for a small part of our real needs.
• If you are friends with your customers they are telling you exactly what they thing … so you get better.

These lead to Revenue:

• If your customers are in your cultural sphere you will get lockout of competitors.
• Patience for release scheduled.
• You want all your customers to be advocates for you.
• Authentic insights are better through friendship.

Relationships increase in value along the following step:

• Encounter -  awareness
• Recognition – resonance
• Development – understanding of compatibility, interest, acknowledgement
• Friendship – contextual trust, loyalty, advocacy
• Partnership – forgiveness, advocacy, defense, trust, loyalty

Social media really helps with the first 2 or 3 three of these!  But it only gets you so far chit-chatting about content.    Community takes you the next step, and can get you to the friendship step.  If you want partnership, then get on a plane and spend a lot of time.  Direct engagement can get you through all the steps, but it is expensive and slow.

Do a gap analysis on the relationships that your organization has, and what do you want?

We get into trouble a lot because enterprises want results NOW.  Investment in a community can be quite long, before there is any return.  And, you need to carry through and continue to invest.  Some companies have outsourced their community effort. Seems strange given how critical it is for success.  Presented the “community maturity model”.  Mapping a community management approach to business management is going to be a big challenge.

My summary: She starts with a recognition that we are moving to a knowledge worker economy and i tis time to focus on support for people to do this.  One key support is for relationships, and she made an excellent case for the power of forming better relationships with customers.  To do this, you need to have a  longer term view, but the results are clear and significant.

Aaron Levy – Box.net

2011 was a good year. $630M spent on social business $21B on cloud buildup. Enterprise software just isn’t sexy.

• 91% of enterprises think the high cost of ownership is a problem
• 62% spend too much time on projects

most enterprises stuck on outdated systems.  They say “I can’t get to to my information; I can’t integrate my apps; I can’t share outside the org; I can’t see what is being done.”  All this really valuable data is stuck behind the firewall.

Before, IT had to manage all this. You had to buy storage, servers, software, applications, firewall, integrate it all, and maintain it all. All this had to be done by the IT group. Cost prohibitive in small to medium enterprise. The rainbow to value was not actually happening and things are beginning to fail. IT spends 80% of time, energy, resources on maintaining their current infrastructure.

The problem with most software is that it continues to propogate these problems. Silos, can’t share with each other. Mapped to technology and hierarrchy of organization. But enterprises are changing faster than the technology.  What do you do about the new workers joining?  The new workplace is all over the place. Need a new IT.

Information and people become more valuable at scale. As you have more teams, more data, this produces more value. Today it is the opposite. Smarter enterprise. Need technology that changes the way that we work. Faster decisions, more visibility, better speed. More social, so that anyone in the organization can see. Better intelligence on the information on the data and the people you are working with. Finally how is it all connected? (social, speed, connected, intelligence)

Last decade we had “on demand”. We can deliver over web/cloud. Now we can be smarter. What if we could start from the ground up?

Yesterday’s technology is far too stale. It does not “get” that people are the center of everything. (He showed a picture of SharePoint 2010 with a bunch of menus open, showing the complexity.)

Social must be pervasive in everything we do. Not just a single application. How do you lower the wall on what people are doing? How do you make sure that people are more connected? So we can ask questions: what content do I need? Like that project three years ago: how do I gain access to that instead of being locked up? Need visibility and exposure.

• what content do I need?
• what are people working on?
• what client shoudl I talk to?
• what is this project’s next step (instead of rigid workflow, how do you build workflow into the people network)

We are getting out of the dark ages. Way more visibility into what is going on.

How do we put compute in cloud computing. Let’s do interesting things with big data. Starting to see happen. How to find meaning all this data? How do you build systems that build better relevance.

Are we going from closed to open?

The cloud brings a new level of oppenness. Take content from all sorts of systems, and have them propogate. This is very different from enterprise software.

We need to balance security and simplicity. Locking down was the definition of security in the past. But we also need visibility. Need something more appropriate for users, not IT: balance.

• speed,
• social, bring down the walls, share with everyone
• intelligence – better relevance
• connected – apps, sharing

Comment from MC: Aaron should be an auctioneer.

Mike Gotta - Architecting the Building Blocks of Enterprise Social Networking

He started by reminding us that almost all human interaction occur in a network context. The IT Department looks too much at an application view of things. Thinking of a transactional view is a potential pitfall.

People connect through activities. They connect through the artifacts that are created through those interactions. We need broaden our understanding of what social networks are and how they effect the workplace. He says that he “sees networks everywhere.”

The role of technology is interaction, but the role of culture is participation. These can’t be separate, but instead need to be blended. There are many different contexts for networking.

Business Value vs. Organizational Value. There is a lot of talk about focus shifting to be more purpose oriented, more activity and goal oriented, because it is hard to calculate ROI. Had to shift the positioning so it is not seen as a rebake of Knowledge Management. Instead apply social to our productivity applications.

Converged a bunch of things on a SNS (Social Network Site). Six things:

• Traditional Collabortation,
• E2.0
• Semantic and Social Analyics
• Visualization
• ECM
• UC

in the past we focused on “a place to go”, somthing like a corporate Facebook.  Now we are getting beyond the “collaboration site”. More of a platform approach. Not about social tools, but how those capabilities are embedded.

Activity streams are getting a lot of attention.

Roles:

• EA Role – organizational architect
• IA Role – social insight
• Solution Role – community management
• Technology Role – social platform

Interested in both front stage, and back stage use.

Identity

Ascribed: Profiles – some see it as a directory, some as contact info. I see it as an identity. Employee number, department name. When we ask employees who they are, they are kind of shocked.

Claimed: Go beyond the identity that is given to them, and thinkg about interests, hobbies, tidbits. Scaffolding for people to connect. Claimed identity because we can’t prove it. This is difficult in a compliance world. Approved fields vs. freeform fields.

Performed: You perform you identity. If I see people working their identity, I am more likely to believe it. Platforms are being automatically attached now. (and want ability to say yes or no)

Reciprocated: others confirm and agree.

Mentioned Mark Smith, Social Roles, looking at people in email and discussion forums. In many cases people know that “John” is the person to go to, but only because they know him. Looking at new algorithms to be able to figure out that “John” is the person without having to explicitly specify in advance. We all perform roles that are not recognized by the organization. One idea is to track “My Questions and Answers”

Social Graph: we think of a very simplified model. Communities are a network. Teams are as well. Processes are role based networks. Management hierarchy is just a network based on reporting relationship.

Activity Streams: the “observable work”.  Then we need to filter. Vendors do not give you a lot of capability yet. Would like to taylor to the type of participation we would like to invite to help mediate latent ties.

• Ask a qeustion
• Follow a tag/topic – connect you to a community.
• Exception handling for a task

Latent Ties: ties that are technically possible but not yet activated. I should know “Dan” but we just have never met yet. Activating latent ties will build healthier and more robust networks.

Curation of activity streams. Can we find patterns? Could be really interesting from a process improvement point of view. Activity stream coupled to analytics coupled to social graph. Doing that activates the network.

Social Object: what is it? Activity stream entry, the data in it. Parse that. Connect so that others can interact and reappropriate it. How to share it, put it into a community. Finally analytics. Build over time.

“Cultivating Social Resources” a lot of the chit-chat is improtant for building social relationships. Emails saying “great team!” May dismiss as having no business relevance, but it is important for relationships. Hard to argue ROI of this. Like “pay it forward”.

Teams are not just the team. There might be 8 people on the (official) team, but if you do the network analysis, you find there are 15 others networked in that nobody knows about.  Social maps are a form of inquiry, exploration. Explored the social map, and found significant number of teams are interacting and nobody knew it.

Change Management is a program, not a project. Need standing investment. Talk to Booz Allen – at one point 80% of their activity was change management. Need to think about succession planning: how do the stories get passed on to the next people who take over.

Social Psychology of change. Rear-view analytics can yield stories on how they saved money. Nothing that ever showed up on the forecast of what was to be accomplished. Unpredictable results.

Never found a pattern that says bottom up always wins, nor that top down never wins. It depends upon the specific organization. Must consider the culture of the people.

In conclusion:

• Theory: when consideration of the organizational design constructs, you really should take a look at some of the theory. Don’t need a PhD, but become familiarity and understandign of some of the research, theory, and definitions.
• Methods: there are many ways to tell stories. Use case scenarios help to put a persona into the description of the system.
• Practices: have to create the feedback loop. Expect pushback. The complaints are the requirement. What happens when these systems are in place for 10 years. How will the timeline of the organization effect the design of the site.