The Fallacy of Flexibility

This article is about software design, and makes the case that flexibility for flexibility sake should never be your goal.  There is a very delicate balance between design and implementation in order to provide both usability and capability when it comes to software.  Flexibility is often held up as a axiom, but flexibility should be provided only to the extend that it is actually needed by the end user.

The design principle “simple is better” or “less is more” is essential to good software simply because it is so easy for programmers to make arbitrarily elaborate designs.  The software realm has no physicality to it, no forces to limit overly elaborate design.  If we were to visualize most software programs as a physical building, they would be miles tall, some floors the size of a football field, but the entire thing balancing on a pin point.  It has sections of other buildings design for other purposes simply copied and grafted onto the outside. Elaborate complicated Rube-Goldberg-esque contraption continues to work because nothing ever “wears out” forcing the design to be simpler. However, the one thing dooms overly complex software designs is maintainability.  Software that is poorly designed becomes difficult to migrate to new platforms, and eventually becomes so slow to port, so slow to meet customers changing requirements, that the software eventually is dropped and replaced by something newer.

Keeping software simple in order to be maintainable requires following a couple rules:

  • Use a well known and documented programming language.  The more programmers that know the language, the less costly it will be to maintain.
  • Don’t invent any new languages.  Programmers love to make small scripting language that string together commands so that someone else can change the sequence of commands “without programming”.  What they miss is that this new script is itself a programming language — but a particularly poorly supported one.
  • Remove outdated functionality.  For example: if you program has special support for “floppy disks” remove it, because people do not use floppy disks any more.  Make sure there is a manual work around, but refuse to maintain the outdated functionality ”even if it still works.”
  • Remove commented out lines.  This is really easy: a commented line is a line of code that no longer executes, but is still present in the source files.  It is not actually part of the program, but it complicates the source making it harder for a programmer to read and maintain.  If it is not needed, it should be completely removed.
  • Design principles like encapsulation are assumed here: all the principles in the “Design Patterns” book, but these patterns must be applied judiciously in order to keep the design simple.  Rote implementation of such rules can lead to very elaborate and unnecessarily complicated approaches.

The biggest source of complexity is “Flexibility.”  From a software programmers point of view: more flexibility is always desirable.  Programmers routinely encounter situations where a particular capability was not flexible enough, and required a significant redesign.  This drives programmers to long for enough flexibility to “meet any possible future change in requirements.

Here is a physical analogy: Imagine that a car designed with a steering wheel that could be installed into the back seat of a car, as well as the normal location up front.  From the designer’s point of view, this is really great because there will be no difficulty in case someone wants to have a car with a steering wheel in the back seat.  Such a requirement might exist in the future, so from the designer’s perspective it seems like a prudent thing to go ahead an design the steering so that it might be installed in the back seat as well.  Furthermore this mechanism might be designed to install multiple steering units, for example two steering wheels in the front, or a steering wheel in the front as well as a steering wheel in the back.  Who has not experienced the situation while driving that you wanted someone else to take over for a moment driving?  Clearly, having multiple steering wheels would be better, right?

The fallacy in this is that nobody today actually buys cars with a steering wheel in the back, or with multiple steering wheels.  The fact is that a mechanism that could be installed in the back, or multiply installed, would involve more parts, and would be costlier to build.  This would increase the cost of the car, and make it uncompetitive.  It is fairly easy to ask someone how much they would pay for a second steering wheel, and the answer is going to be pretty close to zero (and possibly negative).  The physicality of the car eliminates that additional complexity in the design, but not so in software. After software is designed, the parts are free and have no weight or cost after that.  It is nearly impossible for anyone to see what is actually being used as parts for the software, so many many elaborate and needlessly complex things are part of a typical software product.

I have been in many design meetings where unrealistic hypothetical situations drove design decisions.  They will say things like:

  • What if someone wants a steering wheel in the back seat?  They should be able to. or
  • What if someone wants to position the steering wheel so that the driver is facing backward?  They should be able to.

These positions seem ridiculous because we all have a lot of familiarity with cars, but these hypothetical questions are hard to answer in the software realm.  These hypothetical questions might be:

  • What if someone want to make a web service request using email?  They should be able to. and
  • What if someone wants to make a web service request to a machine that does not support TCP/IP?  They should be able to

It is much harder to recognize these latter position as bogus.

My answer to this is: design only to the requirements that actual customers have asked for.  Never embellish with additional flexibility for eventualities you can’t be sure exist.

To put this plainly: design the best car you can with a single steering wheel.  Have a customer sit in the mock-up, and position the wheel in the best possible position.  Make the driving of the car (which is a solitary task) as effective as possible.  If a customer shows up that requests a steering wheel in the back, and shows you some money,  ONLY THEN should you extend the design to enable this.  Agile software methodology calls this YAGNI: you aren’t gonna need it.

In software, a design is very easy to extend, but very very difficult to trim unnecessary parts from.  If you are making a car, there is a long and costly process between the design and the factory that produces cars, so if you think you might need a “turbo-charger” on the car in the future, it is a good idea to think about it ahead of time, so that a “turbo charger” can be added without redesigning the car.  If there is a 10% chance that a car model will need a turbo charger, and it will cost $10M to redesign the production line later to accommodate the part, you can easily see that it would be worth investing to design the turbo charger up front, if that design costs less than $1M (which it most probably does).

Software is different: in many ways to can change (by extension) the design at any time, and there is no additional cost.  Another way to say this is: it cost the same amount to design something to meet the requirement after the requirement exist, as it is to meet the requirement before it exists.  If you make a design for a requirement that has a 10% chance of appearing, then you are effectively wasting 90% of that effort.  If would be far smarter to wait, and implement the design after the requirement appears, because 9 times out of ten it will not appear, and if that is the case you save effort.  On that 1 time out of 10 that it does appear, then there is no significant additional cost.  For example, a capability to import a file format might cost $50K to implement up front, and maybe $52K to implement later.  Since there are 9 chances out of ten that you don’t need it, there is no reason to implement it up front.

What I am trying to say here is that our normal intuition about the value of designing up front is wrong most of the time when it comes to software because it is not like a physical production process.

On many many programming projects I have seen programmers implement flexibility just in case someone wants to change something.  In 99% of the cases, that flexibility is NEVER USED.   It often can’t be used because that flexibility has never been tested.  That is the source of the waste: the programmer is designing for a flexibility which has not be requested by the customer, and which is not being tested, and so it actually not be used without significant additional work.

Instead of wasting time designing for capabilities that might appear good software is designed for the requirement that exist today.  100% of the time should be spend satisfying requirements which the customer has actually requested.  A good programming job should be done to meet this exact set of requirement.  It might be more flexible to design a steering wheel that can be mounted in any position, but there will be tradeoffs.  I would rather have a perfect steering wheel in one position, than a wheel which is not perfect, but could be used in more than one position.

So the fallacy of flexibility is that programmers will actually invent new requirements that do not exist for a current product and this is justified in the name of “flexibility” for the case that there is a new requirement in the future that we can not anticipate today.  Software is better served by designing it for exactly the requirements we have today.  Software is easy to change, and so if there is a new requirement tomorrow, we can easily change it at that time.

Advertisements
This entry was posted in Software and tagged , , . Bookmark the permalink.

7 Responses to The Fallacy of Flexibility

  1. Great post! You are absolutely right that flexibility is a key source of software complexity. It ismy experience however that most programmers do exactly the opposite. They never consider flexibility! They code it the way it is the most easy for them to code. Flexibility is usually required by the customer. Flexibility is needed so that there will not be just one user per software version. Flexibility is needed so that the user does not need to buy new software each time the requirements change. If you deliver standard software it has to be flexible as otherwise you have to compile a special version for each customer.

    But you are still right. 90% of the flexibility of a product are not used. But each customer will use another 10% of the flexibility. There is a great benefit in that because there will be just ONE code base to maintain and all fixes go into this one. Because the main problem as you point out is maintaining the code.

    But the world is still moving towards the flexibility you warn about. Why are we using XML, which I am not overly fond of? Why are we using SOA despite all its issues? Why are business writing RFPs with intense feature lists of stuff they will never use. Why do analysts rate products on these feature lists?

    So flexibility does not come from programmers but as product design requirements. Often enough I wish we could create less flexble programs, but they can’t be sold. Flexibility up to a point is perceived by businesses as ease-of-use, but once it gets as complicated to configure the flexibility as it is to program the functionlity configured, what have they won (i.e SAP impementations)?

    • kswenson says:

      Max,

      Good points. I was thinking mainly about extra flexibility added ahead of customer demand, which is waste: YAGNI.

      However, customers will regularly and naturally request capabilities that are not in the product. Adding capability at that time is not just appropriate, it is essential. As each customer asks for a behavior, you don’t drop all the old behaviors, but instead add the new behavior as an addition that can be chosen by the customer. This too could be called “flexibility” and it is part of the percentage that is really useful.

      The difference is that “good” flexibility is driven directly by customer requirements, while “bad” flexibility is invented by the programmer “just in case” it is needed. Carefully distinguish these!

  2. Excellent points, Keith. I especially like your point about not inventing any new languages, even if they are “simple” scripting languages — although the old programmer in me relishes having a scripting language to play with, that’s not the general experience for most users.

  3. Sandy, Keith: Oh, how I wish both you would be right and there would be this generically usable scripting language that everyone knows and fits any purpose. If that is not the case and you don’t want to compile code … what do you do?

    You create a scripting language that is simple enough and common enough. What other option is there?

    • kswenson says:

      In many cases I have found a desire among programmers inventing a script language when a compiled language would be a better choice. I have been on several projects where a developer thought that the best way to implement the automated test suite with a simple text file listing each test case name. The rationale given is that the text file is “easier” for the tester than making the call using Java or C++ syntax. This ease is temporary until the first time the tester needs to call a test cases in a loop, or needs to pass a value from one test case to another. Since both test case development, and main program development, both involve compiling, there is no reason not to simply call the test cases as a Java or C++ method. The syntax of a textual list may be less complicated than Java or C++ syntax, it is rarely fully documented: the original developer knows what is and is not allowed, and so the language is easy and obvious to them, but the tester has to find out the limits of behavior by trial and error. In the end it is far easier to teach the tester standard Java syntax (which is well documented and there are many books to help) than it is to try to teach a half-baked “easier” syntax.

      There are times, however, that you want to provide programmability without having to compile it. There is no single scripting language useful for all occasions, but there are instead dozens of scripting languages to choose from. Recently I have used JavaScript a lot: the syntax is familiar to lots of people, it has a powerful expression parser/evaluator, and it is relatively easy to extend with the new verbs that are needed. I have used JSP, PERL and TCL in the past. All of these are powerful general purpose languages that can provide the basic syntactic structure, which are then extended in normal ways with a set of methods/classes that provide the specific behavior needed. The syntax and behavior of the language is well documented, which is the main benefit over a special purpose language that has a simpler, but unfamiliar syntax.

      I have, however, also invented plenty of special purpose languages, and usually at the time you don’t think of it as a language. Instead you think it is just some data that will cause some behavior to happen. Later the needs grow, and often you find you forgot capabilities, such as a way to express quote characters in literal strings. My advice is to look at every data format that is used to sequence behavior as a language, and use a standard language like JavaScript, before making up a “simpler” language.

  4. Pingback: Process for the Enterprise » Blog Archive » Flexibility, Technical Debt, and Process Debt

  5. Pingback: BP-3: Flexibility, Technical Debt, and Process Debt « Fujitsu » In the News

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s