On failed IT projects

From time to time struggling information technology (IT) projects reaches the news. I have during my career observed and been part of both successful and failed projects. Despite that its quite clear what is required to succeed, organisations repeats the same mistakes again and again and learning across sectors seams impossible. For more theoretical background please read the previous posts on digitalisation and change management.

Success or failure depends on three things:

1. Appropriate acquisition approach, buy when you can, build when the problem domain is wicked
2. Fit for purpose interaction and domain models is paramount
3. Choosing the right acquisition approach for the problem domain at hand

Buy vs build is not a black and white discussion as a build strategy will depends on many commercial components, they being open sourced licensed or commercial. It’s much more about who is the system integrator that matters. But as all of us who have played with Lego knows, small bricks are more flexible than the big ones. That applies for software systems too.

Of these the nature of the interaction model is the most important. To what degree does the digital information system serve the human users in fulfilling their objectives and task? Alternatively how much must the human users adapt their behaviour to suit the digital information systems needs.

My conclusion is that if IT projects was lead and organised according to their true nature a lot of painful problems could have been avoided. Would that made them cheaper? Not necessarily, but we would have had something that was useful when the money was spent.

Models

Digital information systems are models of the domain’s they serve, and as for all models they only captures the original development team’s understanding of that domain. Information systems come with two models.

• The internal domain model capturing the concepts of the domain. That being about patients, diagnosis and treatment’s, or wellbore’s, geological markers and drilling strings, or customers, stores and assortments to mention three examples.
• The interaction model defining how the individual users interacts with the information systems. Can data be entered from a smart phone app or is a full blown PC required. How many clicks and movements are needed to complete a task is part of this model.

In some systems the interaction model is just a view of the internal model, exposing the end user for all the complexity of the domain at hand, in others there are clear separation of the two. Public transport applications are examples of the latter. They provide you as a traveler with an easy to use interface for buying a ticket without exposing you for how to schedule the fleet of busses and trains.

The more diversity and wickedness of the domain, the more complex the information system domain models become. Enterprise systems covers multiple domains making the model integration a key concern. Using a patient record system as an example. Management of blood tests, x-ray images and treatments are all domains in their own that take different form dependent of function and role.

Understanding whether a model is fit for purpose or not is paramount.

Models are just models, what is a good model for one task might be useless for another. Take the London underground map as an example. Its useless for a streets of London walkabout.

Architecture

The architecture of digital information systems defines how the system is structured including the technology used. The software industry has operated with several architectural styles over the years, all of them coming with strengths and weaknesses. Which one is the “best” for a given problem requires a trade-off analysis. To learn more, please read this book.

Information systems that are built as integrated applications also known as monoliths works fine if the problem domain is monolithic. An integrated enterprise might be well served by this kind of architecture. In the case the problem domain is distributed and consists of actors that operate with large degree of independence a more loosely coupled or even distributed architecture might be better.

Choosing a fit for purpose architecture is crucial and is most likely the place where most mistakes are made.

One architectural pattern that looks promising is to separate data from applications. How and why this has been done with success can be studied in my post on the OSDU™ Data Platform. The pattern will be addressed in a future post as well.

Observations

Successes

When analysing successes the following elements stick out:

• Well defined problems, many of them related to scientific / technical computing
• Highly competent teams and most often not to large
• Freedom to use and even develop state of the art technology
• Clear expectations and hard deadlines
• Leaders who understand the name of the game and facilitate and pave way
• Agile and adaptive delivery processes
• Fit for purpose architecture
• Serves human users to succeed in fulfilling their objectives and tasks

When the problem domain is clear and ordered a lot gets easier. Sometimes hard deadlines can help as was the case for the work I did for the Norwegian Police up to the Olympics in 1994. The required software had to work on a particular day. And so it did even parts of it was in a “respirator”.

Failures

When analysing failures the following elements stick out:

• Ill-structured and wicked domains, many of the related to vaguely articulated business problems
• Many opinionated stakeholders that need to be pleased
• Technologically constrained, i.e., forced to use outdated technology or vendor provided solutions
• Unclear expectations and scope
• Incompetent leaders and teams
• Unfit for purpose architecture
• Technology driven
• Enforces a nonintuitive interaction model on the human users tapping them for energy

Attaching ill-structured, wicked and even complex problem domains as they are ordered is probably the worst thing to do. Stakeholder diversity is a contributor to the wickedness and I will argue the more political sensitive the domain is the worse it become.

Build or make

What type of acquisition process is most appropriate is a key question. What is the best approach depends on the problem domain and the level of commoditisation of the domain. If you need a new text editor pick one of the many that exists, you can even pick different ones dependent on what you are writing. As an example, this text is produced with the WordPress editor.

In the other end of the spectrum you find national information systems for tax and benefit calculations. These information systems are governed by national laws and might change on a yearly basis due to political priorities. This is a domain that require a build approach. Choosing a build approach does not mean that everything has to be built, but that the system owner is responsible for a as healthy and cost efficient system as possible within available technological frameworks.

The less commoditised, the more ill-structured and wicked the more a build approach fits the nature of the problem space. The risk rises exponential when enterprises chooses a buy approach on what is a complex and evolving problem domain.

Choosing the appropriate acquisition approach is the difference of success and failure.

Failing here and what you are left with is the projects obituary. Then said thing thou is that too many projects belief in a buy approach because the inherent complexities in the domain and the interplays are underestimated.