Debiasing your software architecture decisions

As software architects, we frequently make design decisions that affect the overall success of the product we’re building, from engineering quality to speed of delivery. We need to consider various factors such as long-term sustainability, skillset of their team, complexity and time constraints.

But, afflicted by the human condition, our decisions are biased.

We use our past experiences, familiarity with technologies, industry trends and other heuristics to come up with a plan of action. Because of these heuristics, the resulting (biased) decisions may lead to sub-optimal solutions, or token efforts at research before arriving upon our favourite (preconceived) outcome. In many cases we aren’t consciously aware of how we made the decision we did. If you asked us, we might struggle to explain our process. (I’m saying “we” here because I’m totally guilty of this!)

Earlier this week I stumbled upon a research paper entitled “Decision making and cognitive biases in designing software architectures” (by Akash Manjunath of Technische Universität München) which intersects two interests of mine: software architecture and cognitive bias. The paper proposes an interesting model to help architects make better decisions by increasing their awareness of the cognitive biases at play at each stage of their decision-making process.

The Observe-Orient-Decide-Act Loop

The author presents a hierarchy of biases based on the Observe-Orient-Decide-Act (OODA) Loop, which is a generic decision-making cycle applicable across several domains:

Within the OODA Loop, an actor makes observations about the surrounding environment, orients his/her thinking process by perceiving the important information based on the context, decides on a course of action, and finally acts on it.

Image Credit: Akash Manjunath

Applying biases to each decision phase

The thesis maps a set of cognitive biases (33 in total) to each OODA phase under a few sub-groupings into a 3-level hierarchy:

Image Credit: Akash Manjunath

Check out the supplementary web app where you can interactively browse the catalogue of all the above cognitive biases, read definitions and discover debiasing techniques for each.

Example: Hard-easy Effect

Here’s an extract from the web app of a bias that’s particularly rife in cloud engineering, the Hard-easy Effect:

Definition 1: Based on a specific level of task difficulty, the confidence in judgements is too conservative and not extreme enough.
Definition 2: The hard-easy effect is a cognitive bias that manifests itself as a tendency to overestimate the probability of one’s success at a task perceived as hard, and to underestimate the likelihood of one’s success at a task perceived as easy.

OODA Class: Decide Phase

OODA Subclass: Complexity

Classification Reasoning: The hard-easy effect comes into play when options in the list of alternatives comprise of varying complexities. The general feeling is that by choosing a more complex alternative, it will yield in a higher success rate as compared to choosing an easier alternative.

Example : Kubernetes for automated deployment and scaling of a website: With the rise of Docker around 2013, Kubernetes has gained traction in the recent years as the choice for automating deployments and scaling of container based applications. Kubernetes is a good choice when it comes to setting up complex websites which handle a large user base, and which requires a high degree of availability. However, setting up the Kubernetes is not a simple task for someone with limited experience. Additionally, when dealing with simple websites with a limited user base, a simple solution with Jenkins would probably suffice.

Impact: The decision to go with a solution of high complexity need not always result in the best solutions. When dealing with more complex solutions, factors such as the skill set of the team, time constraints, feasibility of implementing the solution must be kept in mind. Sometimes, a simpler setup might result in the best possible solution.

Debiasing Techniques: The advantages and disadvantages of a complex versus simple solution must be clearly established. Long term visions must be kept in mind to accommodate future requirements or varying complexities.

Related Biases: Complexity

If you’re interested in learning more about cognitive biases in general (lord knows the world needs more awareness of bias right now!), I highly recommend The Cognitive Bias Podcast by David Dylan Thomas. The podcast is finished now, but listen back from Episode 1 where David provides bite-sized monologues (~5–10 mins) on a different cognitive bias each episode.

— Paul