Archive for January, 2011

In this second part of this discussion we will talk about how applying a structured decision process can help to temper, if not overcome some of the biases we pointed out in the previous post.

Let’s start with a brief description of a structured approach to decision making.  For this example we will consider the Analytic Hierarchy Process (AHP) developed by Dr. Thomas Saaty.  This approach has been made accessible by Decision Lens, Inc., who has developed a software solution that enables application of this process to organizations to collaboratively make complex multi-criteria, multi-stakeholder decision where these organizations are faced with many possible courses of action that can be taken.

The Analytic Hierarchy Process (AHP) is a theory of measurement that uses the principles of comparative judgment, first to make comparisons of decision criteria with respect to some property and then derives scales of priority measurement from these judgments.  In doing this the AHP can be used to derive measurements for anything and the meaning depends on the judgments. This allows creation of a measurement system with regard to the criteria to derive the scales and establish a prioritization framework.

The application of AHP follows a structured, while sometimes iterative, format.  Before we look at our example, let’s briefly review the essential aspects of the AHP process.

  1. Develop a hierarchy or tree of criteria- in clusters from high level categories at the top level to more specific sub-criteria, down to ratings and measures that will be used to differentiate the strategic value of options.
  2. Comparison of criteria to each other using a pairwise approach to establish their relative priorities for use in assessing options
  3. Rating the options against quantitative or qualitative scales that are derived to describe each criterion, and measure how well options reflect the priorities expressed within them.
  4. Optimize the allocation of resources among the options by way of cost/benefit analysis and performing sensitivity analysis to determine the robustness and drivers of decision outcomes.

So how can this impact many of the biases laid out in our previous discussion?  Let’s see.

We will look at an example of how some of the anchoring introduced by our fictional Committee Chair is impacted by item 1) and 2) above.  Similarly these techniques can help with many of the other issues highlighted in the way the group interacts in our example.

Committee Chair:  So we have a decision to make.  Which of these two product options are we going to pursue?  This is a critical strategic decision for us, and key to our ability to hit next year’s numbers.

Ideally the group of stakeholders to the decision would take a collaborative approach to defining a decision goal, and establishing the criteria that best enable achieving the goal. The weights of these criteria can then be derived before going down the narrow path being presented by the committee chair.  In this case, the group would be asked what the key factors that shape their decision are.  They may ultimately conclude that the criteria of “next year’s numbers” is the primary consideration for their decision, but without first defining the goal of the decision and articulating the issues that should shape the framework for making the decisions they can fall subject to this anchoring.  Among the important criteria may be the short term financial impact, but perhaps also the long term impact.  The question is which is more important and by how much?  By surfacing these counterbalancing criteria, and having the group weigh in on which should be the larger consideration in the decision, some anchoring from a single influential member can be buffered.  The group in our conversation goes on to talk about a variety of topics in a somewhat random, circular  and unstructured way; what the customers prefer, how easy it is to manufacture, and the overall technical feasibility of the product, etc.  Considering these other criteria can even further balance the thinking of the group and create a more holistic view of the decision.  Once a determination is made by the group of stakeholders to the decision as to which of these criteria is most important, the options can then be put to the test for how well they meet these criteria and the objectives imbedded within them.

If we look at the nature of the conversation, the key stakeholders are speaking of a decision structure that might look something like the following:

This all becomes very interesting in the process of making the comparative judgments.  It becomes virtually impossible for the group to get into single criteria based arguments when posed with a decision structure like this.  By virtue of the comparison exercise you can’t say something doesn’t matter, you can only say how much more or less it matters than something else.  There are often cases like the following where an outlier may need to defend their position in an attempt to influence a group, or have to concede that the rest of the group has a different point of view.  The question is, is this outlier Henry Fonda in “12 Angry Men” or Jim Carey in “Yes Man”.  In some cases information comes to light in these discussions that either substantiates or negates the position of the person who opposes the group, but one way or another, this kind of conversation moves the group to greater understanding and alignment by making all of these preferences and biases more explicit and transparent.  Making this quantifiable using AHP, may look something like this when engaging on the question, “with regard to this decision, which of the following pair is more important, and by how much?”

Provided the decision makers have the ability to stay independent in their assessments and not play “follow the leader” (which can be accomplished by anonymously sourcing these judgments), we can see that despite the very strong preference of the Committee Chair to have the financial focused considerations trump other factors that the opinions and preferences of the entire group give it only a slightly larger consideration on average than the Customer Preference for instance.  The arguments of others may run along the lines of “if we give the customer a preferred product, they’ll buy it, and we’ll meet our financial objectives.  Anyway, our financial forecasts tend to be less reliable than our customer research”.  Often the person with the extreme judgment will consider the positions of their colleagues and scale back the strength of their preference. In other cases, they may hold their ground, but have to accept that theirs is not the popular opinion.  In any case, voices are heard and fair process is at work.  This approach assures that at least participants have to hear perspectives that may interrupt biases.

Let’s look at the next level down in the decision.  When talking about financial impact, which is more important the short term or the long term?

Here again, we can see that the diversity of opinion in the group allows for some greater consideration of the short term, but the more extreme preference of a single stakeholder (which may be completely justified given their incentives, or political pressures) is again dampened by those who feel the long term should be given a more equal consideration, or even preferential consideration.  Collectively the group leans Short Term, but can create a framework that brings a considerably more balanced perspective to the decision.

Using a similar approach the decision makers, or other subject matter experts that they appoint, can be tasked with evaluating how well each option meets each of these criteria for selection.  The same principles of bringing multiple perspectives together and having a structured conversation about outliers can be used to temper the effect of biases and have a counterbalancing effect on more extreme positions while allowing for all of these perspectives to be considered.

This ultimately has the effect of creating a tangible view of the mental model that a group may be bringing to a decision.  These differences in perspective and strengths of preferences can remain invisible in discussion except for the way they are expressed in disagreement, power driven arguments, and advocacy based discussion.

The basic principles as outlined by James Surowiecki in “The Wisdom of Crowds” can be made real using such an approach.  The use of AHP as a decision framework allows for;

  • Independence:  Each member can independently provide their priorities
  • Diversity of Opinion: The AHP actually promotes the expression of diverse opinion
  • Local Knowledge – Stakeholders from different subject matter expertise are able to evaluate the information through their own lens of local knowledge.
  • Aggregation – The mathematical underpinnings of AHP create a unique aggregating mechanism where the decision makers derive the measurement system and establish priorities in a process of transforming implicit considerations into an explicit value model for their decision.

One possible approach to limiting the influence of biases may be to tackle them head on by bringing them to light with a degree of structure and process rather than allowing them to run wild in an unstructured, circular, power driven, advocacy based debate.


Read Full Post »

%d bloggers like this: