Posts Tagged ‘Analytic Hierarchy Process’

Many people are familiar with the basic principles of the well known book The Wisdom of Crowds.  The principles have been outlined here in previous posts and the book is one I highly recommend from our book club list.  For anyone unfamiliar with this work, the basic argument of the book is that a group that is diverse in opinion, independent and thus not swayed by each others views, decentralized to allow the use of any specialization and local knowledge, that can have their individual opinions aggregated to an average solution, can make a better choice or estimate than any of the individuals within the group can.

An easy to understand example of this was demonstrated by finance professor Jack Treynor in his class, where he decided to put the classic contest of guessing the number of jelly beans in a jar to the test.  He showed that with a jar containing 850 jelly beans, that the average of the guesses of a group adhering to the conditions above estimated that there were 871 jelly beans in the jar.  Only one person in the group made a better individual guess than the estimate of the group.  It is ok that a particular individual outperforms the group.  What is interesting is that if you do the test ten times with a different number of jelly beans in the jar, the same person will not consistently outperform the group with the exception of the outlier genius.  So across multiple chances it is a very safe bet (the odds would be significantly in your favor) that the group outperforms any single individual in the group.  How do these individuals make their guesses? Have you thought about this?  Do they pick a low number because they are conservative and cautious not to overestimate?  Go for a big number because they are risky and overconfident? Reasonably confident in some rational calculation regarding the spatial relationship between the jar and the beans?  All of these kinds of guesses are likely, and to some degree governed by an emotional make up of the person making the guess – though I believe the emotional influences behind the strategy often go unnoticed.  We just think we’re making our best guess.  But what forms our best guess?

So, these individual orientations are interesting and the resulting group dynamics when they come in conflict are fascinating.  I recently had the pleasure to read the piece below written by Dr. Thomas Saaty, the architect and developer of the Analytic Hierarchy Process (AHP) for collaborative decision making.  His methodology, the AHP, has become the most published form of decision analysis in the some 30 plus years since being introduced and has been applied to everything from the choosing the best place to live, to life and death decisions in palliative care.  This recent writing from Dr. Saaty provides a very interesting insight on the nature of our judgments, and how our preferences are formed and contribute to those better answers from a group.

Language is a descriptive approach of the world of experience and does not bring out the intensity of how we feel about things. It lacks the precision needed to represent those intensities and synthesize them in an effort to understand the diversity of feelings we have in response to the happenings around us. An important consequence of this limitation, language alone cannot be used to combine individual judgments into a representative group judgment about what happens in the world. It becomes necessary to have experts who use their own individual judgments to tell us what they think is the real answer. What if they are not as knowledgeable as we think they should be?

At the risk of repetition, people have feelings with varying intensities and how strong these feelings are is very important to them. But language can deal with feelings with imprecise words that can only be roughly interpreted and rarely convey the implications of these feelings using deductive logic that relies on words to draw its conclusions. We need a way to capture the strengths of feelings of individuals and of groups of people and synthesize them accurately so we know better what the overall feeling is about different things they deal with which come together to shape their outlook. Once we learn to capture our feelings numerically in an accurate way, we no longer need to argue vehemently about how well we communicate our thinking and feeling. Sharing ideas is the most important approach to a valid outcome that corresponds to coping with reality to bring about what is in our best interest as individuals and groups seeking to fulfill our needs. Making comparisons is instinctive and fundamental in discovering what we think and how we feel about a complex issue. Not only humans are able to compare but living things like animals and plants also do.

Of course some people have more experience with certain matters and need to be accorded greater influence for their judgments than those who may not have the practice and understanding. Dealing with intricate and elaborate issues this way is a new approach to complex problems involving great misunderstanding and conflict. Such an approach requires that we identify and structure all the important factors bearing on a problem. To do that we need to work creatively together to brainstorm and organize the structure of the complexity we face. All proposed factors and objectives must be included. Their importance will emerge from the process of comparisons and prioritization. Implementing such an approach itself needs a structure and priorities to discover the best agreed upon way to bring about the desired changes.

It is interesting that our ability to make decisions is so influenced by our feelings, and yet we often make little or no allowance for their consideration and articulation within the framework of a decision.  In fact, we often argue against the expression of emotion within our decisions, striving for the unrealistic Spock like emotionless logic and objectivity.  Yet, this takes no account of how we really decide things.  What Dr. Saaty highlights in his writing is how important it is for us to make the provision for comparing key factors in decisions and letting the strength of our preferences (i.e., our feeling about the factors and choices) become part of the measurement system of what we value.  His method, the AHP, is a highly evolved approach for articulating the mental model of a decision and quantifying this strength of preference  within the decision framework through a process comparative judgment.  This all got me thinking about the nature of what we bring to the table when asked to participate in the process of making a choice or determining rank, or priorities.  It seems that without accounting for this unique nature of our preferences and how they are expressed within a decision, we can’t really make good choices, and may be in fact denying our very own experience a role in the process.  The complexity of the process that formulates our reaction to stimulus that ultimately dictates our choices is complex and difficult to grasp, as it is hard to observe ourselves and how our patterns of perception are shaped.  I was doodling a little model of how this might occur that came out looking something like this (click to enlarge)…

I started reading and researching a bit more about these ideas to try to find similar concepts, and was intrigued by the work of psychologist Jean Piaget on Cognitive Development.  He provides a description of how we move through stages to explore and interpret our environment and create meaning.  This is an interesting field of study and relates closely to how we make decisions and the potential nature of biases.  We’ll explore this more in the next post.

In the meantime, important points to consider when making decisions, especially in groups (preferably collaboratively versus adversarially)

  • The aggregate averaged guess of the group is usually better than the guess any one member in the group
  • Don’t deny feelings a role in the process, they are the inextricably linked and embedded in the essence of experience
  • Use a process to enable the measurement of the feelings to express how they interact with the descriptive language framing a decision
More next time on the nature of our preferences and cognitive development

Read Full Post »

In working with many organizations on a wide range of complex decisions, I never fail to be amazed at the reaction to this question; “What is the goal of this decision?, what are we trying to decide?” Almost invariably, the question seems as though it triggers a line of thinking that has not been explored before. The question is usually met initially with a bit of silence, then a moment of anxious glances exchanged around the room, and lastly a free wheeling groping for the answer. In defense of those who seem to be the hot seat on the receiving end of this question, it is an easy although risky thing to overlook.

So how does something so central to decision making (what it is that is actually trying to be decided) get overlooked? Is it that so many casual decisions are made each day that we’ve come to believe it’s easy, and take this step for granted in our more complex deliberations? We shouldn’t assume that all decisions are created equal. Think about the tremendous levels of data collection and the amount of time spent by teams and individuals occupied with trying to inform these decisions.  Given all this activity it seems something must be well enough defined to drive the action, so there may be something more at play. I’ve begun to believe that perhaps the objective isn’t so much being overlooked, but that people often immediately switch in to a mode of mind reading and trying to manage the group agreement without giving the group and opportunity to disagree in order to resolve their conflicts productively and find their points of agreement.  As such, they are reluctant to say too much for fear of saying something that deviates from what may still be the unknown group norm or consensus. The idea is beautifully set out by Jerry B. Harvery in his article The Abilene Paradox: The Management of Agreement.

Here is the short version of the paradox, you can read the more detailed discussion at the link above.

On a hot afternoon visiting in Coleman, Texas, the family is comfortably playing dominoes on a porch, until the father-in-law suggests that they take a trip to Abilene [53 miles north] for dinner. The wife says, “Sounds like a great idea.” The husband, despite having reservations because the drive is long and hot, thinks that his preferences must be out-of-step with the group and says, “Sounds good to me. I just hope your mother wants to go.” The mother-in-law then says, “Of course I want to go. I haven’t been to Abilene in a long time.”

The drive is hot, dusty, and long. When they arrive at the cafeteria, the food is as bad as the drive. They arrive back home four hours later, exhausted.

One of them dishonestly says, “It was a great trip, wasn’t it?” The mother-in-law says that, actually, she would rather have stayed home, but went along since the other three were so enthusiastic. The husband says, “I wasn’t delighted to be doing what we were doing. I only went to satisfy the rest of you.” The wife says, “I just went along to keep you happy. I would have had to be crazy to want to go out in the heat like that.” The father-in-law then says that he only suggested it because he thought the others might be bored.

The group sits back, perplexed that they together decided to take a trip which none of them wanted. They each would have preferred to sit comfortably, but did not admit to it when they still had time to enjoy the afternoon.

What is so compelling to me about the Abilene paradox is that the entire chain of events can be undone by the simplest expression of transparency or personal preference by anyone involved (including the father in law making the suggestion to go to Abilene) that they are content where they are, doing what they are doing.

This of course all points to, and highlights, several well know challenges in group decision making related to pluralistic ignorance, group think, and pseudo-consensus. When psychologist Irving Janis coined the term Groupthink in 1972, he identified eight symptoms of groupthink. Of the eight, these two seem to be most at play in the Abilene paradox.

  • Self-censorship – Doubts and deviations from the perceived group consensus are not expressed.
  • Illusion of unanimity – The majority view and judgments are assumed to be unanimous.

These express themselves in a very distinct way in what is called pluralistic ignorance. The research on pluralistic ignorance is very interesting. When college students were asked about the high levels of alcohol consumption on campus it was found that most did not want to partake at such extreme levels but were all acting as if they did out of fear of rejection from other like minded people also feigning a desire to partake. But it seems no one can break the ice in these situations. There is actually a silent majority awaiting a leader to coalesce and provide an opportunity to express their true feelings and act in accordance with their preferences. Watch Dan Ariely present the topic below, it’s truly interesting and relevant.

So taking all this into consideration, what if we could break through?

Let’s imagine the conversation on that porch in Texas goes a little differently;

On a hot afternoon visiting in Coleman, Texas, the family is comfortably playing dominoes on a porch, until the father-in-law suggests that they take a trip to Abilene [53 miles north] for dinner. The wife says, “Sounds like a great idea.” “Are you kidding, it’s a miserable day to do that drive, and I’m pretty content here enjoying the company” The husband, despite havinghaving had reservations because the drive is long and hot, thinks that his preferences must be out-of-step learns his preferences are right in line with the group and says, “Sounds good to me. I just hope your mother wants to go.”“Yeah, you know, I don’t need to go anywhere, life doesn’t get much better than this right here.” The mother-in-law then says, “Of course I want to go. I haven’t been to Abilene in a long time.” “I hate Abilene, he suggests this silly idea every time someone comes by, always worried that no one is content just setting here having a good time.”  THE END

Easy peasy. Ah, the simplicity of the transparent and explicit expression of real preferences. All it takes is one courageous person to break the dam and start the rush toward real consensus. Here are three things to consider to interrupt these groupthink processes in a collaborative decision;

  1. Assign a facilitator: This is someone who has the role of challenging the group and asking the hard questions. It is best that the facilitator have a limited, or better no, stake in the decision outcome. They can read body language and draw out asymmetrical information and try to find those silent majorities if they exist.
  2. Bring in outside experts: These outsiders to the group can be brought in and asked for expert opinion and can be given the role of devil’s advocate by the facilitator. These experts can be positioned on the fringe of the group making them more inclined to disrupt the group norm and draw out differences. These can be carefully chosen rogue or maverick type experts who are empowered to express their ideas.  The facilitator needs to support these experts and capitalize on the productive conflict they may generate.
  3. Use a decision process: Use a methodology like the Analytic Hierarchy Process (AHP) to anonymously source preferences and aggregate the group point of view. Allowing each individual to state their preference independently and then discussing this as a group can provide a mechanism for finding the points of disagreement that can ultimately become the conflicts that are resolved to create true consensus and agreement.

Please offer your thoughts and insights about these issues, have you experienced any of these kinds of phenomena and how have you or your organization tried to resolve them?

Read Full Post »

In this second part of this discussion we will talk about how applying a structured decision process can help to temper, if not overcome some of the biases we pointed out in the previous post.

Let’s start with a brief description of a structured approach to decision making.  For this example we will consider the Analytic Hierarchy Process (AHP) developed by Dr. Thomas Saaty.  This approach has been made accessible by Decision Lens, Inc., who has developed a software solution that enables application of this process to organizations to collaboratively make complex multi-criteria, multi-stakeholder decision where these organizations are faced with many possible courses of action that can be taken.

The Analytic Hierarchy Process (AHP) is a theory of measurement that uses the principles of comparative judgment, first to make comparisons of decision criteria with respect to some property and then derives scales of priority measurement from these judgments.  In doing this the AHP can be used to derive measurements for anything and the meaning depends on the judgments. This allows creation of a measurement system with regard to the criteria to derive the scales and establish a prioritization framework.

The application of AHP follows a structured, while sometimes iterative, format.  Before we look at our example, let’s briefly review the essential aspects of the AHP process.

  1. Develop a hierarchy or tree of criteria- in clusters from high level categories at the top level to more specific sub-criteria, down to ratings and measures that will be used to differentiate the strategic value of options.
  2. Comparison of criteria to each other using a pairwise approach to establish their relative priorities for use in assessing options
  3. Rating the options against quantitative or qualitative scales that are derived to describe each criterion, and measure how well options reflect the priorities expressed within them.
  4. Optimize the allocation of resources among the options by way of cost/benefit analysis and performing sensitivity analysis to determine the robustness and drivers of decision outcomes.

So how can this impact many of the biases laid out in our previous discussion?  Let’s see.

We will look at an example of how some of the anchoring introduced by our fictional Committee Chair is impacted by item 1) and 2) above.  Similarly these techniques can help with many of the other issues highlighted in the way the group interacts in our example.

Committee Chair:  So we have a decision to make.  Which of these two product options are we going to pursue?  This is a critical strategic decision for us, and key to our ability to hit next year’s numbers.

Ideally the group of stakeholders to the decision would take a collaborative approach to defining a decision goal, and establishing the criteria that best enable achieving the goal. The weights of these criteria can then be derived before going down the narrow path being presented by the committee chair.  In this case, the group would be asked what the key factors that shape their decision are.  They may ultimately conclude that the criteria of “next year’s numbers” is the primary consideration for their decision, but without first defining the goal of the decision and articulating the issues that should shape the framework for making the decisions they can fall subject to this anchoring.  Among the important criteria may be the short term financial impact, but perhaps also the long term impact.  The question is which is more important and by how much?  By surfacing these counterbalancing criteria, and having the group weigh in on which should be the larger consideration in the decision, some anchoring from a single influential member can be buffered.  The group in our conversation goes on to talk about a variety of topics in a somewhat random, circular  and unstructured way; what the customers prefer, how easy it is to manufacture, and the overall technical feasibility of the product, etc.  Considering these other criteria can even further balance the thinking of the group and create a more holistic view of the decision.  Once a determination is made by the group of stakeholders to the decision as to which of these criteria is most important, the options can then be put to the test for how well they meet these criteria and the objectives imbedded within them.

If we look at the nature of the conversation, the key stakeholders are speaking of a decision structure that might look something like the following:

This all becomes very interesting in the process of making the comparative judgments.  It becomes virtually impossible for the group to get into single criteria based arguments when posed with a decision structure like this.  By virtue of the comparison exercise you can’t say something doesn’t matter, you can only say how much more or less it matters than something else.  There are often cases like the following where an outlier may need to defend their position in an attempt to influence a group, or have to concede that the rest of the group has a different point of view.  The question is, is this outlier Henry Fonda in “12 Angry Men” or Jim Carey in “Yes Man”.  In some cases information comes to light in these discussions that either substantiates or negates the position of the person who opposes the group, but one way or another, this kind of conversation moves the group to greater understanding and alignment by making all of these preferences and biases more explicit and transparent.  Making this quantifiable using AHP, may look something like this when engaging on the question, “with regard to this decision, which of the following pair is more important, and by how much?”

Provided the decision makers have the ability to stay independent in their assessments and not play “follow the leader” (which can be accomplished by anonymously sourcing these judgments), we can see that despite the very strong preference of the Committee Chair to have the financial focused considerations trump other factors that the opinions and preferences of the entire group give it only a slightly larger consideration on average than the Customer Preference for instance.  The arguments of others may run along the lines of “if we give the customer a preferred product, they’ll buy it, and we’ll meet our financial objectives.  Anyway, our financial forecasts tend to be less reliable than our customer research”.  Often the person with the extreme judgment will consider the positions of their colleagues and scale back the strength of their preference. In other cases, they may hold their ground, but have to accept that theirs is not the popular opinion.  In any case, voices are heard and fair process is at work.  This approach assures that at least participants have to hear perspectives that may interrupt biases.

Let’s look at the next level down in the decision.  When talking about financial impact, which is more important the short term or the long term?

Here again, we can see that the diversity of opinion in the group allows for some greater consideration of the short term, but the more extreme preference of a single stakeholder (which may be completely justified given their incentives, or political pressures) is again dampened by those who feel the long term should be given a more equal consideration, or even preferential consideration.  Collectively the group leans Short Term, but can create a framework that brings a considerably more balanced perspective to the decision.

Using a similar approach the decision makers, or other subject matter experts that they appoint, can be tasked with evaluating how well each option meets each of these criteria for selection.  The same principles of bringing multiple perspectives together and having a structured conversation about outliers can be used to temper the effect of biases and have a counterbalancing effect on more extreme positions while allowing for all of these perspectives to be considered.

This ultimately has the effect of creating a tangible view of the mental model that a group may be bringing to a decision.  These differences in perspective and strengths of preferences can remain invisible in discussion except for the way they are expressed in disagreement, power driven arguments, and advocacy based discussion.

The basic principles as outlined by James Surowiecki in “The Wisdom of Crowds” can be made real using such an approach.  The use of AHP as a decision framework allows for;

  • Independence:  Each member can independently provide their priorities
  • Diversity of Opinion: The AHP actually promotes the expression of diverse opinion
  • Local Knowledge – Stakeholders from different subject matter expertise are able to evaluate the information through their own lens of local knowledge.
  • Aggregation – The mathematical underpinnings of AHP create a unique aggregating mechanism where the decision makers derive the measurement system and establish priorities in a process of transforming implicit considerations into an explicit value model for their decision.

One possible approach to limiting the influence of biases may be to tackle them head on by bringing them to light with a degree of structure and process rather than allowing them to run wild in an unstructured, circular, power driven, advocacy based debate.

Read Full Post »

%d bloggers like this: