Archive for the ‘Wisdom of Crowds’ Category

Many people are familiar with the basic principles of the well known book The Wisdom of Crowds.  The principles have been outlined here in previous posts and the book is one I highly recommend from our book club list.  For anyone unfamiliar with this work, the basic argument of the book is that a group that is diverse in opinion, independent and thus not swayed by each others views, decentralized to allow the use of any specialization and local knowledge, that can have their individual opinions aggregated to an average solution, can make a better choice or estimate than any of the individuals within the group can.

An easy to understand example of this was demonstrated by finance professor Jack Treynor in his class, where he decided to put the classic contest of guessing the number of jelly beans in a jar to the test.  He showed that with a jar containing 850 jelly beans, that the average of the guesses of a group adhering to the conditions above estimated that there were 871 jelly beans in the jar.  Only one person in the group made a better individual guess than the estimate of the group.  It is ok that a particular individual outperforms the group.  What is interesting is that if you do the test ten times with a different number of jelly beans in the jar, the same person will not consistently outperform the group with the exception of the outlier genius.  So across multiple chances it is a very safe bet (the odds would be significantly in your favor) that the group outperforms any single individual in the group.  How do these individuals make their guesses? Have you thought about this?  Do they pick a low number because they are conservative and cautious not to overestimate?  Go for a big number because they are risky and overconfident? Reasonably confident in some rational calculation regarding the spatial relationship between the jar and the beans?  All of these kinds of guesses are likely, and to some degree governed by an emotional make up of the person making the guess – though I believe the emotional influences behind the strategy often go unnoticed.  We just think we’re making our best guess.  But what forms our best guess?

So, these individual orientations are interesting and the resulting group dynamics when they come in conflict are fascinating.  I recently had the pleasure to read the piece below written by Dr. Thomas Saaty, the architect and developer of the Analytic Hierarchy Process (AHP) for collaborative decision making.  His methodology, the AHP, has become the most published form of decision analysis in the some 30 plus years since being introduced and has been applied to everything from the choosing the best place to live, to life and death decisions in palliative care.  This recent writing from Dr. Saaty provides a very interesting insight on the nature of our judgments, and how our preferences are formed and contribute to those better answers from a group.

Language is a descriptive approach of the world of experience and does not bring out the intensity of how we feel about things. It lacks the precision needed to represent those intensities and synthesize them in an effort to understand the diversity of feelings we have in response to the happenings around us. An important consequence of this limitation, language alone cannot be used to combine individual judgments into a representative group judgment about what happens in the world. It becomes necessary to have experts who use their own individual judgments to tell us what they think is the real answer. What if they are not as knowledgeable as we think they should be?

At the risk of repetition, people have feelings with varying intensities and how strong these feelings are is very important to them. But language can deal with feelings with imprecise words that can only be roughly interpreted and rarely convey the implications of these feelings using deductive logic that relies on words to draw its conclusions. We need a way to capture the strengths of feelings of individuals and of groups of people and synthesize them accurately so we know better what the overall feeling is about different things they deal with which come together to shape their outlook. Once we learn to capture our feelings numerically in an accurate way, we no longer need to argue vehemently about how well we communicate our thinking and feeling. Sharing ideas is the most important approach to a valid outcome that corresponds to coping with reality to bring about what is in our best interest as individuals and groups seeking to fulfill our needs. Making comparisons is instinctive and fundamental in discovering what we think and how we feel about a complex issue. Not only humans are able to compare but living things like animals and plants also do.

Of course some people have more experience with certain matters and need to be accorded greater influence for their judgments than those who may not have the practice and understanding. Dealing with intricate and elaborate issues this way is a new approach to complex problems involving great misunderstanding and conflict. Such an approach requires that we identify and structure all the important factors bearing on a problem. To do that we need to work creatively together to brainstorm and organize the structure of the complexity we face. All proposed factors and objectives must be included. Their importance will emerge from the process of comparisons and prioritization. Implementing such an approach itself needs a structure and priorities to discover the best agreed upon way to bring about the desired changes.

It is interesting that our ability to make decisions is so influenced by our feelings, and yet we often make little or no allowance for their consideration and articulation within the framework of a decision.  In fact, we often argue against the expression of emotion within our decisions, striving for the unrealistic Spock like emotionless logic and objectivity.  Yet, this takes no account of how we really decide things.  What Dr. Saaty highlights in his writing is how important it is for us to make the provision for comparing key factors in decisions and letting the strength of our preferences (i.e., our feeling about the factors and choices) become part of the measurement system of what we value.  His method, the AHP, is a highly evolved approach for articulating the mental model of a decision and quantifying this strength of preference  within the decision framework through a process comparative judgment.  This all got me thinking about the nature of what we bring to the table when asked to participate in the process of making a choice or determining rank, or priorities.  It seems that without accounting for this unique nature of our preferences and how they are expressed within a decision, we can’t really make good choices, and may be in fact denying our very own experience a role in the process.  The complexity of the process that formulates our reaction to stimulus that ultimately dictates our choices is complex and difficult to grasp, as it is hard to observe ourselves and how our patterns of perception are shaped.  I was doodling a little model of how this might occur that came out looking something like this (click to enlarge)…

I started reading and researching a bit more about these ideas to try to find similar concepts, and was intrigued by the work of psychologist Jean Piaget on Cognitive Development.  He provides a description of how we move through stages to explore and interpret our environment and create meaning.  This is an interesting field of study and relates closely to how we make decisions and the potential nature of biases.  We’ll explore this more in the next post.

In the meantime, important points to consider when making decisions, especially in groups (preferably collaboratively versus adversarially)

  • The aggregate averaged guess of the group is usually better than the guess any one member in the group
  • Don’t deny feelings a role in the process, they are the inextricably linked and embedded in the essence of experience
  • Use a process to enable the measurement of the feelings to express how they interact with the descriptive language framing a decision
More next time on the nature of our preferences and cognitive development

Read Full Post »

In this second part of this discussion we will talk about how applying a structured decision process can help to temper, if not overcome some of the biases we pointed out in the previous post.

Let’s start with a brief description of a structured approach to decision making.  For this example we will consider the Analytic Hierarchy Process (AHP) developed by Dr. Thomas Saaty.  This approach has been made accessible by Decision Lens, Inc., who has developed a software solution that enables application of this process to organizations to collaboratively make complex multi-criteria, multi-stakeholder decision where these organizations are faced with many possible courses of action that can be taken.

The Analytic Hierarchy Process (AHP) is a theory of measurement that uses the principles of comparative judgment, first to make comparisons of decision criteria with respect to some property and then derives scales of priority measurement from these judgments.  In doing this the AHP can be used to derive measurements for anything and the meaning depends on the judgments. This allows creation of a measurement system with regard to the criteria to derive the scales and establish a prioritization framework.

The application of AHP follows a structured, while sometimes iterative, format.  Before we look at our example, let’s briefly review the essential aspects of the AHP process.

  1. Develop a hierarchy or tree of criteria- in clusters from high level categories at the top level to more specific sub-criteria, down to ratings and measures that will be used to differentiate the strategic value of options.
  2. Comparison of criteria to each other using a pairwise approach to establish their relative priorities for use in assessing options
  3. Rating the options against quantitative or qualitative scales that are derived to describe each criterion, and measure how well options reflect the priorities expressed within them.
  4. Optimize the allocation of resources among the options by way of cost/benefit analysis and performing sensitivity analysis to determine the robustness and drivers of decision outcomes.

So how can this impact many of the biases laid out in our previous discussion?  Let’s see.

We will look at an example of how some of the anchoring introduced by our fictional Committee Chair is impacted by item 1) and 2) above.  Similarly these techniques can help with many of the other issues highlighted in the way the group interacts in our example.

Committee Chair:  So we have a decision to make.  Which of these two product options are we going to pursue?  This is a critical strategic decision for us, and key to our ability to hit next year’s numbers.

Ideally the group of stakeholders to the decision would take a collaborative approach to defining a decision goal, and establishing the criteria that best enable achieving the goal. The weights of these criteria can then be derived before going down the narrow path being presented by the committee chair.  In this case, the group would be asked what the key factors that shape their decision are.  They may ultimately conclude that the criteria of “next year’s numbers” is the primary consideration for their decision, but without first defining the goal of the decision and articulating the issues that should shape the framework for making the decisions they can fall subject to this anchoring.  Among the important criteria may be the short term financial impact, but perhaps also the long term impact.  The question is which is more important and by how much?  By surfacing these counterbalancing criteria, and having the group weigh in on which should be the larger consideration in the decision, some anchoring from a single influential member can be buffered.  The group in our conversation goes on to talk about a variety of topics in a somewhat random, circular  and unstructured way; what the customers prefer, how easy it is to manufacture, and the overall technical feasibility of the product, etc.  Considering these other criteria can even further balance the thinking of the group and create a more holistic view of the decision.  Once a determination is made by the group of stakeholders to the decision as to which of these criteria is most important, the options can then be put to the test for how well they meet these criteria and the objectives imbedded within them.

If we look at the nature of the conversation, the key stakeholders are speaking of a decision structure that might look something like the following:

This all becomes very interesting in the process of making the comparative judgments.  It becomes virtually impossible for the group to get into single criteria based arguments when posed with a decision structure like this.  By virtue of the comparison exercise you can’t say something doesn’t matter, you can only say how much more or less it matters than something else.  There are often cases like the following where an outlier may need to defend their position in an attempt to influence a group, or have to concede that the rest of the group has a different point of view.  The question is, is this outlier Henry Fonda in “12 Angry Men” or Jim Carey in “Yes Man”.  In some cases information comes to light in these discussions that either substantiates or negates the position of the person who opposes the group, but one way or another, this kind of conversation moves the group to greater understanding and alignment by making all of these preferences and biases more explicit and transparent.  Making this quantifiable using AHP, may look something like this when engaging on the question, “with regard to this decision, which of the following pair is more important, and by how much?”

Provided the decision makers have the ability to stay independent in their assessments and not play “follow the leader” (which can be accomplished by anonymously sourcing these judgments), we can see that despite the very strong preference of the Committee Chair to have the financial focused considerations trump other factors that the opinions and preferences of the entire group give it only a slightly larger consideration on average than the Customer Preference for instance.  The arguments of others may run along the lines of “if we give the customer a preferred product, they’ll buy it, and we’ll meet our financial objectives.  Anyway, our financial forecasts tend to be less reliable than our customer research”.  Often the person with the extreme judgment will consider the positions of their colleagues and scale back the strength of their preference. In other cases, they may hold their ground, but have to accept that theirs is not the popular opinion.  In any case, voices are heard and fair process is at work.  This approach assures that at least participants have to hear perspectives that may interrupt biases.

Let’s look at the next level down in the decision.  When talking about financial impact, which is more important the short term or the long term?

Here again, we can see that the diversity of opinion in the group allows for some greater consideration of the short term, but the more extreme preference of a single stakeholder (which may be completely justified given their incentives, or political pressures) is again dampened by those who feel the long term should be given a more equal consideration, or even preferential consideration.  Collectively the group leans Short Term, but can create a framework that brings a considerably more balanced perspective to the decision.

Using a similar approach the decision makers, or other subject matter experts that they appoint, can be tasked with evaluating how well each option meets each of these criteria for selection.  The same principles of bringing multiple perspectives together and having a structured conversation about outliers can be used to temper the effect of biases and have a counterbalancing effect on more extreme positions while allowing for all of these perspectives to be considered.

This ultimately has the effect of creating a tangible view of the mental model that a group may be bringing to a decision.  These differences in perspective and strengths of preferences can remain invisible in discussion except for the way they are expressed in disagreement, power driven arguments, and advocacy based discussion.

The basic principles as outlined by James Surowiecki in “The Wisdom of Crowds” can be made real using such an approach.  The use of AHP as a decision framework allows for;

  • Independence:  Each member can independently provide their priorities
  • Diversity of Opinion: The AHP actually promotes the expression of diverse opinion
  • Local Knowledge – Stakeholders from different subject matter expertise are able to evaluate the information through their own lens of local knowledge.
  • Aggregation – The mathematical underpinnings of AHP create a unique aggregating mechanism where the decision makers derive the measurement system and establish priorities in a process of transforming implicit considerations into an explicit value model for their decision.

One possible approach to limiting the influence of biases may be to tackle them head on by bringing them to light with a degree of structure and process rather than allowing them to run wild in an unstructured, circular, power driven, advocacy based debate.

Read Full Post »

I  don’t typically like discussions on a topic that start with an etymology lesson or definitions, but as I started thinking about the collaborative aspect of collaborative decision making, I really found myself having to give some consideration to the meaning of the word because it wasn’t immediately apparent to me exactly what everyone’s talking about, other than the basic idea of working together, when the word gets thrown around.  So I’m making an exception despite my misgivings and exploring the definition.  Turns out the word is pretty interesting and nuanced.  When we talk about working together within an organization, collaboration includes the ideas of being jointly accredited or rewarded and recognized.  When it is between different organizations the nuance includes usually, though not necessarily, willing cooperation, and also the idea of partnering outside your ordinary circle of connections.  This first dictionary definition represents the positive connotation I believe is intended by the typical use of the word in most organizations and business environments today.

  • To work together with others to achieve a common goal. (emphasis added)
As in; Let’s collaborate on this article, and get it completed faster.

Note it doesn’t say how to work together.  Interestingly though, this second definition seems to describe something like the atmosphere around the experience of many people I talk to in my work at Decision Lens.

  • To cooperate treasonably as with an enemy occupation force in one’s country.
As in; If you collaborate with the enemy, you will be executed.

The word actually comes from the French, collaborateur (Very saboteur like), which comes from the Latin Collaboratus, past participle of Collaborare which means “work with”.  Look at this discussion on Wikipedia

    Since the Second World War the term “Collaboration” acquired a very negative meaning as referring to persons and groups which help a foreign occupier of their country—due to actual use by people in European countries who worked with and for the Nazi German occupiers. Linguistically, “collaboration” implies more or less equal partners who work together—which is obviously not the case when one party is an army of occupation and the other are people of the occupied country living under the power of this army.

    In order to make a distinction, the more specific term Collaborationism is often used for this phenomenon of collaboration with an occupying army. However, there is no water-tight distinction; “Collaboration” and “Collaborator”, as well as “Collaborationism” and “Collaborationist”, are often used in this pejorative sense—and even more so, the equivalent terms in French and other languages spoken in countries which experienced direct Nazi occupation.

In fact the French word collabo (collaborator) was often coupled with the adjective sale (dirty) and considered a supreme insult.

OK, it’s all starting to make sense to me now.  This feels a bit more like the nature of what’s going on in the political and social dynamics of many organizations with regard to decision making.  Given this context it appears we may actually be getting what we are asking for…

So this got me thinking about that “water tight distinction” mentioned above for this idealized and desired positive form of collaboration, which is what prompted me to come up with the new spelling –  “Collaberration” in the title.  Because in my experience it is rare, and simply not the norm.

We often hear talk, in lofty terms, about working together to achieve a common goal in ways that conjour images of a Koombaya singing Utopian organization where productivity effortlessly expands.  If we’re honest we would have to admit that when this happens it is an aberration. But it does happen on occasion.  Conversely in the worst cases, while often well intended at the outset, hidden motives, personal agendas and reluctant compromises that are kept from the light of day, can create passive aggressive behaviors that can result in coalitions of the unwilling feigning cooperation while actively resisting progress. There’s nothing like a dose of this dysfunction to significantly undermine decisions and stifle real progress against goals.  If you believe that words have power (which I do)  it’s interesting that we’re using a word (collaboration) that is loaded with baggage and inferences to the very kind of behavior I think many are hoping to weed out by evoking it.  Which makes me think perhaps collaboration isn’t always necessarily the objective.  It may be a desire for alignment, cooperation, or maybe even more so, outright submission.  I know I have been in some environments where unconditional surrender seemed to be what was being asked for under the guise of collaboration.

The Corporate Executive Board released a new research report that stated the following;

…even small efforts to improve collaboration through technology can improve bottom-line performance by 0.3 to 1.0 percent of total enterprise costs

No wonder we hear calls for Collaberration.  In “Organizing Genius – the Secrets to Creative Collaboration”, Warren Bennis talks about what he called “Great Groups”, and the dynamics by which they tend to operate.  Often times they are short lived, and members of these groups lose themselves in their work in pursuit of the goal.  These rare, but high impact, environments seems to be a collision of talent and purpose that blows away expectation and any imagined result.  Disney Animations, Xerox PARC, Apple’s MacIntosh project, Lockheed Martin’s Skunk Works, The Manhattan Project, etc.  are some of these cultures he explored.  Interestingly, they are hard environments to maintain, and sometimes they fall apart not long after a monumental achievements leaving their members longing for another chance to work in such an environment, and often saddled with a sort of postpartum  malaise.   Three of the most compelling insights he had about these creatively collaberrative environments were

  • Great Groups think they are on a mission from God
  • Great Groups are optimistic, not realistic
  • Great work is its own reward

So why is it so difficult for us to get people to work together and make decisions to advance their objectives?  I’ve noticed that my various standing alerts and feeds for Decision Making and Collaboration, tend to bring forward stories from Sports & Politics for the former, and the Arts and Academia for the latter, almost every single day.  These human endeavors seem to in some way help define the concepts in our minds, or at least have the most demonstrable examples where the language fits.  I’ve touched on decision making in sports in past posts, and so thought I would look to the arts for a bit more inspiration and insight into collaberration.  I found the video below, and thought it a wonderful example of a group of artists/athletes working together to do something no one of them could do alone.  Now, while we don’t know their back story, or the personal intrigue or their group dynamics there are some thoughts it provoked that I’d like to summarize in tandem with what we’ve read above.

Fascinating. Beautiful individual contributions and simply amazing teamwork.

So if it is collaberration we want, with all of the messiness that comes with it, here are 5 key considerations that can contribute to enabling the kind of creative, and productive environments where it can take place.  These ideas are interdependent, and self reinforcing of each other.

  • Vision & Purpose: In the case of our video, no one member of the troupe can achieve the vision. A single person can propose the concept of a group shadow puppet (perhaps to a reaction of some laughter initially), but teamwork is needed to realize it. There has to be a compelling and inspiring purpose to get people to reach beyond the limits of self, a purpose that stems from a vision that aligns their self interests and individual contributions to mutual benefit and achievement of the group objective.
  • Transparency & Tolerance: For our artists, there is nowhere to hide as they contribute to the silhouette on the screen, their contribution to the whole and the ability of that contribution to advance the objective has to be open and visible. There MUST be a desire and tolerance for Transparency in collaberration, no matter how difficult and uncomfortable. Transparency enables members to respectfully dissent, express and resolve differences in opinion and build alignment (i.e., agreeing to disagree and move forward in a unified direction) This allows investment in the purpose, to achieve commitment and allow members to operate above board.
  • Creative Conflict: We can be sure that there is argument amongst our artists and directors about the optimal configuration of appendages to create wings, wheels and triggers.  Conflict is inevitable, and necessary.  We should assume we will be challenged by the ideas and perspectives of others whatever the position we, or they, hold in the group. Disagreement and contradiction are a source for creative solutions.  Compelling Purpose and the tolerance for Transparency can help sustain the group through the conflict.
  • Incentives & Alignment: If our artists are given the incentive to stand out from the group, they will likely undermine the cohesiveness that creates the composite effect of their work.   Individual incentives can not undermine the group purpose, and need to be crafted in a way that aligns rewards and/or recognition with unique individual contributions that are clearly enablers to the group’s goals (i.e., those kinds of contributions the group would agree “we couldn’t have done it without them”).  Transparency and purpose increase, and conflict decreases, the better this is done.
  • Execution & Results: If our artists believe in the vision, are given the incentive to advance the group objective in a way that is enabling of productive conflict and promotes transparency, they can create a stunning collaberrative result.  The end goal is to “work together” to produce something extraordinary; an outcome or product.  The execution and coordination necessary to achieve that desired result are most likely a byproduct of the points above, whether deliberate or circumstantial, and execution would be difficult as a goal in and of itself.

Breaking through the barriers of competing interests, beliefs, perspectives and motivations is a challenge that is not easily solved.  Please join in the conversation and seize the opportunity to collaberrate to better understand how we might achieve more positive and breakthrough experiences in the process of decision making within groups.  It would be great to hear about the experiences of others in achieving collaberration.

Read Full Post »

This past week on The Hill’s Hillicon Valley technology blog, the founder of Fark.com Drew Curtis is quoted to have said

The ‘wisdom of the crowds’ is the most ridiculous statement I’ve heard in my life. Crowds are dumb

He goes on to say…

It takes people to move crowds in the right direction, crowds by themselves just stand around and mutter.
Everyone I’ve spoken to about this since reading this disagrees with him, so I guess he is right.  Well… that’s kind of what you would have to conclude if he is correct to some extent, isn’t it?
I think it’s interesting to look at the examples that were provided to justify this position, and without opening a political debate given their focus, it raises some interesting questions and a pretty compelling point regarding the nature of crowdsourcing.
In James Surowiecki’s hugely popular and compelling book “The Wisdom of Crowds” (see book club section of this blog), Surowiecki is very clear about, and qualifies, that not all crowds are wise.  They need certain characteristics to be present to make them such.  He outlines in significant detail in the book these characteristics, and they are:
  • The group must be Diverse in Opinion
  • The members of the group have to have Independence
  • They have to be Decentralized so that they can maintain their independence to exercise local knowledge
  • There has to be an Aggregating mechanism to converge the group’s opinion to a single point of view
I think there may be something missing from the formula which we’ll get to in a moment.  What struck me about these comments was that crowds engaged in many of the kinds of forums referred to in the post should possess the first three qualities almost by their nature as web forums.  The exception seems to come in the “Diverse in Opinion” aspect as like minded people tend to gravitate to certain online cultures and in some extreme cases the most frequent contributors to these message boards attempt to run off all but the most persistent non-conformists.  The homophily principle can play a part in limiting diversity in online communities.  Given the present state of technology, there does seem to be a gap in any true aggregating mechanism that can bring together a view of what a group may be saying.  Word Clouds may try to visualize it, and polls can provide some insight but can often be taken many times over.  I’m bothered by something more fundamental that may be missing in this discussion about these kinds of social network and their ability to be wise crowds, and it is…
  • A willingness and commitment, to try in earnest through the crowd interaction, to make a decision, solve a problem or achieve a goal through compromise for mutual benefit .

I think compromise may be an additional corollary to the four conditions of wise crowds when it comes to some aspects of social media.  Stubborn “my way or the highway” opposition needs to be reserved for very special and very rare cases, and can’t be the norm.  There needs to be some kind of appropriate motivation or incentive to participate in earnest, otherwise the divisive power driven arguments that are aimed at silencing people of differing opinions, will likely result in crowds that are quite irrational.  When the purpose of engaging in a discussion in a social media forum is to score points, intentionally obfuscate another’s position at any cost, be decidedly closed off to anything someone of another affiliation than your own offers, or just plain be mean, the crowd will be cynical and not engaged in trying to achieve a compromise, but rather squared off to win at any cost!   In these cases the inability to extract wisdom from a group might be attributed more to a victory of foolish individuals than a failure of wise crowds.  In most of the examples in “The Wisdom of Crowds”, participants either had a self interest in getting the right answer, or were working as part of a group against a well defined goal.  I’m not at all convinced this can’t be the result of emergent leadership within a group that builds this alignment and solicits input toward a goal.  Designing diversity into a conversation and being open and accepting of what it brings to the process of drawing out the wisdom in the crowd certainly fits the definition of leadership.  If however the suggestion is that there a few wise individuals,  smarter than the rest, that need to edit inputs and tell the “muttering” crowds what to do because they don’t know what’s good for them, then that’s another matter.  The semantics of the post are interesting and can be interpreted a few ways, my initial take was that the comments were a bit condescending.  Curious what others think.  Feel free to mutter, and I’ll try to limit editing.

Read Full Post »

%d bloggers like this: