Taylor, Shelly E. and Jennifer Crocker. "Schematic Bases of Social Information Processing" from Social Cognition: The Ontario Symposium Hillsdale: Erlabaum, 1981

A short (30pp), clearly written explanation of how the way our brains process information affects what we remember, believe, think, experience and how it can cause us to make reasoning errors.

Ross, Lee "The Intuitive Psychologist and His Shortcomings: Distortions in the Attribution Process" in Advances in Experimental Social Psychology Ed. Leonard Berkowitz NY: Academic Press, Vol. 10, 1977

A fascinating, non-technical explanation of common reasoning errors and their psychological roots. Short and very informative.

Papert, Seymour. Mindstorms USA: Basic Books, 1993

This paper describes how systemic practitioners, by which I shall mean those trained in system dynamics and/or systems thinking, avoid the frequent reasoning pitfalls that affect the majority of people because of common flaws in their tautologies. I will discuss how systemic thinking provides advantages in two fundamental areas: in avoiding underestimation or misattribution of relationships when constructing an understanding of a situation, and in producing better action based on that understanding. Its strength in these areas gives it a great advantage over the thinking commonly employed by those who have not been trained systemically and is, I believe, one of the main reasons for the superior results obtained by interventions based on system dynamics and systems thinking.

In order to act, humans must take in information from their environment. However, the richness of the environment means that processing all available information is impossible ­ there is simply too much of it. Reducing the tremendous amount of input from the environment requires that two things happen to the stream of information provided us by the environment: that it be condensed and that it be structured to reveal linkages between different stimuli. Cognitive information processing schemas, or sets of propositions that serve to structure information, serve both of these functions. They first serve to condense experience from a string of unique stimuli into sets of similar experiences. This is essential, as we could not afford to approach every door, for example, as if it were completely a unique structure and spend the time necessary to test all possibilities for opening it. As Taylor and Crocker (1981) point out:

Given some stimulus configuration in the environment, it would be prohibitively time-consuming to match it against all prior experiences and given some problem to solve, the problem would never get solved quickly enough if one were to try out every possible solution. Even if the stimulus configuration were identified quickly and a strategy for solving a problem were selected immediately, it would still be prohibitively costly to process data piece by piece instead of in larger chunks. What schemas do is enable the perceiver to identify stimuli quickly, "chunk" an appropriate unit, fill in information missing from the stimulus configuration, and select a strategy for obtaining further information.

After processing the environmental information into manageable chunks and looking for similarities with past experiences, we must then construct an explanation for what we are perceiving. Here schemas serve as the tautologies which structure our experience by providing the links between the elements we experience. Taylor and Crocker continue:

When a stimulus configuration is encountered in the environment, it is matched against a schema, and the ordering and relations among the elements of the schema are imposed on the elements of the stimulus configuration. This process of ordering and structuring the elements of the stimulus configuration is important because it lays the groundwork for subsequent inferences.

While the condensing and structuring functions of schemas are essential, the importance of schemas in these areas means that errors in the schemas can have a potentially great impact on the quality of our perceptions, and thus our decisions. It is in addressing the weaknesses that commonly appear in schemas that systemic thinking provides its advantage.

In order to operate effectively, we must be able to affect at least a substantial portion of our environment. To do so we need to develop sets of propositions about what the main influences are on the things in our environment so that we can affect them. This is the process of explanation ­ we create (and modify, if necessary) explanations about our environment in order to be able to act in it. Because it is impossible to interact directly with the environment ­ we can only experience it through our perceptions ­ the process of explanation consists of mapping one's perception of reality onto a tautology, or set of relationships that are true by definition, that specifies the relations between the elements present in the environment. As Gregory Bateson (1979) notes: "[A]n explanation is a mapping of the pieces of a description [of reality] onto a tautology, and an explanation becomes acceptable to the degree that you are willing and able to accept the links of the tautology." In other words, our mental models of the relationships between things determine what we use for an explanation, and our satisfaction with the mental model we use determines our satisfaction with the explanation we build on it.

The primary focus of this paper will be the ways in which perceptions of a situation are improved by the disciplines of systemic thinking. Since we form perceptions before we can act, receive feedback on our actions, or learn, there is great benefit in understanding how thinking systemically prevents common perception errors. There are other advantages provided by systemic thinking, such as its benefits for encouraging learning and avoiding misperceptions of feedback. However, space limitations and the fact that many of these have been dealt with by other writers ­ for example, Professor Sterman's (1994) excellent discussion of how systemic thinking helps overcome weaknesses in learning due to misperceptions of feedback ­ lead me not to consider them here.

The first area in which systemic thinkers avoid well-documented reasoning errors is in avoiding underestimation or misattribution of relationships when constructing an understanding of a situation, an error that has come to be called the Fundamental Attribution Error. Numerous experiments have shown that the tendency to neglect systemic influences on behavior is pervasive: Ross (1977) reports on an experiment in which Bierbraur (1973) asked subjects to predict the rates of disobedience to being asked to administer shock to the learner in the classic Milgram (1963) experiment. Bierbraur's subjects consistently underestimated the effect of systemic influences in producing obedience to authority in the Milgram experiment and consistently overestimated the effect of unique personality factors in determining whether Milgram's subjects would comply with the experimenter's direction to administer shocks to the learner. Bierbraur's subjects exhibited this error even after witnessing a faithful reenactment of the experiment. Other experiments (e.g. Jones &;Harris, 1967) have found that the Fundamental Attribution Error is so strong that people underestimate systemic forces in explaining the behavior of someone they are observing even when they know that the person they are observing is acting under conditions in which they are given no choice (Ross 1977).

Especially in cases such as the Milgram experiment where the effect of systemic forces was so dramatic ­ small experimental changes to decrease the operation of the authority principle (e.g. having two experimenters disagree over whether to continue administering shocks) produced marked changes in subject behavior ­ how do observers continue to attribute the behavior to unique personal factors instead of systemic ones? I suggest that this is because instances such as the Milgram experiment are only the most obvious cases of a very common cognitive phenomenon I call "dormitization." This is the act of attributing the effect of a relationship to one of the constituents of the relationship. The name "dormitization" comes from a story in Molière's Le Malade Imaginaire, retold by Gregory Bateson (1979): "We see on stage a medieval oral doctoral examination. The examiners ask the candidate why opium puts people to sleep. The candidate triumphantly answers, 'Because, learned doctors, it contains a dormative
principle.' "

The candidate's explanation has the quality of attributing the result of a relationship ­ in this case, between the opium and the person ­ to one of the two elements of the relationship. The problem with responses like the one given by Molière's candidate is clear when we consider another case in which the same mistake was made: the case of early explanations for why things fall toward the earth. For many years prior to Newton, the attraction between bodies was explained as resulting from the objects "wanting" to be nearer to each other ­ objects "wanted" to fall toward the ground. This was a case again of attributing the product of a relationship ­ the attraction between to bodies ­ to one of the constituents of that relationship, a case of dormitization. In the same way, dormitization can explain why people fall victim to the Fundamental Attribution Error: they attribute the result of the relationship between an individual and his or her environment to the individual.

Behind dormitization and the Fundamental Attribution Error is the result of cognitive processes for sifting through information from the environment. One way in which we prioritize information is by paying more attention to the inputs that are most salient, that is, that stick out the most from the situation, and experiments have confirmed that highly salient information is given more importance and remembered better than less-salient information. Paying more attention to salient information is a useful cognitive filter, as in a great number of everyday situations, information that is among the most salient is also among the most important.

A problem arises, however, when relationships are important to determining the outcome of a situation, since relationships have no look, feel, smell, etc. that would make them more salient and therefore have more attention paid to them in explanation. Because of the salience they bring to relationships ­ creating a cognitive space for them, studying and mapping them, for example ­ system dynamics and systems thinking help those who use them avoid dormitization and the Fundamental Attribution Error. And system dynamics modeling has the additional advantage of helping reduce the tendency to overlook the informational value of things that do not happen because they are not salient by forcing the model builder to explicitly build a structure that produces non-occurrences, as well as occurrences, when called for.

The importance of this advantage for systemic thinking grows further in the second area of advantage I will address, which is when it becomes time to act. In order to filter out unimportant information, schemas tend to screen out information from the environment that is inconsistent with the tenets of the schema. This leads to the well-known self-reinforcing cycle where what people believe influences what they see, and the filtered version of reality they then see reinforces what they believe. The effect of this is that people caught in the dormitization or the Fundamental Attribution Error then see a self-confirming reality which is less affected by relationships and interactions.

This effect is compounded by the fact that schemas not only influence perception but also processing and recall. People are more likely to remember and to think about those things that they have experienced the most and those that fit with their schemas ­ both of which are schema influenced because schemas influence memory and therefore affect the perception of past experiences as well. Because people are more likely to take the actions that are foremost in their minds, this means that people whose perception has been affected by dormitization are more likely to think of and take actions that underestimate the effects of relationships and interactions. For example, companies or managers may fall into doing the same thing over and over, even when the circumstances are no longer appropriate, because it fits with what they have done in the past and with their schema-influenced ideas of how the system they're in works.

How we think affects what we see, how we interpret our experience, and how we act. System dynamics and systems thinking have shown their benefits in many areas, including shaping the way we think. Those who practice system dynamics and systems thinking not only gain powerful tools that they can apply, but they also gain a powerful ally which can shield them from common cognitive errors and the price they exact in thought and in action.