Simplifying Complexity: Life is Uncertain, Unfair and Unequal

PREFACE

Bruce J. West

The present book is an extensive revision of the previously published Complex Worlds, Uncertain, Unequal and Unfair and the title was changed to reflect that change. I believe an ebook publication will increase the likelihood of reaching an audience that is curious about what science can offer the first generation born into a mature information age.

A general polishing of the presentation has been made throughout the revision, but the most significant changes involve incorporating suggestions made by readers. One such change is the emphasis on using the powerful methodology of network science to guide the making of individual and corporate decisions in our complex society. There is also additional discussion on how a new way of thinking is required to fully utilize the results coming out of the new intellectual maps of the complex world of the 21st century.

Bruce J. West

Research Triangle Park

Durham

NC 27709

USA

Conflict of Interest

The author confirms that author has no conflict of interest to declare for this publication.

acknowledgements

The author wishes to thank the Army Research Office for supporting the research on which this essay is based and Sharon for her love, understanding and infinite patience.

PROLOGUE

Bruce J. West

One of our strongest urges as human beings is to know the future in order to control our destiny and the destiny of those in our charge. Therefore, proposing the notion that life is uncertain, unequal and unfair seems to undercut a basic human need. But my intention in writing this book is not to subvert that primal need, but just the opposite. My belief is that the more clearly a person understands how the world actually works, the more effective they can be in achieving what they want in life. My experience is that individuals are ineffective, in large part, because they fail to distinguish between how the world is and how they want it to be.

I am not a psychologist, sociologist or social worker. I am a physicist and as such I am more comfortable with mathematical equations and experimental data than I am with words. So this is not a self-help book. It is a somewhat personalized account of what the average college graduate in western society has been systematically taught about the scientific understanding of the world that is not true. The greatest myth that has been delivered with all the pomp and circumstance of scientific truth is that for most purposes a linear view of the world is more than adequate. This is where the book begins, tracking down some of the less obvious implications of a linear world view and how that view conflicts with available data.

I heard somewhere that an editor told a would-be author of popular science books that with each equation she would lose half her intended audience. I do not know if this is true, but to be on the safe side I have not included any equations in the book, but there are charts and graphs along with ample interpretive discussion to replace them. This is a book about science and the role science plays in our technological society, both on stage and behind the scenes.

A brief review of the historical evidence that the mental maps of the world we construct for ourselves consist primarily of elements that are linearly connected provides a context to understanding why some people refuse to act in their own self-interest. Even when uncertainty is introduced into the description of events, as a way of including the influence of the broader world into their development, that uncertainty takes the form of small, additive, random fluctuations. The world’s ambiguity is represented by a bell-shaped distribution of fluctuations in the outcomes of experiments and the variability of observations. The bell-shaped distribution reveals certain general properties of the world’s influence on simple predictions, whether it is your broker’s estimate of the likelihood of a market crash, your doctor’s estimate of the severity of your disease, or whether you should have received a raise rather than the new guy.

I examine how the neatly constructed linear world view has been challenged by the complexity of modern society. It is not the case that humans have changed how they construct their mental maps of the world. It is that the linear assumptions made in the past are no longer as useful, as they once were, in guiding decisions made, particularly when social interactions are long range, multiple, and anonymous. I will indicate how the disintegration of simplicity disrupts our lives and leads to such things as the mismanagement of the health care system, particularly through the dominance of extreme events, when the assumption of Normal statistics no longer suppresses outliers. Specifically I am concerned with the form in which the notions of fairness and equity, born in the social unrest and industrialization of the nineteenth and early twentieth centuries, survive in the data of the twenty-first century; or more accurately how they do not survive or have been transformed.

Much of the fear that is generated by those that misapply the notion of complexity is based on extrapolating recent fluctuations into the future. Such extrapolation is invariably done using linear models that almost never have anything to do with the phenomenon being extrapolated. This was done using the science of eugenics at the turn of the twentieth century, and was the scientific basis for the Aryan race so loved by Hitler and still considered fondly by white supremacists and skin heads today. A similar kind of scientific basis was made in the 60s and 70s for overpopulation and global winter. These things are mentioned in passing, but what is important is that we must abandon the idea that complex phenomena lend themselves to simple linear predictions.

Such a strong conclusion requires an abundance of evidence regarding the value of replacing linearity with complexity. The implications of a complex representation of the world are immediate and profound. One inherent advantage is that the complex vantage point provides a single coherent view of disruptive mechanisms in complex phenomena; mechanisms ranging in physical science from earthquakes to floods; in social science from stock market crashes to the failure of power grids; in medical science from heart attacks to flash crashes in health care; and in biological science from the extinction of species to allometry relations. Extrema are more frequent in the complex world than they are in the linear world. The effects of extreme events are certainly unfair, and fortunately they do not occur every day. But when disruptive events do occur they introduce crossroads, and the selection of which road to take determines the subsequent course of events in a person’s life. Consequently, understanding the source of extremes enables an individual to take back control from the hands of fate.

The transition of our mental models from a simple to a complex world view, entails the breakdown of bell shaped statistics and necessitates the adoption of inverse power-law distributions. This is nowhere more evident than in the distribution of wealth. The long tail in the inverse power law implies that there is a fundamental imbalance in how wealth is distributed and this imbalance was identified by Pareto, the engineer that first identified the effect over a century ago. We shall explore whether or not such imbalance is necessary in a stable social society. This is done by studying other, less emotionally charged, phenomena that share many of its properties. To compare physical, social and biological systems it is necessary to have a common language and for this the idea of an information-dominated system is introduced and developed. The appropriate quantities to measure in complex dynamical systems are not easy to identify, in fact, what we choose to measure may well be determined by how we define information and how that information changes in time. How information flows in complex networks, or how information moves back and forth between two or more complex networks, is of fundamental importance in understanding how such networks or networks-of-networks operate. This information variability is shown to be determined by inverse power-law distributions, which in turn are generated by a number of generic mechanisms that couple contributing scales together. We identify different mechanisms that produce empirically observed variability; each one prescribing how the scales in the underlying process are interrelated.

Science is about finding order in the panorama of the world and embracing a perspective that includes the falling of apples and the motion of planets; the behavior of the individual and the actions of groups, large and small; the information content of an encyclopedia and the wikipedia; in short, science does not, and should not, have any boundaries with regard to content. The terrestrial and the cosmic are part of the give and take in science, with the goal of uncovering the principles and laws that determine how the universe functions, along with the individuals within it. For most people, science appears to be separate and apart from the world in which they live. The principles and laws of science do not seem to apply to the general interactions among people; due, in part, to the fact that principles have not been found for everyday decision making; laws have been notoriously absent from mundane thinking; rules have been sought in vain in the growth of society; and indeed canons go begging in the multiple complex phenomena within the human sciences, despite over two hundred years of effort to either invent or find them. A possible exception to this pessimistic summary of history is given by the Principle of Complexity Management, whereby a system with greater information, but perhaps lesser energy, can dominate a system with lesser information, but greater energy. The principle is a recently proven generalization of an observation made by the mathematician Norbert Wiener, and may be one of these long sought universal principles.

The final chapter contains my understanding of the formal justification for complexity in the real world. In turn, it is an examination of what complexity implies, about the difference between how we react to what we have, as opposed to reacting to what we want, but do not have. People always respond to events according to their mental maps of the world. Consequently, when they find the response to be inappropriate, the most reasonable thing to do is change the map. However, people are not always reasonable or logical. My hope is that the potential for understanding presented in this book can initiate the wisdom that St. Francis addresses in his brief prayer:

God, grant me the serenity

To accept the things I cannot change,

Courage to change the things that I can,

And the wisdom to know the difference

At the suggestion of an anonymous reviewer to present additional discussion on the interpretive strength of nonlinear models I have elected to include an epilogue.

How Scientists Think

Bruce J. West

Abstract

We begin by focusing on the ways we record the myriad of events that make up our lives, using simple models that are intended to capture the dominant features of those events and to provide coherent interlinking of events. If the world did not change in time, more and more detail could be added to these models, with each repetition of an event. Eventually we would have an accurate reconstruction of a successful economic relationship, of a nurturing family, or of a supportive organization. But things do change, even if our reactions to them do not. To understand these changes scientists have developed techniques that quantify and communicate objective models of these subjective events. Without presenting the technical details of how scientists construct such models, I use a combination of personal history and discussions of the science hidden by a variety of social problems, to lay the foundation for the understanding and resolution of these problems in subsequent chapters.

Keywords: Chaos, Exponential growth, Grand visions, Mental models, Multiple saturations, Saturation, Technology evolution.

Science, as well as the typical scientist, has changed along with society. From its slow paced agrarian roots, to the faster paced industrial form, to the nearly instantaneous informational society, the concerns of science and scientists have steadily expanded. The basic science describing the mechanical motion of the planets orbiting the sun, matched the relatively simple social forms that were directly supported by farms and farmers. The increased complication of the statistical description of the interaction of large numbers of particles in a gas was more compatible with industrial mores and the networks necessary to support them. Finally, tipping points and global interdependence spawned the analysis of complex phenomena in harmony with the information society. This historical tagging of the concerns of science indicates that we tend to think of these distinct

social modes as being separated by large intervals of time, say centuries. Although historically accurate, such a picture distorts the influence that these distinct social modes have on individual scientists. So it is not without value to include some personal history in my presentations, since all three social modes have influenced my own development as a scientist.

So I begin with my father, who was brought up on a five hundred acre farm in upstate New York; the oldest of thirteen brothers and sisters. The farm was without benefit of electricity or indoor plumbing, except for a hand pump providing water with which to wash and cook. He graduated eighth grade when he was 12, but there was no high school in his rural community, so he had a choice to either leave school and work the fields with his step father, or stay in school and read every book in the library including two encyclopedias. He choose the latter. Like many young men his vision of the future did not coincide with that of his parents. The world beckoned to him and he left the farm when he was 16; it was the depth of the depression.

My mother was raised on an even smaller farm in upstate New York; the oldest of seven brothers and sisters. Her town did have the luxury of electricity, as well as, a high school from which she graduated. The daughter of Italian immigrants, she was the first in her family to receive a high school diploma. I have one book she kept from that time, The Logic of Epistemology, not the kind of high school reading seen today.

My parents were married when my mother was 21 and my father was a few years older. They gave birth to seven boys, three before the Second World War and four after my father came home after serving in the Army on an island in the Pacific. The ages of my brothers were spread over seventeen years; the youngest was in a crib in my room when I left home at 17. I shared my room with four younger brothers. Like my father I was restless and did not share my parents view of the future. I was the third oldest of seven sons, born into a labor class family, and this circumstance contributed significantly to my decision to be a scientist.

My first memory of wishing to be a scientist is associated with a eulogy I wrote on Albert Einstein for an eighth grade English assignment. Thinking about it now I can see how the idea must have been swirling around in my head for some time, but it took the death of this great man to focus the desire. It was 1955 and once a month there were school drills in which students were guided to duck under their desks in response to an imagined, but no less real, bright flash of light in the sky. We were periodically shown films of cities being destroyed by atom bombs and every Catholic mass ended with the phrase Savior of the world, save Russia. At that age the ‘how’ of things seemed much more important than the ‘why’. It is only after years of study that I began to understand the reasons underlying the ‘why’ and to appreciate their entanglement with the ‘how’.

Modern science, or more precisely physics, began with Sir Isaac Newton (1642-1727), who famously wrote in response to critics who wanted him to ‘explain’ the causes of gravity, that he constructed no hypotheses. Newton believed that what could not be directly inferred from experiment constituted hypothesis and he was having none of it. A hypothesis is a refined version of the vague impressions, half-backed ideas, ill-conceived assumptions and intuition that are often generated during the scientific investigation and solution of complex problems. The hypothesis summaries what is learned in the feverish attempt to understand a mystery, but only after the fever has subsided. Scientists typically formulate a hypothesis near the end of a study to make clear to others exactly what it was they were attempting to prove, but only after they are pretty sure they know the answer.

Only extremely simple problems have solutions that can be put into the form of a hypothesis before any research has been done. So when I refer to how scientists think it is not about the formation and testing of hypotheses, but it is about how we acquire knowledge from experiment. What a scientist works to avoid in this acquisition of knowledge is confirmation bias. Such bias was identified by the mathematician/philosopher B. Russel:

If a man is offered a fact which goes against his instincts, he will scrutinize it closely, and unless the evidence is overwhelming he will refuse to believe it. If, on the other hand, he is offered something which affords a reason for acting in accordance to his instincts, he will accept it even on the slightest evidence.

Of course everyone is guilty of confirmation bias. I am, you are and everyone I know more readily accepts arguments that supports what they already believe to be true, than arguments that do not. As a private individual such bias determines the books I read, the movies I watch and the friends I have. However as a scientist I must proceed differently and actively seek out those things with which I disagree. Why? Because if I disagree with the science then one of us is wrong and as a scientist my search is for consistency and ultimately what can be verified by experiment. Part of what scientists do is read papers that draw conclusions with which they disagree. In my own case I then try to understand the flaw in the paper’s argument, and failing this, I try to identify the mistakes in my own reasoning that led me down the garden path. Of course such detailed analysis often shows that we were both partly right and partly wrong and the clarifications lead to deeper understanding.

The reader is probably not interested in how scientists generally carry out their research activities, so I do not discuss those activities here. The presentation is concerned with explaining the scientific foundation of three concepts: uncertainty, inequality and unfairness. I will side step the temptation of defining these concepts here and assert that all three emerge from the complexity of phenomenon in the physical, social and life sciences. Since I am neither a physician nor a social scientist the reader is certainly justified in asking how I have come to some of my conclusions, particularly those that differ from the opinion of a large segment of society. This is the reason I begin with a discussion of how scientists in general think when they are attempting to understand complicated problems, or more specifically how I think about complexity. Since this is not a scientific publication I freely express my opinions along side what I can prove, with the self-imposed constraint that I explain how I formed the opinion expressed.

I am a physicist so I tend to think about solving problems in a particular way and this gives me the advantage of method. I apply the same method I use in my work to such questions as the existence or non-existence of certainty, equality and fairness in society. I start Chapter Two from the premise that equality and fairness are a consequence of simplicity and the entire chapter is used to explore the consequences of that hypothesis and explore the evidence in support of it.

A simple process can be predictable or not. For example, flipping a coin is simple but not predictable, whereas tossing a horse shoe is both simple and to a large extent predictable. There are two ways a process can be unpredictable. In the case of a coin toss the process is random and therefore is by definition unpredictable.

The second way to lose predictability is by increasing the complexity of a process, which we take up in Chapter Three. Therefore a phenomenon need not be complex to be unpredictable; it only needs to be random. It is worth mentioning here that a simple random processes is one described by Normal statistics and is described in great detail without mathematics in Chapter Two. In that chapter the properties of Normal statistics are shown to be the basis of many of our modern ideas including certainty, equality and fairness.

On the other hand, complexity is measured by the deviation of the statistics from the familiar bell shape. The bizarre properties of these statistics are discussed in Chapter Three. These new statistics are used to describe phenomena dominated by events out in the extremes; stock market crashes, earthquakes, floods and other extrema that determine the drama in our lives. These statistics are descriptively called heavy-tailed to capture their emphasis on extreme values. Random fluctuations described by these inverse power-law statistics represent processes that are intermittent in time, like the splashing drops from a leaky faucet, or processes that cluster in space, such as the formation of spontaneous traffic jams. The entire content of this book is to provide a rationale for those events that can have a significant impact on daily life and how we think about them, while being guided by the difference between bell-shaped and heavy-tailed statistics.

My oldest son has always thought the world ought to be fair. When he was eight or so he noticed during a class that his teacher kept glancing back and forth between arithmetic problems she was solving on the blackboard and a sheet of paper on her desk. Straining forward from his front row seat he was able to determine that the paper had all the problems worked out in advance. His response was to stand up in front of the class and reprimand her by saying: Miss __ you are cheating. The resulting student-teacher interchange was the topic of an impromptu parent-teacher conference later that day.

This incident occurs to me now because it was such a clear and personal experience of how preconceptions determine the ways we interact with one another. My son’s concern for fairness overruled his sense of good manners and judgement about classroom behavior. But then he was only eight and to be fair he was also right. Of course thinking about an incident of this kind and deciding to put it in a book are very different things. My reason for writing about it is to emphasize that his view of the world and fairness, which he still holds some thirty-five years later, are very different from my own. The question addressed in this book is not whether the world is fair, the evidence clearly shows that it is not, but whether it ought to be fair. The answer to the latter question is not so obvious.

There are many ways to address questions of what ought to be true about the world. There is the philosophical, where one can draw from the great thinkers of the past and construct impressive arguments complete with footnotes; there is the strictly theological, where one accepts a few uncontested truths to start and from them draw a series of logical conclusions; and finally there is the historical in which one can trace what has occurred in the past and argue that this is what will occur in the future. These approaches and many others have a common failing in that explaining complex phenomena, such as how wars begin and end, how economies are destabilized, or why couples divorce, involve ignoring and/or suppressing the very features that make them complex or assuming they cannot be known. This is the most common method of simplification, leave out those annoying details that detract from the main message of the person doing the explaining.

Complexity is not merely complicated simplicity and simplifying complexity is not the same as ignoring the properties that make a phenomenon complex. What I hope to make reasonable is that many complex phenomena are not understood because they are not properly represented. In the proper representation what happens in even very complex events seem natural and often even predictable, but such representations are not obvious and are frequently counter-intuitive. Thus, finding the proper representation is an adventure in itself.

On the other hand, there is the more disruptive situation in which the apparently simple is shown to be complex. This is the situation where the zigs and zags of reality have been smoothed over using a simple map that explains things in the way we choose to see them. Very often it is familiarity that gives the illusion of simplicity and leads to misunderstandings. For example, in the United States the view that the slave economy of the antebellum south was unprofitable, stagnant, inefficient, and moribund was wide spread according to the 1993 Noble Prize winner in economics Robert William Fogel [1]. Fogel corrects the historical miss impressions regarding slavery as follows:

...the new economic historians have demonstrated that slavery was quite profitable. To put it in contemporary terms, as an investment opportunity, slavery was the growth stock of the 1850’s. Thus when slaveowners invested in slaves, it was not because they were doddering idiots wedded to an economically moribund institution. Nor was it because they were noble men who were sacrificing their personal economic interests to save the country from the threat of barbarism. Perhaps slaveowners were nobly motivated. If so they were well rewarded for their nobility - with average rates of return in the neighborhood of 10 or 20 per cent per annum. New measurements also indicate that the slave economy was growing between 1840 and 1860. Far from being the laggard region, the rate of growth of per capita income in the South exceeded the national average. But perhaps the most startling of the new findings is the discovery that Southern agriculture was nearly 40 per cent more efficient in the utilization of its productive resources than was Northern agriculture.

Consequently it is not only accurate simplification of the complex that we pursue, but uncovering the complexity hidden beneath previously established, but misguided simple cognitive maps also concerns us. The latter is important because it is only by revealing the underlying complexity that we can hope to understand phenomena well enough to make them simple again and through understanding control them.

Much of our technological society appears to be simple because what makes it work is hidden under multiple layers of technology. A modern city cannot function without transportation networks for logistic support of its markets, sewers and water delivery networks for hygiene, the power grid, communication networks, and on and on. Each of these networks is in itself a complicated interconnected system of dynamic elements that is organized and/or designed to carry out specific functions. When the city is operating as intended these various networks are part of the background and are intended to function invisibly. On the other hand when one or more of these networks does not function correctly such as having an increasing number of homeless, skyrocketing health care costs, incredible gasoline prices and so on, the background becomes the foreground and we question why the city, county, state and federal governments cannot solve the social problem. This book is not about why we cannot solve these problems. It is about why we see certain things as problems in the first place.

Why are so many people poor? The existence of a poverty class is perhaps understandable in Third World countries, but why in the richest country in the world are there so many poor? Can’t we do something about it? The simple and direct answer to this social problem is that through the equitable redistribution of a nation’s wealth we can end poverty. But will giving everyone an equal share of a nation’s wealth end poverty? Or is this a pleasant myth created, spread and accepted because of our limited understanding of how our complex institutions work?

The idea of fairness is recent in human history and has slowly evolved in the classical writings of the last few hundred years. It took on much of its modern form in the nineteenth century when scientists turned their attention from the understanding of physical to that of social phenomena. The mathematics that made this transition possible was the introduction and development of statistics, which enabled scientists to identify the common aspects of apparently random data sets from a variety of phenomena. It should also be recalled that the mathematics of statistics was developed to predict the most favorable outcomes of games of chance in which the notion of a fair bet was central to the understanding of a wager. Recognizing the uncertainty in the outcome of human interactions the nineteenth century social scientists adopted statistics and probability as the proper calculus for describing social phenomena. This choice of mathematical repre-sentation introduced a number of foundational concepts into our understanding of society; these include equality and fairness.

1.1. One Scientist’s View

My approach to understanding the complex issues of today’s world is