Complexity: The Emerging Science at the Edge of Order and Chaos

Book by Waldrop M Mitchell

Chapter 1

The Irish Idea of a Hero

Sitting alone at his table by the bar, Brian Arthur stared out the front window of the tavern and did his best to ignore the young urban professionals drifting in to get an early start on Happy Hour. Outside, in the concrete canyons of the financial district, the typical San Francisco fog was turning into a typical San Francisco drizzle. That was fine by him. On this late afternoon of March 17, 1987, he wasn't in the mood to be impressed with brass fittings, ferns, and stained glass. He wasn't in a mood to celebrate Saint Patrick's Day. And he most definitely wasn't in a mood to carouse with ersatz Irishmen wearing bits of green on their pinstripes. He just wanted to silently sip his beer in frustrated rage. Stanford University Professor William Brian Arthur, native son of Belfast, Northern Ireland, was at rock bottom.

And the day had started so well.

That was the irony of it all. When he'd set out for Berkeley that morning, he'd actually been looking forward to the trip as a kind of triumphal reunion: local boy makes good. He'd really loved his years in Berkeley, back in the early 1970s. Perched on the hillsides north of Oakland, just across the bay from San Francisco, it was a pushy, vital, alive kind of place full of ethnics and street people and outrageous ideas. Berkeley was where he'd gotten his Ph. D. from the University of California, where he'd met and married a tall blonde doctoral student in statistics named Susan Peterson, where he'd spent his first "postdoc" year in the economics department. Berkeley, of all the places he'd lived and worked ever since, was the place he wanted to come home to.

Well now he was coming home, sort of. The event itself wouldn't be a big deal: just lunch with the chairman of the Berkeley economics department and one of his former professors there. But it was the first time he'd come back to his old department in years, and certainly the first time he'd ever done so feeling like an academic equal. He was coming back with twelve years of experience working all over the globe and a major reputation as a scholar of human fertility in the Third World. He was coming back as the occupant of an endowed chair of economics at Stanford -- the sort of thing that rarely gets handed out to anyone under age fifty. At age forty-one, Arthur was coming back as someone who had made it in academia. And who knew? The folks at Berkeley might even start talking about a lob offer.

Oh yes, he'd really been high on himself that morning. So why hadn't he, years ago, just stuck to the mainstream instead of trying to invent a whole new approach to economics? Why hadn't he played it safe instead of trying to get in step with some nebulous, half-imaginary scientific revolution?

Because he couldn't get it out of his head, that's why. Because he could see it almost everywhere he looked. The scientists barely seemed to recognize it themselves, most of the time. But after three hundred years of dissecting everything into molecules and atoms and nuclei and quarks, they finally seemed to be turning that process inside out. Instead of looking for the simplest pieces possible, they were starting to look at how those pieces go together into complex wholes.

He could see it happening in biology, where people had spent the past twenty years laying bare the molecular mechanisms of DNA, and proteins, and all the other components of the cell. Now they were also beginning to grapple with the essential mystery: how can several quadrillion such molecules organize themselves into an entity that moves, that responds, that reproduces, that is alive?

He could see it happening in the brain sciences, where neuroscientists, psychologists, computer scientists, and artificial intelligence researchers were struggling to comprehend the essence of mind: How do those billions of densely interconnected nerve cells inside our skulls give rise to feeling, thought, purpose, and awareness?

He could even see it happening in physics, where the physicists were still trying to come to terms with the mathematical theory of chaos, the intricate beauty of fractals, and the weird inner workings of solids and liquids. There was profound mystery here: Why is it that simple particles obeying simple rules will sometimes engage in the most astonishing, unpredictable behavior? And why is it that simple particles will spontaneously organize themselves into complex structures like stars, galaxies, snowflakes, and hurricanes -- almost as if they were obeying a hidden yearning for organization and order?

The signs were everywhere. Arthur couldn't quite put the feeling into words. Nobody could, so far as he could tell. But somehow, he could sense that all these questions were really the same question. Somehow, the old categories of science were beginning to dissolve. Somehow, a new, unified science was out there waiting to be born. It would be a rigorous science, Arthur was convinced, just as "hard" as physics ever was, and just as thoroughly grounded in natural law. But instead of being a quest for the ultimate particles, it would be about flux, change, and the forming and dissolving of patterns. Instead of ignoring everything that wasn't uniform and predictable, it would have a place for individuality and the accidents of history. Instead of being about simplicity, it would be about -- well, complexity.

And that was precisely where Arthur's new economics came in. Conventional economics, the kind he'd been taught in school, was about as far from this vision of complexity as you could imagine. Theoretical economists endlessly talked about the stability of the marketplace, and the balance of supply and demand. They transcribed the concept into mathematical equations and proved theorems about it. They accepted the gospel according to Adam Smith as the foundation for a kind of state religion. But when it came to instability and change in the economy -- well, they seemed to find the very idea disturbing, something they'd just as soon not talk about.

But Arthur had embraced instability. Look out the window, he'd told his colleagues. Like it or not, the marketplace isn't stable. The world isn't stable. It's full of evolution, upheaval, and surprise. Economics had to take that ferment into account. And now he believed he'd found the way to do that, using a principle known as "increasing returns" -- or in the King James translation, "To them that hath shall be given." Why had high-tech companies scrambled to locate in the Silicon Valley area around Stanford instead of in Ann Arbor or Berkeley? Because a lot of older high-tech companies were already there. Them that has gets. Why did the VHS video system run away with the market, even though Beta was technically a little bit better? Because a few more people happened to buy VHS systems early on, which led to more VHS movies in the video stores, which led to still more people buying VHS players, and so on. Them that has gets.

The examples could be multiplied endlessly. Arthur had convinced himself that increasing returns pointed the way to the future for economics, a future in which he and his colleagues would work alongside the physicists and the biologists to understand the messiness, the upheaval, and the spontaneous self-organization of the world. He'd convinced himself that increasing returns could be the foundation for a new and very different kind of economic science.

Unfortunately, however, he hadn't had much luck convincing anybody else. Outside of his immediate circle at Stanford, most economists thought his ideas were -- strange. Journal editors were telling him that this increasing-returns stuff "wasn't economics." In seminars, a good fraction of the audience reacted with outrage: how dare he suggest that the economy was not in equilibrium! Arthur found the vehemence baffling. But clearly he needed allies, people who could open their minds and hear what he was trying to tell them. And that, as much as any desire for a homecoming, was the reason he'd gone to Berkeley.

So there they had all been, sitting down to sandwiches at the faculty club. Tom Rothenberg, one of his former professors, had asked the inevitable question: "So, Brian, what are you working on these days?" Arthur had given him the two-word answer just to get started: "Increasing returns." And the economics department chairman, Al Fishlow, had stared at him with a kind of deadpan look.

"But -- we know increasing returns don't exist."

"Besides," jumped in Rothenberg with a grin, "if they did, we'd have to outlaw them!"

And then they'd laughed. Not unkindly. It was just an insider's joke. Arthur knew it was a joke. It was trivial. Yet that one sound had somehow shattered his whole bubble of anticipation. He'd sat there, struck speechless. Here were two of the economists he respected most, and they just -- couldn't listen. Suddenly Arthur had felt naive. Stupid. Like someone who didn't know enough not to believe in increasing returns. Somehow, it had been the last straw.

He'd barely paid attention during the rest of the lunch. After it was over and everyone had said their polite good-byes, he'd climbed into his faded old Volvo and driven back over the Bay Bridge into San Francisco. He'd taken the first exit he could, onto the Embarcadero. He'd stopped at the first bar he found. And he'd come in here to sit amidst the ferns and to give some serious thought to getting out of economics entirely.

Somewhere around the bottom of his second beer, Arthur realized that the place was beginning to get seriously noisy. The yuppies were arriving in force to celebrate the patron saint of Ireland. Well, maybe it was time to go home. This certainly wasn't accomplishing anything. He got up and walked out to his car; the foggy drizzle was still coming down.

Home was in Palo Alto, thirty-five miles south of the city in the suburban flats around Stanford. It was sunset when he finally pulled into the driveway. He must have made some noise. His wife, Susan, opened the front door and watched him as he was walking across the lawn: a slim, prematurely gray man who doubtless looked about as fed up and bedraggled as he felt.

"Well," she said, standing there in the doorway, "how did it go in Berkeley? Did they like your ideas?"

"It was the pits," said Arthur. "Nobody there believes in increasing returns."

Susan Arthur had seen her husband returning from the academic wars before. "Well," she said, trying to find something comforting to say, "I guess it wouldn't be a revolution, would it, if everybody believed in it at the start?"

Arthur looked at her, struck speechless for the second time that day. And then he just couldn't help it. He started to laugh.

The Education of a Scientist

When you're growing up Catholic in Belfast, says Brian Arthur, speaking in the soft, high cadences of that city, a certain rebelliousness sets in naturally. It wasn't that he ever felt oppressed, exactly. His father was a bank manager and his family was solidly middle class. The only sectarian incident that ever involved him personally came one afternoon as he was walking home in his parochial school uniform: a bunch of Protestant boys started pelting him with bits of brick and stone, and one piece of brick hit him in the forehead. (He could hardly see for the blood pouring into his eyes -- but he damn well threw that brick back.) Nor did he really feel that the Protestants were devils; his mother was a Protestant who converted to Catholicism when she married. He never even felt especially political. He tended to be much more interested in ideas and philosophy.

No, the rebelliousness is just something you pick up from the air. "The culture doesn't equip you to lead, but to undermine," he says. Look at whom the Irish admire: Wolfe Tone, Robert Emmet, Daniel O'Connell, Padraic Pearse. "All the Irish heroes were revolutionaries. The highest peak of heroism is to lead an absolutely hopeless revolution, and then give the greatest speech of your life from the dock -- the night before you're hanged.

"In Ireland," he says, "an appeal to authority never works."

In an odd sort of way, Arthur adds, that streak of Irish rebelliousness is what got him started in his own academic career. Catholic Belfast tended to be rather contemptuous of intellectuals. So, of course, he became one. In fact, he can remember wanting to be a "scientist" as early as age four, long before he knew what a scientist was. The idea just seemed deliciously exotic and mysterious. And yet, having gotten that idea in his head, young Brian was nothing if not determined. At school he plunged into engineering and physics and hard-edged mathematics as soon as he could. And in 1966 he had taken first-class honors in electrical engineering at Queen's University in Belfast. "Oh, I suppose you'll end up a wee professor somewhere," said his mother, who was in fact very proud; no one in her generation of the family had ever even attended a university.

Later in 1966 that same determination had led him across the Irish Sea to England and the University of Lancaster, where he started graduate studies in a highly mathematical form of engineering known as operations research -- basically, a set of techniques for calculating such things as how to organize a factory to get the most output for the least input, or how to keep a fighter jet under control when it is buffeted by unexpected forces. "At the time, British industry was in terrible shape," says Arthur. "I thought that maybe through science we could reorganize it and sort it out."

And in 1967, after the professors at Lancaster had proved insufferably stuffy and condescending -- "Well," says Arthur, doing his best imitation of bored British snobbery, "it's nice to have an Irishman in the department; it adds a little colour" -- he left for America and the University of Michigan in Ann Arbor. "From the moment I set foot here, I felt right at home," he says. "This was the sixties. The people were open, the culture was open, the scientific education was second to none. In the United States, anything seemed possible."

The one thing that wasn't possible in Ann Arbor, unfortunately, was ready access to the mountains and the sea, both of which Arthur loved. So he arranged to finish his Ph.D. work at Berkeley starting in the fall of 1969. And to support himself in the summer beforehand he applied for a job with McKinsey and Company, one of the top management consulting companies in the world.

That was a piece of incredibly good fortune. Arthur didn't realize until later just how lucky he was; people were clamoring to be hired at McKinsey. But it turned out that the company liked his operations research background and the fact that he knew German. They needed someone to work out of the Düsseldorf office. Was he interested?

Was he? Arthur had the time of his life. The last time he'd been in Germany he'd worked at a blue-collar summer job at 75 cents an hour. Now here he was, twenty-three years old, advising the board of directors of BASF on what to do with an oil and gas division or a fertilizer division worth hundreds of millions of dollars. "I learned that operating at the top was just as easy as operating at the bottom," he laughs.

But it was more than just an ego trip. Basically, McKinsey was selling modern American management techniques (a concept that didn't sound as funny in 1969 as it would have fifteen years later). "Companies in Europe at that time typically had hundreds of subdivisions," he says. "They didn't even know what they owned." Arthur discovered that he had a real taste for wading into mess...