Do Dice Play God?

1

SIX AGES OF UNCERTAINTY

UNCERTAINTY ISN’T ALWAYS BAD. We like surprises, as long as they’re pleasant ones. Many of us enjoy a flutter on the horses, and most sports would be pointless if we knew at the start who was going to win. Some prospective parents are keen not to be told the sex of the baby. Most of us, I suspect, would prefer not to know in advance the date of their own death, let alone how it will occur. But those are exceptions. Life is a lottery. Uncertainty often breeds doubt, and doubt makes us feel uncomfortable, so we want to reduce, or better still eliminate, uncertainty. We worry about what will happen. We look out for the weather forecast, even though we know that weather is notoriously unpredictable and the forecast is often wrong.

When we watch the news on television, or read a newspaper, or surf the web, the extent to which we don’t know what’s going to happen can be overwhelming. Aircraft crash at random. Earthquakes and volcanoes devastate communities, even large parts of cities. The financial sector booms and busts, and although we speak of the ‘boom and bust cycle’, all we mean is that boom follows bust and bust follows boom. We have little idea when one of them will switch to the other. We might as well speak of the ‘rainy and dry cycle’ and claim to forecast the weather. When elections are in the offing, we keep an eye on the opinion polls, hoping to get some inkling about who is likely to win. Polls in recent years seem to have become less reliable, but they still have the power to reassure or annoy us.

Sometimes we’re not just uncertain; we’re uncertain about what we ought to be uncertain about. Most of us worry about climate change, but a vocal minority insists it’s all a hoax–perpetrated by scientists (who couldn’t organise a hoax to save their lives), or by the Chinese, or maybe Martians… pick your favourite conspiracy theory. But even the climatologists who predicted climate change offer few certainties about its precise effects. They do have a pretty clear handle on their general nature, though, and in practical terms that’s more than enough to set alarm bells ringing.

Not only are we uncertain about what Mother Nature will throw at us; we’re not too sure about what we throw at ourselves. The world’s economies are still reeling from the 2008 financial crisis, while the people who caused it are mostly conducting their business as before, which is likely to bring about an even bigger financial disaster. We have very little idea how to forecast global finances.

After a period of relative (and historically unusual) stability, world politics is becoming increasingly fractured, and old certainties are being shaken. ‘Fake News’ is submerging genuine facts in a barrage of disinformation. Predictably, those who complain most loudly about it are often the ones responsible for the fakery. The internet, instead of democratising knowledge, has democratised ignorance and bigotry. By removing the gatekeepers, it has left the gates hanging off their hinges.

Human affairs have always been messy, but even in science, the old idea of nature obeying exact laws has given way to a more flexible view. We can find rules and models that are approximately true (in some areas ‘approximate’ means ‘to ten decimal places’, in others it means ‘between ten times as small and ten times as large’) but they’re always provisional, to be displaced if and when fresh evidence comes along. Chaos theory tells us that even when something does obey rigid rules, it may still be unpredictable. Quantum theory tells us that deep down at its smallest levels, the universe is inherently unpredictable. Uncertainty isn’t just a sign of human ignorance; it’s what the world is made of.

WE COULD JUST BE FATALISTIC about the future, as many people are. But most of us are uncomfortable about that way of living. We suspect that it will probably lead to disaster, and we have a sneaking feeling that with a little foresight, disaster might be averted. A common human tactic, when faced with something we dislike, is either to guard against it, or try to change it. But what precautions should we take, when we don’t know what’s going to happen? After the Titanic disaster, ships were required to fit extra lifeboats. Their weight caused the S.S. Eastland to capsize on Lake Michigan, and 848 people died. The Law of Unintended Consequences can foil the best of intentions.

We’re concerned about the future because we’re time-binding animals. We have a strong sense of our location in time, we anticipate future events, and we act now because of those anticipations. We don’t have time machines, but we often behave as if we do, so that a future event causes us to take action before it occurs. Of course the real cause of today’s action isn’t the wedding or the thunderstorm or the rent bill that will happen tomorrow. It’s our present belief that it’s going to happen. Our brains, shaped by both evolution and individual learning, let us choose our actions today to make our lives easier tomorrow. Brains are decision-making machines, making guesses about the future.

The brain makes some decisions a split second ahead. When a cricketer or baseball player catches the ball, there’s a small but definite time delay between the visual system detecting the ball and the brain working out where it is. Remarkably, they usually catch the ball, because their brain is pretty good at anticipating its trajectory, but when they fumble an apparently easy catch, either the prediction or their reaction to it went wrong. The whole process is subconscious and apparently seamless, so we don’t notice that we live our entire lives in a world that’s running a split second ahead of our brain.

Other decisions may be taken days, weeks, months, years, or even decades ahead. We get up in time to get to the bus or train to work. We buy food for tomorrow’s meals, or next week’s. We plan a family outing for the coming public holiday, and everyone involved does things now to prepare for then. Wealthy parents in the UK sign their children up for the posh schools before they’re born. Wealthier ones plant trees that won’t mature for centuries, so that their great-great-great-grandchildren will get an impressive view.

How does the brain foretell the future? It builds simplified internal models of how the world works, or may work, or is presumed to work. It feeds what it knows into the model, and observes the outcome. If we spot a loose carpet, one of these models tells us that this could be dangerous, causing someone to trip and fall down the stairs. We take preventative action and fix the carpet in the correct position. It doesn’t really matter whether this particular forecast is right. In fact, if we’ve fixed the carpet properly, it can’t be right, because the conditions fed into the model no longer apply. However, evolution or personal experience can test the model, and improve it, by seeing what happens in similar cases when preventative action isn’t taken.

Models of this kind need not be accurate descriptions of how the world works. Instead, they amount to beliefs about how the world works. And so, over tens of thousands of years, the human brain evolved into a machine that makes decisions based on its beliefs about where those decisions will lead. It’s therefore no surprise that one of the earliest ways we learned to cope with uncertainty was to construct systematic beliefs about supernatural beings who were in control of nature. We knew we weren’t in control, but nature constantly surprised us, often unpleasantly, so it seemed reasonable to assume that some inhuman entities–spirits, ghosts, gods, goddesses–were in control. Soon a special class of people came into being, who claimed they could intercede with the gods to help us mortals achieve our aims. People who claimed to foretell the future–prophets, seers, fortunetellers, oracles–became especially valued members of the community.

This was the first Age of Uncertainty. We invented belief systems, which became ever more elaborate because every generation wanted to make them more impressive. We rationalised the uncertainty of nature as the will of the gods.

THIS EARLIEST STAGE OF CONSCIOUS human engagement with uncertainty lasted for thousands of years. It agreed with the evidence, because the will of the gods could credibly be whatever happened. If the gods were pleased, good things happened; if they were angry, bad things happened. As proof, if good things happened to you then you were obviously pleasing the gods, and if bad things happened it was your own fault for making them angry. So beliefs about gods became entangled with moral imperatives.

Eventually, it began to dawn on increasing numbers of people that belief systems with that flexibility didn’t really explain anything. If the reason for the sky being blue is that the gods made it that way, it might just as well have been pink or purple. Humanity began to explore a different way of thinking about the world, based on logical inference supported (or denied) by observational evidence.

This was science. It explains blue sky in terms of scattering of light by fine dust in the upper atmosphere. It doesn’t explain why blue looks blue; the neuroscientists are working on that, but science has never claimed to understand everything. As it grew, science achieved increasingly many successes, along with some ghastly failures, and it began to give us the ability to control some aspects of nature. The discovery of the relation between electricity and magnetism in the 19th century was one of the first truly revolutionary instances of science being turned into technology that affected the lives of almost everybody.

Science showed us that nature can be less uncertain than we think. Planets don’t wander about the sky according to godly whim: they follow regular elliptical orbits, aside from tiny disturbances that they inflict on each other. We can work out which ellipse is appropriate, understand the effect of those tiny disturbances, and predict where a planet will be centuries ahead. Indeed, nowadays, millions of years, subject to the limitations imposed by chaotic dynamics. There are natural laws; we can discover them and use them to predict what will happen. The uncomfortable feeling of uncertainty gave way to the belief that most things would be explicable if we could tease out the underlying laws. Philosophers began to wonder whether the entire universe is just the working out, over aeons of time, of those laws. Maybe free will is an illusion, and it’s all a huge clockwork machine.

Perhaps uncertainty is merely temporary ignorance. With enough effort and thought, all can become clear. This was the second Age of Uncertainty.

SCIENCE ALSO FORCED US TO find an effective way to quantify how certain or uncertain an event is: probability. The study of uncertainty became a new branch of mathematics, and the main thrust of this book is to examine the different ways in which we’ve exploited mathematics in the quest to render our world more certain. Many other things have contributed too, such as politics, ethics, and art, but I’ll focus on the role of mathematics.

Probability theory grew from the needs and experiences of two very different groups of people: gamblers and astronomers. Gamblers wanted a better grasp of ‘the odds’, astronomers wanted to obtain accurate observations from imperfect telescopes. As the ideas of probability theory sank into human consciousness, the subject escaped its original confines, informing us not just about dice games and the orbits of asteroids, but about fundamental physical principles. Every few seconds, we breathe in oxygen and other gases. The vast number of molecules that make up the atmosphere bounce around like diminutive billiard balls. If they all piled up in one corner of the room while we were in the opposite corner, we’d be in trouble. In principle, it could happen, but the laws of probability imply that it’s so rare that in practice it never does. Air stays uniformly mixed because of the second law of thermodynamics, which is often interpreted as stating that the universe is always becoming more disordered. The second law also has a somewhat paradoxical relationship to the direction in which time flows. This is deep stuff.

Thermodynamics was a relative latecomer to the scientific scene. By the time it arrived, probability theory had entered the world of human affairs. Births, deaths, divorces, suicides, crime, height, weight, politics. The applied arm of probability theory, statistics, was born. It gave us powerful tools to analyse everything from measles epidemics to how people will vote in a forthcoming election. It shed some light, though not as much as we’d like, on the murky world of finance. It told us that we’re creatures afloat on a sea of probabilities.

Probability, and its applied arm statistics, dominated the third Age of Uncertainty.

THE FOURTH AGE OF UNCERTAINTY arrived with a bang, at the start of the 20th century. Until then, all forms of uncertainty that we had encountered had a common feature: uncertainty reflected human ignorance. If we were uncertain about something, it was because we didn’t have the information needed to predict it. Consider tossing a coin, one of the traditional icons of randomness. However, a coin is a very simple mechanism, mechanical systems are deterministic, and in principle any deterministic process is predictable. If we knew all the forces acting on a coin, such the initial speed and direction of the toss, how fast it was spinning, and about which axis, we could use the laws of mechanics to calculate which way up it would land.

New discoveries in fundamental physics forced us to revise that view. It may be true of coins, but sometimes the information we need simply isn’t available, because even nature doesn’t know it. Around 1900, physicists were starting to understand the structure of matter on very small scales–not just atoms, but the subatomic particles into which atoms can be divided. Classical physics, of the kind that emerged from Isaac Newton’s breakthroughs with the laws of motion and gravity, had given humanity an extensive understanding of the physical world, tested using measurements of increasingly high precision. Out of all the theories and experiments, two different ways of thinking about the world crystallised: particles and waves.

A particle is a tiny lump of matter, precisely defined and localised. A wave is like ripples on water, a disturbance that moves; more ephemeral than a particle, and extending throughout a larger region of space. Planetary orbits can be calculated by pretending the planet is a particle, because the distances between planets and stars are so gigantic that if you scale everything down to human size, planets become particles. Sound is a disturbance in the air that travels, even though all the air stays in pretty much the same place, so it’s a wave. Particles and waves are icons of classical physics, and they’re very different.

In 1678 there was a big controversy about the nature of light. Christiaan Huygens presented his theory that light is a wave to the Paris Academy of Sciences. Newton was convinced that light is a stream of particles, and his view prevailed. Eventually, after a hundred years spent barking up the wrong tree, new experiments settled the issue. Newton was wrong, and light is a wave.

Around 1900 physicists discovered the photoelectric effect: light hitting certain types of metal can cause a small electrical current to flow. Albert Einstein deduced that light is a stream of tiny particles–photons. Newton had been right all along. But Newton’s theory had been discarded for a good reason: lots of experiments showed very clearly that light is a wave. The debate opened up all over again. Is light a wave, or a particle? The eventual answer was ‘both’. Sometimes light behaves like a particle, sometimes like a wave. It depends on the experiment. This was all very mysterious.

A few pioneers quickly began to see a way to make sense of the puzzle, and quantum mechanics was born. All of the classical certainties, such as the position of a particle and how fast it moves, turned out not to apply to matter on subatomic scales. The quantum world is riddled with uncertainty. The more precisely you measure the position of a particle, the less sure you can be about how fast it’s moving. Worse, the question ‘where is it?’ has no good answer. The best you can do is to describe the probability that it’s located in a given place. A quantum particle isn’t a particle at all, just a fuzzy cloud of probabilities.

The more deeply physicists probed the quantum world, the fuzzier everything became. They could describe it mathematically, but the mathematics was weird. Within a few decades they had become convinced that quantum phenomena are irreducibly random. The quantum world really is made from uncertainty, there’s no missing information, and no deeper level of description exists. ‘Shut up and calculate’ became the watchword; don’t ask awkward questions about what it all means.

WHILE PHYSICS WENT DOWN THE quantum route, mathematics blazed its own new trail. We used to think that the opposite of a random process is a deterministic one: given the present, only one future is possible. The fifth Age of Uncertainty emerged when mathematicians, and a few scientists, realised that a deterministic system can be unpredictable. This is chaos theory, the media’s name for nonlinear dynamics. The development of quantum theory might have been rather different if mathematicians had made that vital discovery much earlier than they did. In fact, one example of chaos was discovered before quantum theory, but it was seen as an isolated curiosity. A coherent theory of chaos didn’t appear until the 1960s and 1970s. Nevertheless, I’ll tackle chaos before quantum theory, for presentational reasons.

‘Prediction is very difficult, especially about the future,’ said physicist Niels Bohr (or was it Yogi Berra? See, we can’t even be certain of that).¹ It’s not as funny as it sounds, because prediction is different from forecasting. Most predictions in science predict that an event will happen under certain conditions, but not when. I can predict that an earthquake happens because stresses build up in rocks, and that prediction can be tested by measuring the stresses. But that’s not a method for predicting an earthquake, which requires determining, ahead of time, when it will happen. It’s even possible to ‘predict’ that some event did happen in the past, which is a legitimate test of a theory if no one had noticed until they went back to the old records and looked. I know this is often called ‘postdiction’, but as far as testing a scientific hypothesis goes, it’s the same thing. In 1980 Luiz and Walter Alvarez predicted that 65 million years ago an asteroid hit the Earth and killed the dinosaurs. It was a genuine prediction because, after making it, they could search the geological and fossil records for evidence for or against.

Observations over decades show that the sizes of beaks among some species of Darwin’s finches, on the Galápagos Islands, are entirely predictable–provided you can predict the average yearly rainfall. The sizes change in lockstep with how wet or dry the years are. In dry years, seeds are harder, so bigger beaks are needed. In wet years, smaller beaks work better. Here, beak size is conditionally predictable. If a reliable oracle told us next year’s rainfall, we could confidently predict the beak sizes. That’s definitely different from the beak sizes being random. If they were, they wouldn’t follow the rainfall.

It’s not unusual for some features of a system to be predictable while others are unpredictable. My favourite example is astronomical. In 2004 astronomers announced that an obscure asteroid called 99942 Apophis might collide with the Earth on 13 April 2029, or if it missed in just the right way, there could be a second opportunity on 13 April 2036. One journalist (to be fair, in a humorous column) asked: How can they be so sure about the date when they don’t know the year?

Stop reading and think about it. Hint: what is a year?

It’s very simple. Potential collisions occur when the orbit of the asteroid intersects, or nearly intersects, that of the Earth. These orbits change slightly as time passes, affecting how closely the two bodies approach each other. If we don’t have enough observations to determine the asteroid’s orbit with sufficient precision, we can’t be sure how close it will come to the Earth. The astronomers had enough orbital data to rule out most years over the next few decades, but not 2029 or 2036. In contrast, the date of a possible collision behaves quite differently. The Earth returns to (almost) the same location in its orbit after one year has passed. That’s the definition of ‘year’. In particular, our planet comes close to the intersection with the asteroid’s orbit at intervals of one year; that is, on the same day every year. (Maybe one day ahead or behind if the timing is close to midnight.) As it happens, that day is 13 April for Apophis.

So Bohr or Berra was absolutely right, and his statement is really quite profound. Even when we understand how things work, in considerable detail, we may have no idea what will happen next week, next year, or next century.

WE HAVE NOW ENTERED THE sixth Age of Uncertainty, characterised by the realisation that uncertainty comes in many forms, each being comprehensible to some extent. We now possess an extensive mathematical toolkit to help us make sensible choices in a world that’s still horribly uncertain. Fast, powerful computers let us analyse huge amounts of data quickly and accurately. ‘Big data’ is all the rage, although right now we’re better at collecting it than we are at doing anything useful with it. Our mental models can be augmented with computational ones. We can perform more calculations in a second than all the mathematicians in history managed with pen and paper. By combining our mathematical understanding of the different forms that uncertainty can take, with intricate algorithms to tease out patterns and structures, or just to quantify how uncertain we are, we can to some extent tame our uncertain world.

We’re much better at predicting the future than we used to be. We still get annoyed when the weather forecast tells us it’s not going to rain tomorrow, and it does; but the accuracy of weather prediction has improved considerably since 1922 when the visionary scientist Lewis Fry Richardson wrote Weather Prediction by Numerical Process. Not only is the forecast better: it’s accompanied by an assessment of the probability that it’s right. When the weather website says ‘25% chance of rain’ it means that on 25% of occasions when the same statement has been made, rain has duly fallen. If it says ‘80% chance of rain’ then it’s likely to be right four times out of five.

When the Bank of England issues forecasts of changes to the rate of inflation, it similarly provides an estimate of how reliable its mathematical modellers think the forecast is. It also found an effective way to present this estimate to the public: a ‘fan chart’ which plots the evolution, over time, of the predicted inflation rate, but not as a single line: as a shaded band. As time passes, the band gets wider, indicating a loss of accuracy. The density of the ink indicates the level of probability: a dark region is more likely than a fainter one. The shaded area covers 90% of the probable forecasts.

The messages here are twofold. First: as understanding advances, predictions can be made more accurate. Second: we can manage uncertainty by working out how confident we should be in the prediction.

A third message is also beginning to be understood. Sometimes uncertainty can actually be useful. Many areas of technology deliberately create controlled amounts of uncertainty, in order to make devices and processes work better. Mathematical techniques for finding the best solution to an industrial problem use random disturbances to avoid getting stuck on strategies that are the best compared to near neighbours, but not as good as more distant ones. Random changes to recorded data improve the accuracy of weather forecasts. SatNav uses streams of pseudorandom numbers to avoid problems with electrical interference. Space missions exploit chaos to save expensive fuel.

FOR ALL THAT, WE’RE STILL children ‘playing on the seashore’, as Newton put it, ‘finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before [us].’ Many deep questions remain unanswered. We don’t really understand the global financial system, even though everything on the planet depends on it. Our medical expertise lets us spot most disease epidemics early on, so we can take steps to mitigate their effects, but we can’t always predict how they spread. Every so often new diseases appear, and we’re never sure when and where the next one will strike. We can make exquisitely accurate measurements of earthquakes and volcanoes, but our track record of predicting them is as shaky as the ground beneath our feet.

The more we find out about the quantum world, the more hints there are that some deeper theory can make its apparent paradoxes more reasonable. Physicists have given mathematical proofs that quantum uncertainty can’t be resolved by adding a deeper layer of reality. But proofs involve assumptions, which are open to challenge, and loopholes keep turning up. New phenomena in classical physics have uncanny similarities to quantum puzzles, and we know that their workings have nothing to do with irreducible randomness. If we’d known about them, or about chaos, before discovering quantum weirdness, today’s theories might have been very different. Or perhaps we’d have wasted decades looking for determinism where none exists.

I’ve bundled everything up neatly into six Ages of Uncertainty, but the reality was less tidy. Principles that ultimately turned out to be very simple emerged in complex and confusing ways. There were unexpected twists and turns, big leaps forward, and dead ends. Some mathematical advances turned out to be red herrings; others languished for years before anyone recognised their significance. There were ideological splits, even among the mathematicians. Politics, medicine, money, and the law got in on the act, sometimes all at once.