By Toby Ord
Formats and Prices
This item is a preorder. Your payment method will be charged immediately, and the product is expected to ship on or around March 23, 2021. This date is subject to change due to shipping delays beyond our control.
If all goes well, human history is just beginning. Our species could survive for billions of years – enough time to end disease, poverty, and injustice, and to flourish in ways unimaginable today. But this vast future is at risk. With the advent of nuclear weapons, humanity entered a new age, where we face existential catastrophes – those from which we could never come back. Since then, these dangers have only multiplied, from climate change to engineered pathogens and artificial intelligence. If we do not act fast to reach a place of safety, it will soon be too late.
Drawing on over a decade of research, The Precipice explores the cutting-edge science behind the risks we face. It puts them in the context of the greater story of humanity: showing how ending these risks is among the most pressing moral issues of our time. And it points the way forward, to the actions and strategies that can safeguard humanity.
An Oxford philosopher committed to putting ideas into action, Toby Ord has advised the US National Intelligence Council, the UK Prime Minister's Office, and the World Bank on the biggest questions facing humanity. In The Precipice, he offers a startling reassessment of human history, the future we are failing to protect, and the steps we must take to ensure that our generation is not the last.
"A book that seems made for the present moment." —New Yorker
Explore book giveaways, sneak peeks, deals, and more.
LIST OF FIGURES
1.1 How we settled the world
1.2 The cradles of civilization
1.3 Striking improvements over the last 200 years
2.1 A classification of existential catastrophes
4.1 The number of stockpiled nuclear warheads over time
4.2 World population from 1700 to 2100
5.1 Measures of progress and interest in AI
5.2 An extended classification of existential catastrophes
6.1 How risks can combine
8.1 A timeline showing the scale of the past and future
D.1 How a 10% and 90% risk may combine
LIST OF TABLES
3.1 Progress in tracking near-Earth asteroids
3.2 The probability per century of a supervolcanic eruption
3.3 The probability per century of a stellar explosion
3.4 Estimates of total natural extinction risk via humanity’s age
3.5 Estimates of total natural extinction risk via related species
3.6 The Big Five extinction events
4.1 Where is the carbon?
6.1 My existential risk estimates
If all goes well, human history is just beginning. Humanity is about two hundred thousand years old. But the Earth will remain habitable for hundreds of millions more—enough time for millions of future generations; enough to end disease, poverty and injustice forever; enough to create heights of flourishing unimaginable today. And if we could learn to reach out further into the cosmos, we could have more time yet: trillions of years, to explore billions of worlds. Such a lifespan places present-day humanity in its earliest infancy. A vast and extraordinary adulthood awaits.
Our view of this potential is easily obscured. The latest scandal draws our outrage; the latest tragedy, our sympathy. Time and space shrink. We forget the scale of the story in which we take part. But there are moments when we remember—when our vision shifts, and our priorities realign. We see a species precariously close to self-destruction, with a future of immense promise hanging in the balance. And which way that balance tips becomes our most urgent public concern.
This book argues that safeguarding humanity’s future is the defining challenge of our time. For we stand at a crucial moment in the history of our species. Fueled by technological progress, our power has grown so great that for the first time in humanity’s long history, we have the capacity to destroy ourselves—severing our entire future and everything we could become.
Yet humanity’s wisdom has grown only falteringly, if at all, and lags dangerously behind. Humanity lacks the maturity, coordination and foresight necessary to avoid making mistakes from which we could never recover. As the gap between our power and our wisdom grows, our future is subject to an ever-increasing level of risk. This situation is unsustainable. So over the next few centuries, humanity will be tested: it will either act decisively to protect itself and its longterm potential, or, in all likelihood, this will be lost forever.
To survive these challenges and secure our future, we must act now: managing the risks of today, averting those of tomorrow, and becoming the kind of society that will never pose such risks to itself again.
It is only in the last century that humanity’s power to threaten its entire future became apparent. One of the most harrowing episodes has just recently come to light. On Saturday, October 27, 1962, a single officer on a Soviet submarine almost started a nuclear war. His name was Valentin Savitsky. He was captain of the submarine B-59—one of four submarines the Soviet Union had sent to support its military operations in Cuba. Each was armed with a secret weapon: a nuclear torpedo with explosive power comparable to the Hiroshima bomb.
It was the height of the Cuban Missile Crisis. Two weeks earlier, US aerial reconnaissance had produced photographic evidence that the Soviet Union was installing nuclear missiles in Cuba, from which they could strike directly at the mainland United States. In response, the US blockaded the seas around Cuba, drew up plans for an invasion and brought its nuclear forces to the unprecedented alert level of DEFCON 2 (“Next step to nuclear war”).
On that Saturday, one of the blockading US warships detected Savitsky’s submarine and attempted to force it to the surface by dropping low-explosive depth charges as warning shots. The submarine had been hiding deep underwater for days. It was out of radio contact, so the crew did not know whether war had already broken out. Conditions on board were extremely bad. It was built for the Arctic and its ventilator had broken in the tropical water. The heat inside was unbearable, ranging from 113°F near the torpedo tubes to 140°F in the engine room. Carbon dioxide had built up to dangerous concentrations, and crew members had begun to fall unconscious. Depth charges were exploding right next to the hull. One of the crew later recalled: “It felt like you were sitting in a metal barrel, which somebody is constantly blasting with a sledgehammer.”
Increasingly desperate, Captain Savitsky ordered his crew to prepare their secret weapon:
Maybe the war has already started up there, while we are doing somersaults here. We’re going to blast them now! We will die, but we will sink them all—we will not disgrace our Navy!1
Firing the nuclear weapon required the agreement of the submarine’s political officer, who held the other half of the firing key. Despite the lack of authorization by Moscow, the political officer gave his consent.
On any of the other three submarines, this would have sufficed to launch their nuclear weapon. But by the purest luck, submarine B-59 carried the commander of the entire flotilla, Captain Vasili Arkhipov, and so required his additional consent. Arkhipov refused to grant it. Instead, he talked Captain Savitsky down from his rage and convinced him to give up: to surface amidst the US warships and await further orders from Moscow.2
We do not know precisely what would have happened if Arkhipov had granted his consent—or had he simply been stationed on any of the other three submarines. Perhaps Savitsky would not have followed through on his command. What is clear is that we came precariously close to a nuclear strike on the blockading fleet—a strike which would most likely have resulted in nuclear retaliation, then escalation to a full-scale nuclear war (the only kind the US had plans for). Years later, Robert McNamara, Secretary of Defense during the crisis, came to the same conclusion:
No one should believe that had U.S. troops been attacked by nuclear warheads, the U.S. would have refrained from responding with nuclear warheads. Where would it have ended? In utter disaster.3
Ever since the advent of nuclear weapons, humans have been making choices with such stakes. Ours is a world of flawed decision-makers, working with strikingly incomplete information, directing technologies which threaten the entire future of the species. We were lucky, that Saturday in 1962, and have so far avoided catastrophe. But our destructive capabilities continue to grow, and we cannot rely on luck forever.
We need to take decisive steps to end this period of escalating risk and safeguard our future. Fortunately, it is in our power to do so. The greatest risks are caused by human action, and they can be addressed by human action. Whether humanity survives this era is thus a choice humanity will make. But it is not an easy one. It all depends on how quickly we can come to understand and accept the fresh responsibilities that come with our unprecedented power.
This is a book about existential risks—risks that threaten the destruction of humanity’s longterm potential. Extinction is the most obvious way humanity’s entire potential could be destroyed, but there are others. If civilization across the globe were to suffer a truly unrecoverable collapse, that too would destroy our longterm potential. And we shall see that there are dystopian possibilities as well: ways we might get locked into a failed world with no way back.
While this set of risks is diverse, it is also exclusive. So I will have to set aside many important risks that fall short of this bar: our topic is not new dark ages for humanity or the natural world (terrible though they would be), but the permanent destruction of humanity’s potential.
Existential risks present new kinds of challenges. They require us to coordinate globally and intergenerationally, in ways that go beyond what we have achieved so far. And they require foresight rather than trial and error. Since they allow no second chances, we need to build institutions to ensure that across our entire future we never once fall victim to such a catastrophe.
To do justice to this topic, we will have to cover a great deal of ground. Understanding the risks requires delving into physics, biology, earth science and computer science; situating this in the larger story of humanity requires history and anthropology; discerning just how much is at stake requires moral philosophy and economics; and finding solutions requires international relations and political science. Doing this properly requires deep engagement with each of these disciplines, not just cherry-picking expert quotes or studies that support one’s preconceptions. This would be an impossible task for any individual, so I am extremely grateful for the extensive advice and scrutiny of dozens of the world’s leading researchers from across these fields.4
This book is ambitious in its aims. Through careful analysis of the potential of humanity and the risks we face, it makes the case that we live during the most important era of human history. Major risks to our entire future are a new problem, and our thinking has not caught up. So The Precipice presents a new ethical perspective: a major reorientation in the way we see the world, and our role in it. In doing so, the book aspires to start closing the gap between our wisdom and power, allowing humanity a clear view of what is at stake, so that we will make the choices necessary to safeguard our future.
I have not always been focused on protecting our longterm future, coming to the topic only reluctantly. I am a philosopher, at Oxford University, specializing in ethics. My earlier work was rooted in the more tangible concerns of global health and global poverty—in how we could best help the worst off. When coming to grips with these issues I felt the need to take my work in ethics beyond the ivory tower. I began advising the World Health Organization, World Bank and UK government on the ethics of global health. And finding that my own money could do hundreds of times as much good for those in poverty as it could do for me, I made a lifelong pledge to donate at least a tenth of all I earn to help them.5 I founded a society, Giving What We Can, for those who wanted to join me, and was heartened to see thousands of people come together to pledge more than £1 billion over our lifetimes to the most effective charities we know of, working on the most important causes. Together, we’ve already been able to transform the lives of tens of thousands of people.6 And because there are many other ways beyond our donations in which we can help fashion a better world, I helped start a wider movement, known as effective altruism, in which people aspire to use evidence and reason to do as much good as possible.
Since there is so much work to be done to fix the needless suffering in our present, I was slow to turn to the future. It was so much less visceral; so much more abstract. Could it really be as urgent a problem as suffering now? As I reflected on the evidence and ideas that would culminate in this book, I came to realize that the risks to humanity’s future are just as real and just as urgent—yet even more neglected. And that the people of the future may be even more powerless to protect themselves from the risks we impose than the dispossessed of our own time.
Addressing these risks has now become the central focus of my work: both researching the challenges we face, and advising groups such as the UK Prime Minister’s Office, the World Economic Forum and DeepMind on how they can best address these challenges. Over time, I’ve seen a growing recognition of these risks, and of the need for concerted action.
To allow this book to reach a diverse readership, I’ve been ruthless in stripping out the jargon, needless technical detail and defensive qualifications typical of academic writing (my own included). Readers hungry for further technical detail or qualifications can delve into the many endnotes and appendices, written with them in mind.7
I have tried especially hard to examine the evidence and arguments carefully and even-handedly, making sure to present the key points even if they cut against my narrative. For it is of the utmost importance to get to the truth of these matters—humanity’s attention is scarce and precious, and must not be wasted on flawed narratives or ideas8.
Each chapter of The Precipice illuminates the central questions from a different angle. Part One (The Stakes) starts with a bird’s-eye view of our unique moment in history, then examines why it warrants such urgent moral concern. Part Two (The Risks) delves into the science of the risks facing humanity, both from nature and from ourselves, showing that while some have been overstated, there is real risk and it is growing. So Part Three (The Path Forward) develops tools for understanding how these risks compare and combine, and new strategies for addressing them. I close with a vision of our future: of what we could achieve were we to succeed.
This book is not just a familiar story of the perils of climate change or nuclear war. These risks that first awoke us to the possibilities of destroying ourselves are just the beginning. There are emerging risks, such as those arising from biotechnology and advanced artificial intelligence, that may pose much greater risk to humanity in the coming century.
Finally, this is not a pessimistic book. It does not present an inevitable arc of history culminating in our destruction. It is not a morality tale about our technological hubris and resulting fall. Far from it. The central claim is that there are real risks to our future, but that our choices can still make all the difference. I believe we are up to the task: that through our choices we can pull back from the precipice and, in time, create a future of astonishing value—with a richness of which we can barely dream, made possible by innovations we are yet to conceive. Indeed, my deep optimism about humanity’s future is core to my motivation in writing this book. Our potential is vast. We have so much to protect.
STANDING AT THE PRECIPICE
It might be a familiar progression, transpiring on many worlds—a planet, newly formed, placidly revolves around its star; life slowly forms; a kaleidoscopic procession of creatures evolves; intelligence emerges which, at least up to a point, confers enormous survival value; and then technology is invented. It dawns on them that there are such things as laws of Nature, that these laws can be revealed by experiment, and that knowledge of these laws can be made both to save and to take lives, both on unprecedented scales. Science, they recognize, grants immense powers. In a flash, they create world-altering contrivances. Some planetary civilizations see their way through, place limits on what may and what must not be done, and safely pass through the time of perils. Others, not so lucky or so prudent, perish.
We live at a time uniquely important to humanity’s future. To see why, we need to take a step back and view the human story as a whole: how we got to this point and where we might be going next.
Our main focus will be humanity’s ever-increasing power—power to improve our condition and power to inflict harm. We shall see how the major transitions in human history have enhanced our power, and enabled us to make extraordinary progress. If we can avoid catastrophe we can cautiously expect this progress to continue: the future of a responsible humanity is extraordinarily bright. But this increasing power has also brought on a new transition, at least as significant as any in our past, the transition to our time of perils.
HOW WE GOT HERE
Very little of humanity’s story has been told; because very little can be told. Our species, Homo sapiens, arose on the savannas of Africa 200,000 years ago.2 For an almost unimaginable time we have had great loves and friendships, suffered hardships and griefs, explored, created, and wondered about our place in the universe. Yet when we think of humanity’s great achievements across time, we think almost exclusively of deeds recorded on clay, papyrus or paper—records that extend back only about 5,000 years. We rarely think of the first person to set foot in the strange new world of Australia some 70,000 years ago; of the first to name and study the plants and animals of each place we reached; of the stories, songs and poems of humanity in its youth.3 But these accomplishments were real, and extraordinary.
We know that even before agriculture or civilization, humanity was a fresh force in the world. Using the simple, yet revolutionary, technologies of seafaring, clothing and fire, we traveled further than any mammal before us. We adapted to a wider range of environments, and spread across the globe.4
What made humanity exceptional, even at this nascent stage? We were not the biggest, the strongest or the hardiest. What set us apart was not physical, but mental—our intelligence, creativity and language.5
Yet even with these unique mental abilities, a single human alone in the wilderness would be nothing exceptional. He or she might be able to survive—intelligence making up for physical prowess—but would hardly dominate. In ecological terms, it is not a human that is remarkable, but humanity.
Each human’s ability to cooperate with the dozens of other people in their band was unique among large animals. It allowed us to form something greater than ourselves. As our language grew in expressiveness and abstraction, we were able to make the most of such groupings: pooling together our knowledge, our ideas and our plans.
Crucially, we were able to cooperate across time as well as space. If each generation had to learn everything anew, then even a crude iron shovel would have been forever beyond our technological reach. But we learned from our ancestors, added minor innovations of our own, and passed this all down to our children. Instead of dozens of humans in cooperation, we had tens of thousands, cooperating across the generations, preserving and improving ideas through deep time. Little by little, our knowledge and our culture grew.7
At several points in the long history of humanity there has been a great transition: a change in human affairs that accelerated our accumulation of power and shaped everything that would follow. I will focus on three.8
The first was the Agricultural Revolution.9 Around 10,000 years ago the people of the Fertile Crescent, in the Middle East, began planting wild wheat, barley, lentils and peas to supplement their foraging. By preferentially replanting the seeds from the best plants, they harnessed the power of evolution, creating new domesticated varieties with larger seeds and better yields. This worked with animals too, giving humans easier access to meat and hides, along with milk, wool and manure. And the physical power of draft animals to help plow the fields or transport the harvest was the biggest addition to humanity’s power since fire.10
While the Fertile Crescent is often called “the cradle of civilization,” in truth civilization had many cradles. Entirely independent agricultural revolutions occurred across the world in places where the climate and local species were suitable: in east Asia; sub-Saharan Africa; New Guinea; South, Central and North America; and perhaps elsewhere too.11 The new practices fanned out from each of these cradles, changing the way of life for many from foraging to farming.
This had dramatic effects on the scale of human cooperation. Agriculture reduced the amount of land needed to support each person by a factor of a hundred, allowing large permanent settlements to develop, which began to unite together into states.12 Where the largest foraging communities involved perhaps hundreds of people, some of the first cities had tens of thousands of inhabitants. At its height, the Sumerian civilization contained around a million people.13 And 2,000 years ago, the Han dynasty of China reached sixty million people—about a hundred thousand times as many as were ever united in our forager past, and about ten times the entire global forager population at its peak.14
As more and more people were able to share their insights and discoveries, there were rapid developments in technology, institutions and culture. And the increasing numbers of people trading with one another made it possible for them to specialize in these areas—to devote a lifetime to governance, trade or the arts—allowing us to develop these ideas much more deeply.
Over the first 6,000 years of agriculture, we achieved world-changing breakthroughs including writing, mathematics, law and the wheel.15 Of these, writing was especially important for strengthening our ability to cooperate across time and space: increasing the bandwidth between generations, the reliability of the information, and the distance over which ideas could be shared.
The next great transition was the Scientific Revolution.16 Early forms of science had been practiced since ancient times, and the seeds of empiricism can be found in the work of medieval scholars in the Islamic world and Europe.17 But it was only about 400 years ago that humanity developed the scientific method and saw scientific progress take off.18 This helped replace a reliance on received authorities with careful observation of the natural world, seeking simple and testable explanations for what we saw. The ability to test and discard bad explanations helped us break free from dogma, and allowed for the first time the systematic creation of knowledge about the workings of nature.
Some of our new-found knowledge could be harnessed to improve the world around us. So the accelerated accumulation of knowledge brought with it an acceleration of technological innovation, giving humanity increasing power over the natural world. The rapid pace allowed people to see transformative effects of these improvements within their own lifetimes. This gave rise to the modern idea of progress. Where the world had previously been dominated by narratives of decline and fall or of a recurring cycle, there was increasing interest in a new narrative: a grand project of working together to build a better future.
Soon, humanity underwent a third great transition: the Industrial Revolution. This was made possible by the discovery of immense reserves of energy in the form of coal and other fossil fuels. These are formed from the compressed remains of organisms that lived in eons past, allowing us access to a portion of the sunlight that shone upon the Earth over millions of years.19 We had already begun to drive simple machines with the renewable energy from the wind, rivers and forests; fossil fuels allowed access to vastly more energy, and in a much more concentrated and convenient form.
But energy is nothing without a way of converting it to useful work, to achieve our desired changes in the world. The steam engine allowed the stored chemical energy of coal to be turned into mechanical energy.20 This mechanical energy was then used to drive machines that performed massive amounts of labor for us, allowing raw materials to be transformed into finished products much more quickly and cheaply than before. And via the railroad, this wealth could be distributed and traded across long distances.
Productivity and prosperity began to accelerate, and a rapid sequence of innovations ramped up the efficiency, scale and variety of automation, giving rise to the modern era of sustained economic growth.21
The effects of these transitions have not always been positive. Life in the centuries following the Agricultural Revolution generally involved more work, reduced nutrition and increased disease.22 Science gave us weapons of destruction that haunt us to this day. And the Industrial Revolution was among the most destabilizing periods in human history. The unequal distribution of gains in prosperity and the exploitative labor practices led to the revolutionary upheavals of the early twentieth century.23 Inequality between countries increased dramatically (a trend that has only begun to reverse in the last two decades).24 Harnessing the energy stored in fossil fuels has released greenhouse gases, while industry fueled by this energy has endangered species, damaged ecosystems and polluted our environment.
Yet despite these real problems, on average human life today is substantially better than at any previous time. The most striking change may be in breaking free from poverty. Until 200 years ago—the last thousandth of our history25
- On Sale
- Mar 23, 2021
- Page Count
- 480 pages
- Hachette Books