Use code DAD23 for 20% off + Free shipping on $45+ Shop Now!
Lost in Math
How Beauty Leads Physics Astray
Formats and Prices
- Trade Paperback $17.99 $22.99 CAD
- ebook $3.99 $3.99 CAD
- Hardcover $30.00 $38.00 CAD
This item is a preorder. Your payment method will be charged immediately, and the product is expected to ship on or around June 2, 2020. This date is subject to change due to shipping delays beyond our control.
Also available from:
THEY WERE so sure, they bet billions on it. For decades physicists told us they knew where the next discoveries were waiting. They built accelerators, shot satellites into space, and planted detectors in underground mines. The world prepared to ramp up the physics envy. But where physicists expected a breakthrough, the ground wouldn’t give. The experiments didn’t reveal anything new.
What failed physicists wasn’t their math; it was their choice of math. They believed that Mother Nature was elegant, simple, and kind about providing clues. They thought they could hear her whisper when they were talking to themselves. Now Nature spoke, and she said nothing, loud and clear.
Theoretical physics is the stereotypical math-heavy, hard-to-understand discipline. But for a book about math, this book contains very little math. Strip away equations and technical terms and physics becomes a quest for meaning—a quest that has taken an unexpected turn. Whatever laws of nature govern our universe, they’re not what physicists thought they were. They’re not what I thought they were.
Lost in Math is the story of how aesthetic judgment drives contemporary research. It is my own story, a reflection on the use of what I was taught. But it is also the story of many other physicists who struggle with the same tension: we believe the laws of nature are beautiful, but is not believing something a scientist must not do?
The Hidden Rules of Physics
In which I realize I don’t understand physics anymore. I talk to friends and colleagues, see I’m not the only one confused, and set out to bring reason back to Earth.
The Conundrum of the Good Scientist
I invent new laws of nature; it’s what I do for a living. I am one of some ten thousand researchers whose task is to improve our theories of particle physics. In the temple of knowledge, we are the ones digging in the basement, probing the foundations. We inspect the cracks, the suspicious shortcomings in existing theories, and when we find ourselves onto something, we call for experimentalists to unearth deeper layers. In the last century, this division of labor between theorists and experimentalists worked very well. But my generation has been stunningly unsuccessful.
After twenty years in theoretical physics, most people I know make a career by studying things nobody has seen. They have concocted mind-boggling new theories, like the idea that our universe is but one of infinitely many that together form a “multiverse.” They have invented dozens of new particles, declared that we are projections of a higher-dimensional space, and that space is spawned by wormholes that tie together distant places.
These ideas are highly controversial and yet exceedingly popular, speculative yet intriguing, pretty yet useless. Most of them are so difficult to test, they are practically untestable. Others are untestable even theoretically. What they have in common is that they are backed up by theoreticians convinced that their math contains an element of truth about nature. Their theories, they believe, are too good to not be true.
The invention of new natural laws—theory development—is not taught in classes and not explained in textbooks. Some of it physicists learn studying the history of science, but most of it they pick up from older colleagues, friends and mentors, supervisors and reviewers. Handed from one generation to the next, much of it is experience, a hard-earned intuition for what works. When asked to judge the promise of a newly invented but untested theory, physicists draw upon the concepts of naturalness, simplicity or elegance, and beauty. These hidden rules are ubiquitous in the foundations of physics. They are invaluable. And in utter conflict with the scientific mandate of objectivity.
The hidden rules served us badly. Even though we proposed an abundance of new natural laws, they all remained unconfirmed. And while I witnessed my profession slip into crisis, I slipped into my own personal crisis. I’m not sure anymore that what we do here, in the foundations of physics, is science. And if not, then why am I wasting my time with it?
I WENT INTO physics because I don’t understand human behavior. I went into physics because math tells it how it is. I liked the cleanliness, the unambiguous machinery, the command math has over nature. Two decades later, what prevents me from understanding physics is that I still don’t understand human behavior.
“We cannot give exact mathematical rules that define if a theory is attractive or not,” says Gian Francesco Giudice. “However, it is surprising how the beauty and elegance of a theory are universally recognized by people from different cultures. When I tell you, ‘Look, I have a new paper and my theory is beautiful,’ I don’t have to tell you the details of my theory; you will get why I’m excited. Right?”
I don’t get it. That’s why I am talking to him. Why should the laws of nature care what I find beautiful? Such a connection between me and the universe seems very mystical, very romantic, very not me.
But then Gian doesn’t think that nature cares what I find beautiful, but what he finds beautiful.
“Most of the time it’s a gut feeling,” he says, “nothing that you can measure in mathematical terms: it is what one calls physical intuition. There is an important difference between how physicists and mathematicians see beauty. It’s the right combination between explaining empirical facts and using fundamental principles that makes a physical theory successful and beautiful.”
Gian is head of the theory division at CERN, the Conseil Européen pour la Recherche Nucléaire. CERN operates what is currently the largest particle collider, the Large Hadron Collider (LHC), humankind’s closest look yet at the elementary building blocks of matter: a $6 billion, 16-mile underground ring to accelerate protons and smash them together at almost the speed of light.
The LHC is a compilation of extremes: supercooled magnets, ultrahigh vacuum, computer clusters that, during the experiments, record about three gigabytes of data—comparable to several thousand ebooks—per second. The LHC brought together thousands of scientists, decades of research, and billions of high-tech components for one purpose: find out what we’re made of.
“Physics is a subtle game,” Gian continues, “and discovering its rules requires not only rationality but also subjective judgment. For me it is this unreasonable aspect that makes physics fun and exciting.”
I am calling from my apartment, cardboard boxes are stacked around me. My appointment in Stockholm has come to an end; it’s time to move on and chase another research grant.
When I graduated, I thought this community would be a home, a family of like-minded inquirers seeking to understand nature. But I have become increasingly alienated by colleagues who on the one hand preach the importance of unbiased empirical judgment and on the other hand use aesthetic criteria to defend their favorite theories.
“When you find a solution to a problem you have been working on, you get this internal excitement,” says Gian. “It is the moment in which you suddenly start seeing the structure emerging behind your reasoning.”
Gian’s research has focused on developing new theories of particle physics that hold the prospect of solving problems in existing theories. He has pioneered a method to quantify how natural a theory is, a mathematical measure from which one can read off how much a theory relies on improbable coincidences.1 The more natural a theory, the less coincidence it requires, and the more appealing it is.
“The sense of beauty of a physical theory must be something hardwired in our brain and not a social construct. It is something that touches some internal chord,” he says. “When you stumble on a beautiful theory you have the same emotional reaction that you feel in front of a piece of art.”
It’s not that I don’t know what he is talking about; I don’t know why it matters. I doubt my sense of beauty is a reliable guide to uncovering fundamental laws of nature, laws that dictate the behavior of entities that I have no direct sensory awareness of, never had, and never will have. For it to be hardwired in my brain, it ought to have been beneficial during natural selection. But what evolutionary advantage has there ever been to understanding quantum gravity?
And while creating works of beauty is a venerable craft, science isn’t art. We don’t seek theories to evoke emotional reactions; we seek explanations for what we observe. Science is an organized enterprise to overcome the shortcomings of human cognition and to avoid the fallacies of intuition. Science isn’t about emotion—it’s about numbers and equations, data and graphs, facts and logic.
I think I wanted him to prove me wrong.
When I ask Gian what he makes of the recent LHC data, he says: “We are so confused.” Finally, something I understand.
In the first years of its operation, the LHC dutifully delivered a particle called the Higgs boson, the existence of which had been predicted back in the 1960s. My colleagues and I had high hopes that this billion-dollar project would do more than just confirm what nobody doubted. We had found some promising cracks in the foundations that convinced us the LHC would also create other, so far undiscovered particles. We were wrong. The LHC hasn’t seen anything that would support our newly invented laws of nature.
Our friends in astrophysics haven’t fared much better. In the 1930s they had discovered that galaxy clusters contain a lot more mass than all visible matter combined can possibly account for. Even allowing for large uncertainty in the data, a new type of “dark matter” is needed to explain the observations. Evidence for the gravitational pull of dark matter has piled up, so we are sure it is there. What it is made of, however, has remained a mystery. Astrophysicists believe it is some type of particle not present here on Earth, one that neither absorbs nor emits light. They thought up new laws of nature, unconfirmed theories, to guide the construction of detectors meant to test their ideas. Starting in the 1980s, dozens of experimental crews have been hunting for these hypothetical dark matter particles. They haven’t found them. The new theories have remained unconfirmed.
It looks similarly bleak in cosmology, where physicists try in vain to understand what makes the universe expand faster and faster, an observation attributed to “dark energy.” They can mathematically show that this strange substrate is nothing but the energy carried by empty space, and yet they cannot calculate the amount of energy. It’s one of the cracks in the foundations that physicists attempt to peer through, but so far they have failed to see anything in support of the new theories they designed to explain dark energy.
Meanwhile, in the field of quantum foundations, our colleagues want to improve a theory that has no shortcomings whatsoever. They act based on the conviction that something is wrong with mathematical structures that don’t correspond to measurable entities. It irks them that, as Richard Feynman, Niels Bohr, and other heroes of last century’s physics complained, “nobody understands quantum mechanics.” Researchers in quantum foundations want to invent better theories, believing, as everyone else does, they are on the right crack. Alas, all experiments have upheld the predictions of the not-understandable theory from the last century. And the new theories? They are still untested speculations.
An enormous amount of effort went into these failed attempts to find new laws of nature. But for more than thirty years now we have not been able to improve the foundations of physics.
SO YOU want to know what holds the world together, how the universe was made, and what rules our existence goes by? The closest you will get to an answer is following the trail of facts down into the basement of science. Follow it until facts get sparse and your onward journey is blocked by theoreticians arguing whose theory is prettier. That’s when you know you’ve reached the foundations.
The foundations of physics are those ingredients of our theories that cannot, for all we presently know, be derived from anything simpler. At this bottommost level we presently have space, time, and twenty-five particles, together with the equations that encode their behavior. The subjects of my research area, then, are particles that move through space and time, occasionally hitting each other or forming composites. Don’t think of them as little balls; they are not, because of quantum mechanics (more about that later). Better think of them as clouds that can take on any shape.
In the foundations of physics we deal only with particles that cannot be further decomposed; we call them “elementary particles.” For all we presently know, they have no substructure. But the elementary particles can combine to make up atoms, molecules, proteins—and thereby create the enormous variety of structure we see around us. It’s these twenty-five particles that you, I, and everything else in the universe are made of.
But the particles themselves aren’t all that interesting. What is interesting are the relations between them, the principles that determine their interaction, the structure of the laws that gave birth to the universe and enabled our existence. In our game, it’s the rules we care about, not the pieces. And the most important lesson we have learned is that nature plays by the rules of mathematics.
Made of Math
In physics, theories are made of math. We don’t use math because we want to scare away those not familiar with differential geometry and graded Lie algebras; we use it because we are fools. Math keeps us honest—it prevents us from lying to ourselves and to each other. You can be wrong with math, but you can’t lie.
Our task as theoretical physicists is to develop the mathematics to either describe existing observations, or to make predictions that guide experimental strategies. Using mathematics in theory development enforces logical rigor and internal consistency; it ensures that theories are unambiguous and conclusions are reproducible.
The success of math in physics has been tremendous, and consequently this quality standard is now rigorously enforced. The theories we build today are sets of assumptions—mathematical relations or definitions—together with interpretations that connect the math with real-world observables.
But we don’t develop theories by writing down assumptions and then derive observable consequences in a sequence of theorems and proofs. In physics, theories almost always start out as loose patchworks of ideas. Cleaning up the mess that physicists generate in theory development, and finding a neat set of assumptions from which the whole theory can be derived, is often left to our colleagues in mathematical physics—a branch of mathematics, not of physics.
For the most part, physicists and mathematicians have settled on a fine division of labor in which the former complain about the finickiness of the latter, and the latter complain about the sloppiness of the former. On both sides, though, we are crucially aware that progress in one field drives progress in the other. From probability theory to chaos theory to the quantum field theories at the base of modern particle physics, math and physics have always proceeded hand in hand.
But physics isn’t math. Besides being internally consistent (not giving rise to conclusions that contradict each other) a successful theory must also be consistent with observation (not be in contradiction with the data). In my area of physics, where we deal with the most fundamental questions, this is a stringent demand. There is so much existing data that doing all the necessary calculations for newly proposed theories simply isn’t feasible. It is also unnecessary because there is a shortcut: We first demonstrate that a new theory agrees with the well-confirmed old theories to within measurement precision, thus reproducing the old theory’s achievements. Then we only have to add calculations for what more the new theory can explain.
Demonstrating that a new theory reproduces all the achievements of successful old theories can be extremely difficult. This is because a new theory might use an entirely different mathematical framework that looks nothing like that of the old theory. Finding a way to show that both nevertheless arrive at the same predictions for already-made observations often requires finding a suitable way to reformulate the new theory. This is straightforward in cases where the new theory directly employs the math of the old one, but it can be a big hurdle with entirely new frameworks.
Einstein, for example, struggled for years to prove that general relativity, his new theory of gravity, would reproduce the successes of the predecessor, Newtonian gravity. The problem wasn’t that he had the wrong theory; the problem was that he didn’t know how to find Newton’s gravitational potential in his own theory. Einstein had all the math right, but the identification with the real world was missing. Only after several wrong attempts did he hit on the right way to do it. Having the right math is only part of having the right theory.
There are other reasons we use math in physics. Besides keeping us honest, math is also the most economical and unambiguous terminology that we know of. Language is malleable; it depends on context and interpretation. But math doesn’t care about culture or history. If a thousand people read a book, they read a thousand different books. But if a thousand people read an equation, they read the same equation.
The main reason we use math in physics, however, is because we can.
While logical consistency is always a requirement for a scientific theory, not all disciplines lend themselves to mathematical modeling—using a language so rigorous doesn’t make sense if the data don’t match the rigor. And of all the scientific disciplines, physics deals with the simplest of systems, making it ideally suited for mathematical modeling.
In physics, the subjects of study are highly reproducible. We understand well how to control experimental environments and which effects can be neglected without sacrificing accuracy. Results in psychology, for example, are hard to reproduce because no two people are alike, and it is rarely known exactly which human quirks might play a role. But that’s a problem we don’t have in physics. Helium atoms don’t get hungry and are just as well-tempered on Monday as on Friday.
This precision is what makes physics so successful, but also what makes it so difficult. To the uninitiated, the many equations might appear inaccessible, but handling them is a matter of education and habituation. Understanding the math is not what makes physics difficult. The real difficulty is finding the right math. You can’t just take anything that looks like math and call it a theory. It’s the requirement that a new theory needs to be consistent, both internally consistent and consistent with experiment—with each and every experiment—that makes it so difficult.
Theoretical physics is a highly developed discipline. The theories that we work with today have stood up to a great many experimental tests. And every time the theories passed another test, it has become a little more difficult to improve anything about them. A new theory needs to accommodate all of the present theories’ success and still be a little better.
As long as physicists developed theories to explain existing or upcoming experiments, success meant getting the right numbers with the least amount of effort. But the more observations our theories could describe, the more difficult it became to test a proposed improvement. It took twenty-five years from the prediction of the neutrino to its detection, almost fifty years to confirm the Higgs boson, a hundred years to directly detect gravitational waves. Now the time it takes to test a new fundamental law of nature can be longer than a scientist’s full career. This forces theorists to draw upon criteria other than empirical adequacy to decide which research avenues to pursue. Aesthetic appeal is one of them.
In our search for new ideas, beauty plays many roles. It’s a guide, a reward, a motivation. It is also a systematic bias.
The movers have picked up my boxes, most of which I never bothered to unpack, knowing I wouldn’t stay here. Echoes of past moves return from empty cabinets. I call my friend and colleague Michael Krämer, professor of physics in Aachen, Germany.
Michael works on supersymmetry, “susy” for short. Susy predicts a large number of still undiscovered elementary particles, a partner for each of the already known particles and a few more. Among the proposed new laws of nature, susy is presently the most popular one. Thousands of my colleagues bet their careers on it. But so far, none of those extra particles have been seen.
“I think I started working on susy because that’s what people worked on when I was a student, in the mid- to late nineties,” says Michael.
The mathematics of susy is very similar to that of already established theories, and the standard physics curriculum is good preparation for students to work on susy. “It’s a well-defined framework; it was easy,” says Michael. It was a good choice. Michael received tenure in 2004 and now heads the research group New Physics at the Large Hadron Collider, funded by the German Research Foundation.
“I also like symmetries. That made it attractive for me.”
AS I’VE noted, on our quest to understand what the world is made of, we have found twenty-five different elementary particles. Supersymmetry completes this collection with a set of still undiscovered partner particles, one for each of the known particles, and some additional ones. This supersymmetric completion is appealing because the known particles are of two different types, fermions and bosons (named after Enrico Fermi and Satyendra Bose, respectively), and supersymmetry explains how these two types belong together.
Fermions are extreme individuals. No matter how hard you try, you will not get two of them to do the same thing in the same place—there must always be a difference between them. Bosons, on the other hand, have no such constraint and are happy to join each other in a common dance. This is why electrons, which are fermions, sit on separate shells around atomic nuclei. If they were bosons, they would instead sit together on the same shell, leaving the universe without chemistry—and without chemists, as our own existence rests on the little fermions’ refusal to share space.
Supersymmetry postulates that the laws of nature remain the same when bosons are exchanged with fermions. This means that every known boson must have a fermionic partner, and every known fermion must have a bosonic partner. But besides differing in their fermionic or bosonic affiliation, partner particles must be identical.
Since none of the already known particles match as required, we have concluded there are no supersymmetric pairs among them. Instead, new particles must be waiting to be discovered. It’s like we have a collection of odd pots and lids and are convinced that certainly the matching pieces must be around somewhere.
Unfortunately, the equations of supersymmetry do not tell us what the masses of the susy partners are. Since it takes more energy to produce heavier particles, a particle is more difficult to find when its mass is larger. All we have learned so far is that the superpartners, if they exist, are so heavy that the energy of our experiments isn’t yet large enough to create them.
Supersymmetry has much going for it. Besides revealing that bosons and fermions are two sides of the same coin, susy also aids in the unification of fundamental forces and has the potential to explain several numerical coincidences. Moreover, some of the supersymmetric particles have just the right properties to make up dark matter. I’ll tell you more about that in the later chapters.
SUPERSYMMETRY FITS so snugly with the existing theories that many physicists are convinced it must be right. “Despite the efforts of many hundreds of physicists conducting experiments in search of these particles, no superpartners have ever been observed or detected,” writes Fermilab physicist Dan Hooper. Yet “this has had little effect in deterring the theoretical physicists who passionately expect nature to be formulated this way—to be supersymmetric. To many of these scientists, the ideas behind supersymmetry are simply too beautiful and too elegant not to be part of our universe. They solve too many problems and fit into our world too naturally. To these true believers, the superpartner particles simply must exist.”2
Hooper isn’t the only one to emphasize the strength of this conviction. “For many theoretical physicists, it is hard to believe that supersymmetry does not play a role somewhere in nature,” notes physicist Jeff Forshaw.3 And in a 2014 Scientific American article titled “Supersymmetry and the Crisis in Physics,” particle physicists Maria Spiropulu and Joseph Lykken support their hope that evidence will come in, eventually, with the assertion that “it is not an exaggeration to say that most of the world’s particle physicists believe that supersymmetry must be true” (their emphasis).4
It adds to susy’s appeal that a symmetry relating bosons and fermions was long thought impossible because a mathematical proof seemed to forbid it.5 But no proof is better than its assumptions. It turned out that if the proof’s assumptions are relaxed, supersymmetry instead is the largest possible symmetry that can be accommodated in the existing theories.6 And how could nature not make use of such a beautiful idea?
- "[A] provocative new book."—New York Times
- "According to the physicist and prolific blogger Sabine Hossenfelder, Einstein and others who work in a similar way are 'lost in math,' the title of her lively and provocative book."—Wall Street Journal
- "Hossenfelder ably mixes simplified explanation of the science with compelling portraits of the fascinating characters who study it."—Vanity Fair
- "In her new book, Lost In Math, Sabine Hossenfelder adroitly confronts this crisis head on.... The book is a wild, deep, thought-provoking read that would make any reasonable person in the field who's still capable of introspection doubt themselves."—Forbes
- "Lost in Math is self-aware and dosed with acerbic wit, and it asks bold questions."—Nature
- "Sabine Hossenfelder's new book Lost in Math provides a well-informed take on the current situation in fundamental physical theory. The author is completely honest, utterly fearless, and often quite funny."—MAA Reviews
- "[Hossenfelder's] critical assessment of the field is appropriately timed." —Science
- "Eavesdrop on accessible and frank conversations in Hossenfelder's Lost in Math, which wrestles with big questions of quantum mechanics and beauty in a fun, fascinating way."—Popular Science
- Choice award for outstanding academic title
- "Entertaining and engaging."—Ars Technica
- "Hossenfelder's jaunt through the world of theoretical physics explicitly raises the question of whether the activities of thousands of physicists should actually count as 'science.' And if not, then what in tarnation are they doing?"—Weekly Standard (UK)
- "Even educated readers will struggle to understand the elements of modern physics, but they will have no trouble enjoying this insightful, delightfully pugnacious polemic about its leading controversy."—Kirkus (starred review)
- "This layreader-friendly, amusing treatise gives an enlightening look at a growing issue within physics."—Publishers Weekly
- "Emphasizing how much researchers have achieved in quantum mechanics while using math that is decidedly ugly, Hossenfelder urges her colleagues to start focusing on reality, not conceptual style. A provocative appeal for unattractive but fruitful science."—Booklist
- "Hossenfelder, a philosophically inclined physicist, presents the informed reader with a fascinating panorama of the current state of physics, replete with imaginative entities like wormholes, parallel universes, and bubbles associated with the baby universe whose existence cannot be established or falsified through standard experimental modes."—CHOICE
- "Born too late to savor the heady era when the standard model of particle physics came together, Sabine Hossenfelder is impatient for new waves of discovery. Might the pace of insights be slowing because illusions of mathematical beauty have beguiled her fellow theorists? Lost in Math chronicles her quest--through interviews and conversations--to set her own course for exploration."—Chris Quigg, Distinguished Scientist Emeritus, Fermi National Accelerator Laboratory
- "Lost in Math is a delight. It is engaging, witty, and utterly profound. If you want to know why so many contemporary theoretical physicists choose to believe unbelievable things, this is a great place to start."—Jim Baggott, author of Farewell to Reality
- "Centered around insightful interviews with leading theorists, Lost in Math provides a well-informed take on the current state of fundamental physical theory, from a physicist who is utterly fearless, completely honest, and quite funny."—Peter Woit, mathematical physicist at Columbia University and author of Not Even Wrong
- On Sale
- Jun 2, 2020
- Page Count
- 304 pages
- Basic Books