Reinventing Capitalism in the Age of Big Data

– 1 –

REINVENTING CAPITALISM

IT SHOULD HAVE BEEN A VICTORY CELEBRATION. BY THE time eBay’s new CEO, Devin Wenig, climbed the stage for the online marketplace’s twentieth-anniversary event in September 2015, goods worth more than $700 billion had been traded on eBay’s platform, and active eBay users had reached 160 million. The company Pierre Omidyar had started in 1995 as a small side-business turned into what looked like a perpetual money-maker. EBay had taken an old but highly successful idea, the market, and put it online.

Because eBay’s market was no longer a physical place, it never closed. And thanks to the Internet’s global reach, pretty much everyone connected to it could buy and sell on it. Through eBay’s unique rating system, it created a way to trust market participants without knowing them. Together that made the new virtual marketplace tremendously attractive, resulting in what economists call a thick market, a market with lots of buyers and sellers. Thick markets are good markets, because they increase the likelihood of finding what one is looking for. EBay also took a feature of traditional markets and improved on it: it replaced fixed prices with an auction mechanism, a far better way to achieve optimal price, as economics students learn in their first semester.

A marketplace with global reach that’s always open and makes transacting simple, easy, and efficient—that’s the recipe for eBay’s meteoric rise. It not only ushered in the Internet economy but also seemed to reconfirm the preeminent role markets play in our economy.

But to journalists attending the celebration, Wenig looked more like “a general rallying the troops of a beleaguered army,” and his speech felt like a pep talk—with good reason. The world’s largest marketplace had lost some of its mojo. Analysts on Wall Street even labeled eBay “due for a reset.” With so much going for it, some may see eBay’s recent troubles as a bout of bad management, aggravated by bad luck. But to us it’s an indication of a much larger, structural shift.

Just months before eBay’s twentieth anniversary, Yahoo, another early Internet pioneer, was suffering its own market woes. Yahoo owned a substantial chunk of Chinese online marketplace Alibaba, and based on Alibaba’s share price, its holding of Alibaba’s shares was more valuable than Yahoo’s total market capitalization. So sellers of Yahoo’s shares were essentially paying buyers to take on their stock and shares of Yahoo were trading at an effectively negative price. That doesn’t make sense, of course, because the value of a share of common stock can’t be negative. But stock prices, economists tell us, should reflect the collective wisdom of the market; so they ought to be right. Something was wrong—terribly wrong.

EBay’s surprising troubles and Yahoo’s crazy share price aren’t random events. They signify a fundamental weakness of existing marketplaces, a weakness, as we’ll explain, that is tied to price. Because the flaw is linked to price, not all marketplaces are suffering. In fact, some markets, less reliant on price, are outright thriving.

Just about the time eBay and Yahoo got into trouble, a more recent Internet start-up, BlaBlaCar, was doing amazingly well. Founded in Europe by a young Frenchman bitten by the Internet bug during graduate studies at Stanford, BlaBlaCar, much like eBay, operates an online marketplace, albeit a highly specialized one. It is in the business of helping people share car rides by matching those offering a ride with those looking for one. And it does so very well, matching millions of riders every month and growing quickly. Whereas eBay’s original focus was on price-based auctions, BlaBlaCar’s marketplace offers participants rich data about each other, ranking details such as driver chattiness (hence its name), so users can easily search and identify the best matches for them, and downplaying the importance of price (ride-sharers can select price only within a limited range). BlaBlaCar’s ride-sharing market isn’t alone in using rich data. From Internet travel site Kayak to online investment company SigFig, to digital labor platform Upwork, more and more markets that use data to help participants find better matches are gaining traction and attracting attention.

In this book, we connect the dots between the difficulties faced by traditional online markets; the error of the stock market’s trusted pricing mechanism; and the rise of markets rich with data. We argue that a reboot of the market fueled by data will lead to a fundamental reconfiguration of our economy, one that will be arguably as momentous as the Industrial Revolution, reinventing capitalism as we know it.

The market is a tremendously successful social innovation. It’s a mechanism to help us divvy up scarce resources efficiently. That’s a simple statement—with enormous impact. Markets have enabled us to feed, clothe, and house most of 8 billion humans, and to greatly improve their life expectancy as well as life quality. Market transactions have long been social interactions, making them superbly well aligned with human nature. That’s why markets seem so natural to most of us and are so deeply ingrained in society’s fabric. They are the building blocks of our economy.

To do their magic, markets depend on the easy flow of data, and the ability of humans to translate this data into decisions—that’s how we transact on markets, where decision-making is decentralized. This is what makes markets robust and resilient, but it requires that everyone has easy access to comprehensive information about what’s available. Until recently, communicating such rich information in markets was difficult and costly. So we used a workaround and condensed all of this information into a single metric: price. And we conveyed that information with the help of money.

Price and money have proved to be an ingenious stopgap to mitigate a seemingly intractable challenge, and it worked—to a degree. But as information is compressed, details and nuance get lost, leading to suboptimal transactions. If we don’t fully know what is on offer or are misled by condensed information, we will choose badly. For millennia, we tolerated this inadequate solution, as no better alternative was available.

That’s changing. Soon, rich data will flow through markets comprehensively, swiftly, and at low cost. We’ll combine huge volumes of such data with machine learning and cutting-edge matching algorithms to create an adaptive system that can identify the best possible transaction partner on the market. It will be easy enough that we’ll do this even for seemingly straightforward transactions.

Suppose, for instance, you are looking for a new frying pan. An adaptive system, residing perhaps on your smartphone, accesses your past shopping data to gather that you bought a pan for induction cooktops last time, and also that you left a so-so review of it. Parsing the review, the system understands that the pan’s coating really matters to you, and that you favor a ceramic one (it also notes your preferred material for the grip). Equipped with these preferences, it then looks at online markets for optimal matches, even factoring in the carbon footprint of the delivery (because it knows how worried you are about that). It negotiates automatically with sellers, and because you are ready to pay by direct transfer it is able to get a discount. With a single tap, your transaction is complete.

It sounds seamless and simple—because it should be. It’s far faster and less painful than having to do the search yourself, but it also takes into account more variations and evaluates more offers than you would do. Neither does the system tire easily (as we humans do when searching for something offline or online), nor is it distracted in its decision advice by price, derailed by cognitive bias, or lured by clever marketing. Of course, we’ll still use money as a store of value, and price will still be valuable information; but no longer being focused on price broadens our perspective, yields better matches, a more efficient transaction, and, we believe, less trickery in the market.

Such decision-assistance systems based on data and machine learning will help us identify optimal matches in these data-rich markets, but we humans will retain the ultimate decision-making power and will decide how much or how little we delegate as we transact. That way we can happily have our decision-assistance system hail a ride for us, but when it comes to our next job, we’ll choose ourselves from among the employment options our data-driven advisers suggest.

Conventional markets have been highly useful, but they simply can’t compete with their data-driven kin. Data translates into too much of an improvement in transactions and efficiency. Data-rich markets finally deliver what markets, in theory, should always have been very good at—enabling optimal transactions—but because of informational constraints really weren’t.

The benefits of this momentous change will extend to every marketplace. We’ll see it in retail and travel, but also in banking and investment. Data-rich markets promise to greatly reduce the kind of irrational decision-making that led to Yahoo’s crazy stock price in 2014 and to diminish bubbles and other disasters of misinformation or erroneous decision-making that afflict traditional money-based markets. We have experienced the debilitating impact of such market disasters in the recent subprime mortgage crisis and in the 2001 burst of the dot-com bubble, but also in the countless calamities that have affected money-based markets over the past centuries. The promise of data-rich markets is not that we’ll eradicate these market failures completely, but that we’ll be able to greatly reduce their frequency and the resulting financial devastation.

Data-rich markets will reshape all kinds of markets, from energy markets, where built-in inefficiencies have lined the pockets of large utilities and deprived households of billions in savings, to transportation and logistics, and from labor markets to health care. Even in education, we can use markets fueled by data to better match teachers, pupils, and schools. The goal is the same for all data-rich markets: to go beyond “good enough” and aim for perfection, giving us not just more bang for the buck, but more satisfaction in the choices we make, and a more sustainable future for our planet.

THE KEY DIFFERENCE BETWEEN CONVENTIONAL MARKETS and data-rich ones is the role of information flowing through them, and how it gets translated into decisions. In data-rich markets, we no longer have to condense our preferences into price and can abandon the oversimplification that was necessary because of communicative and cognitive limits. This makes it possible to pair decentralized decision-making, with its valuable qualities of robustness and resilience, with much-improved transactional efficiency. To achieve data-richness, we need to reconfigure the flow and processing of data by market participants, an idea that was already suggested as far back as 1987. Massachusetts Institute of Technology (MIT) professor Thomas Malone and his colleagues foresaw “electronic markets,” but only recently have we achieved the technical progress to extend that early vision and bring it into full bloom.

One may assume that the advent of data-rich markets rests mainly on advances in data-processing capacity and network technology. After all, far more information permeates data-rich markets compared with conventional ones, and Internet bandwidth has been increasing steadily with no end in sight. Leading network technology providers such as Cisco suggest that growth rates in Internet traffic will continue to exceed 20 percent per year until at least 2021—a rate that when compounded over just a decade will add up to a staggering 500 percent upturn. Processing capacity has risen dramatically, too: we now measure our personal computer’s power in thousands of billions of calculations per second, and we still have room for improvement, even if that power may no longer be doubling every two years as it has in the past.

These are necessary developments toward data-rich markets, but they aren’t sufficient. What we need is to do things not just faster but to do them differently. In our data-rich future, it will matter less how fast we process information than how well and how deeply we do so. Even if we speed up the communication of price on traditional markets to milliseconds (as we have already done with high-frequency trading), we’d still be oversimplifying. Instead, we suggest that we need to put recent breakthroughs to use in three distinct areas: the standardized sharing of rich data about goods and preferences at low cost; an improved ability to identify matches along multiple dimensions; and a sophisticated yet easy-to-use way to comprehensively capture our preferences.

Just getting raw data isn’t enough; we need to know what it signifies, so that we don’t compare apples with oranges. With recent technical breakthroughs, we can do that far more easily than in the past. Just think of how far we have come in the ability to search our digital photos for concepts, such as people, beaches, or pets. What works for images in our photo collections can be applied to markets and can translate data into insights that inform our decision-making.

Identifying best matches is easy when we compare only by price; but as we look for matches along numerous dimensions, the process gets complex and messy, and humans easily get overwhelmed. We need smart algorithms to help us. Fortunately, here, too, substantial progress has been made in recent years. Finally, knowing exactly what we want isn’t easy. We may forget an important consideration or erroneously disregard it; for humans, it’s actually quite difficult to articulate our multifaceted needs in a simple, structured way. That’s the third area in which recent technical advancements matter. And today, adaptive systems can learn our preferences over time as they watch what we are doing and track our decisions.

In all three of these areas, highly evolved data analytics and advanced machine learning (or “artificial intelligence,” as it is often called) have fueled important progress. When combined, we have all the key building blocks of data-rich markets. Digital thought leaders and energetic online entrepreneurs are already taking note. There is a gold rush just around the corner, and it will soon be in full swing. It’s a rush toward data-rich markets that deliver ample efficiency dividends to their participants and offer to the providers a sizable chunk of the total transaction volume.

The digital innovations of the last two decades are finally beginning to alter the foundations of our economy. Some companies have already set their sights on data-rich markets and put the necessary pieces in place. As eBay celebrated its twentieth anniversary and pondered its future, its new CEO announced a highly ambitious, multiyear crash program and forged a number of key acquisitions. The aim is to greatly improve the flow of rich information on the marketplace at all levels, to ease discovery of matches, and to assist eBay users in their transaction decisions.

EBay is not alone. From retail behemoth Amazon and niche players, such as BlaBlaCar, to talent markets, marketplaces are reconfiguring themselves and pushing into a data-rich future. Because data-rich markets are so much better at helping us get what we need, we’ll use them a lot more than traditional markets, further fueling the shift from conventional markets to data-rich ones. But the impact of data-rich markets is far larger, the consequences far bigger.

MARKETS AREN’T JUST FACILITATING TRANSACTIONS. When we interact on markets, we coordinate with each other and achieve beyond our individual abilities. By reconfiguring markets and making them data rich, we shape human coordination more generally. If done well, market-driven coordination greased by rich data will allow us to meet vexing challenges and work toward sustainable solutions, from enhancing education to improving health care and addressing climate change. Gaining the ability to better coordinate human activity is a big deal.

This will have repercussions for more conventional ways of coordinating our activities. Among them, the most well known and best studied is the firm. The stories we usually tell about firms are about vicious competition between them, whether it is General Motors versus Ford, Boeing versus Airbus, CNN versus Fox News, Nike versus Adidas, Apple versus Google, or Baidu versus Tencent. We love tales about individual battles that bloodied one of the contestants and advanced the position of the other. Entire libraries of business books and hundreds of business-school cases are dedicated to chronicling and analyzing these epic battles. But rather than battles between firms, we now see a more general shift from firms to markets, as the market, thanks to data, gets so much better at what it does. This shift doesn’t mean the end of the firm, but it represents its most formidable challenge in many decades.

Responding to the rise of data-rich markets isn’t going to be simple. If firms could utilize the technical breakthroughs we describe, reshape the flow of information within them, and capture similar efficiency gains, it would be straightforward. Alas, as we’ll explain, the technical advances that underlie and power data-richness can’t be used as easily in firms as they can in markets. They are constrained by the way information flows in firms. To adapt, the nature of the firm will need to be reimagined.

Possible responses to the challenge from data-rich markets involve finding ways to either more narrowly complement or emulate them. Firms might automate decision-making of (certain) managerial decisions and introduce more marketlike features, such as decentralized information flows and transaction-matching. These strategies offer medium-term advantages, and they are being adopted in a growing number of companies. They are useful for ensuring the continuing existence of firms in the medium term (although they bring their own set of weaknesses), but they are unlikely in the long run to stop the slide of the firm’s relevance in organizing human activity.

Just as firms will continue to have some, albeit diminished, role to play in the economy, in the future we’ll also still use money, but in data-rich markets money will no longer play first violin. As a result, banks and other financial intermediaries will need to refocus their business models. And they are going to need to move quickly, as a new breed of data-driven financial technology companies, the so-called fintechs, are embracing data-rich markets and challenge the conventional financial services sector. It is easy to see how banking will be severely affected by the decline of money, but the implications are larger, and more profound. At least in part, the role of finance capital rests on the informational function it plays in the economy. But as data takes over from money, capital no longer provides as strong a signal of trust and confidence as it currently does, undermining the belief that capital equates with power that underlies the concept of finance capitalism. Data-richness enables us to disentangle markets and finance capital by furthering the one while depreciating the other. We are about to witness both the rather immediate reconfiguration of the banking and finance sector, and the later but more profound curbing of the role of money, shifting our economy from finance to data capitalism.

DATA-DRIVEN MARKETS OFFER SUCH COMPELLING ADVANTAGES over traditional, money-based ones that their advent is assured. But they are not without shortcomings of their own. The fundamental problem is the reliance on data and machine learning and the lack of diversity of data and algorithms. These make them particularly vulnerable to troubling concentration as well as systemic failure. Because of this structural weakness (which we’ll explain further), data-rich markets could turn into enticing targets for ruthless companies and radical governments to not only cripple the economy but also undermine democracy. To mitigate this vulnerability, we propose an innovative regulatory measure. A progressive data-sharing mandate would ensure a comprehensive but differentiated access to feedback data and would maintain choice and diversity in decision assistance. It’s not only the antitrust measure of the data age, but it also guards against far bigger and more sinister developments that could threaten society.

The rise of a market in which a substantial part of the transactional process is automated, and the decline of the firm as the dominating organizational structure to organize human activity efficiently will uproot labor markets around the world. Nations will face the need to respond to this profound shift in the economy as it endangers many millions of jobs, fuels widespread worries in countless nations, and is already driving populist political movements. As we’ll detail, many of the conventional policy measures at our disposal are unfortunately no longer effective.

A shift from finance to data capitalism will question many long-held beliefs, such as work as a standardized bundle of duties and benefits. Breaking up this bundle is going to be a challenging but necessary strategy for firms looking for the right human talent, and for societies worried about mass unemployment, to bring back to employees jobs as well as meaning and purpose. Central to the changes we’ll witness in labor markets is data. Comprehensive and rich data flows drive the revival of the market and the decline of firms and money, prompting massive upheavals in the labor market. By the same token, rich data also enables us to upgrade labor markets so that they’ll offer far more individualized and satisfying work far more easily and more frequently than before (although, as we explain, this will need to be supported by innovative policy measures).

From the early days of money-based markets, critics have pointed at the gap between the idea of choice, so fundamental in markets, and the actual cognitive limitations that constrain our ability to choose well. For centuries, two antagonistic views have been pitted against each other: one side has advocated for a central authority to take over decision-making in markets from vulnerable humans, while the other has defended conventional markets, and with that the concept of decentralized information flows and decision-making, arguing that crippled individual choice was far better than none. These arguments were often stark—painted in black or white.

Over the last decades, a kind of truce has taken hold around the world, an acceptance that money-based markets work, but only with the appropriate regulations in place (and with no consensus on what “appropriate” entails). The compromise is that even though we can’t overcome the cognitive constraints that lead to erroneous decisions, we can put in place rules and processes that help mitigate their most negative effects. This was pragmatic, given the realities that hold sway on money-based markets, and the absence of a more enticing, workable alternative. But it was also an acceptance of defeat; real progress in improving the inner working of the market seemed forever illusive. The market was tainted, but the alternatives were worse. So, we learned to live with it.

The availability of rich data and recent technical breakthroughs mean that we now can move beyond money-based markets to data-rich ones and overcome some of the key informational and decisional constraints that we have been grappling with. The vision is ambitious. Rather than making for better mitigation of the conventional market’s weaknesses, we are about to see a rewiring of the market that renders mitigation far less necessary. In the future, data-rich markets will offer individual choice without the constraints of inescapable cognitive limitations.

Of course, we won’t be able to overcome all human biases and decision flaws (nor avoid savvy marketers exploiting them); even if humans choose to use smart machine learning systems on data-rich markets, that choice will still be a human one to make. When we empower ourselves to choose, we also retain that human error. Even rich data markets won’t be perfect; but pragmatically, they will be far superior to what we have today. We may still err, but we’ll surely err less frequently. Data-rich markets will change the role of markets and money, and question well-worn concepts, from competitiveness and employment all the way to finance capitalism itself. Because they will readjust the role of markets in coordinating human activities, they will have a huge impact on how we live and work with each other.

Some may fret over the role retained for human beings—that of the ultimate decision maker—and hope for a more rational central decision authority to take over. But we are convinced that keeping this fundamental role for humans isn’t a bug; it’s a feature. With the crucially important and valuable push for efficiency, sustainability, and rationality (because we really do need to improve our decision-making!), we must never forget the need to preserve and even embrace what makes us human. The ultimate goal of data-rich markets is not overall perfection but individual fulfillment, and that means celebrating the individuality, diversity, and occasional craziness that is so quintessentially human.