Hack Your Bureaucracy

TALK TO REAL PEOPLE

Get Out of the Office

How do you persuade President Obama’s senior advisors to change their minds? Lisa Gelobter, the new chief digital services officer at the US Department of Education, knew that pushing back against the West Wing wasn’t going to be easy. As a former product and engineering executive at BET Networks, Hulu, and NBC, Lisa had a lot of experience building compelling digital products. And she knew that to be successful at her mission, she would need to understand the behavior of people not usually represented in the West Wing: students.

Lisa and her team were building the College Scorecard, a second-term addition to President Obama’s array of initiatives aimed at decreasing the cost of college, like increasing Pell grants and making financial aid forms easier to use. The president was rankled by the fact that the country’s most popular college ranking system, published by U.S. News and World Report, values what he considered to be the wrong things, like selectivity (the fewer students an institution accepts, the higher the ranking) and facilities (creating incentives to build new climbing walls and gyms, rather than invest in instructor training).

President Obama wanted a scorecard that would compete with it, but one that would give students and their families the same kind of “value for your money” information that you can find when you are buying, say, a new car or a new refrigerator. Colleges with low graduation rates or whose graduates didn’t earn enough to repay their loans would have to disclose that information, presumably causing students looking for good value to give other, higher-performing schools a second look.

It turns out that using the federal government to rank seven thousand post-secondary institutions is trickier than it seems. How do you score the value of degrees from seminaries that are training clergy members who will never seek to earn much money? What about liberal arts degrees, which may ripen into lucrative careers over time, but not at first? What about schools that are serving students from marginalized areas with low-performing high schools? Rather than rank schools against each other, President Obama focused his team on presenting data that would allow students and their families to be better informed consumers.

The College Scorecard, as the initiative became known, is both an official US government website for students to learn about college costs and an effort to make the raw data available to the companies and nonprofits that help Americans with college choice—including other ranking systems. Arguably, getting the data into the hands of thousands of companies, nonprofits, and college counseling services could reach more people than an official government website alone. But the website was something that the president and senior West Wing officials could tangibly understand—and make suggestions about.

The original website was launched in February 2013 as part of President Obama’s State of the Union speech. And it was a big flop. According to the New York Times in 2013:

Further, and perhaps more critically, the website lacked the ability to show how much recent graduates of each college were making—an important feature if you are trying to show a prospective student’s “bang for the buck.” This was an innovation that the administration had been very excited about: using earnings data available from the Department of the Treasury and applying it to show how much students who attended particular universities actually earned a few years after graduating. But it was proving difficult to effectively use this earnings data alongside the other information the Department of Education was already making available. That’s why West Wing officials asked Lisa to help.

Lisa and her team at the Department of Education were sensitive to the fact that they weren’t starting from scratch. They did their best to get to know their counterparts across the government who had already been working on this project. But they also knew they would have to get out of the office and talk to real people to make the next version of the tool meaningfully better. Instead of focusing solely on untangling the knots tying up the project internally, Lisa’s team took a fresh approach to the scorecard, asking what it would take to achieve its original purpose of being useful to students and their families as they made big investments in their futures.

With paper drawing prototypes—some made from cardboard salvaged from the trash—Lisa and her team walked around the Mall in front of the US Capitol to chat with college-bound students and their parents. After seeing the prototypes of a mobile website, students gave rapid, in-person feedback. Armed with the data from actual students—who weren’t shy about saying what made sense and what didn’t—Lisa was able to show President Obama’s senior advisors insights like their idea for a “compare” button (something that some of them really wanted) wasn’t a feature students would actually use, especially on the limited real estate of a mobile phone screen. President Obama’s team—not to mention the New York Times—genuinely thought that the ability to compare schools would be a useful feature, but it turns out it didn’t have any value to the students.

This insight didn’t require fancy design or coding skills—the prototypes were literally hand-drawn on leftover cardboard. What it did require was getting out of the office and speaking with the actual target customers to find answers.

As a result of their research, Lisa and her team launched an updated College Scorecard as a mobile website with features that met the needs of actual students, rather than what senior White House officials thought students wanted. The team also persuaded the West Wing to release the data as an application programming interface (API), to make the data available to ranking systems, college counselors, and anyone else who wanted to use the data.

The updated website and underlying data were finally making an impact. Three years later, the New York Times was singing a different tune:

Look Beyond Your Organization

Great companies are obsessed with their customers. They know they need to deeply understand their customers’ needs and desires if they want to have any chance of winning in a hypercompetitive market. Sometimes a company founder has intimately lived the problem she is trying to solve; other times, the founder is good (or gets good quickly) at talking to as many stakeholders as possible to understand their problems. Over time, companies employ people with job titles like “designer,” “user experience researcher,” and “product manager,” all of whom are laser-focused on understanding customer behavior and attitudes.

But large bureaucracies—like government agencies, nonprofits, schools, and even some companies—are not necessarily designed to meaningfully listen to and learn from their end users. When they do, it’s often in a very structured way, such as focus groups, surveys, or commissioned market research. In your organization, it might be the role of the marketing department to listen to prospective customers, and the role of the customer service department to listen to existing customers. These silos can prevent your organization from developing an accurate, holistic view of the real end user experience.

At Harvard, after his White House experience, Nick designed and taught a field class that forced students to get out of the ivory tower and listen to real people in their community. The student teams worked with government clients, like the City of Boston, to understand a specific problem and conduct original field research—doing ride-alongs with cops, interviewing voters in laundromats, talking to parents in libraries, and observing city workers struggling to fill out forms. While the students often hesitated, spending too much time planning these visits, Nick would insist, friendly but firmly, that they leave campus and embed themselves within their target population.

As a result, the students formed much richer understandings of the problem area they were tasked with solving. One student, Berkeley Brown, who spent a considerable amount of time with the Boston Police Gang Unit, observed how valuable it was to escape the “Cambridge bubble” and how her previous assumptions about how the police used technology—and what really mattered—were quickly shattered by spending time with two detectives, John and Brian. She noted:

Berkeley also admitted, in her post-class reflection paper, that coming back to Cambridge could be a bit culturally jarring; back in Harvard Square after a several-hour ride-along with the gang unit, she overheard her fellow students debating the pros and cons of taking Dutch versus Swedish as a foreign language, and using phrases like “a priori reasoning.” Berkeley believed she was much wiser from having spent real time in the field, beyond the classroom.

Marina’s first foray into foster care after working at the VA was thanks to an afternoon of talking to real people. She was invited to spend an afternoon in Rhode Island by her former boss Richard Culatta, who had recently become the state’s chief innovation officer. The state was interested in tapping her technical expertise to recruit more foster parents, who were needed badly. Knowing how valuable talking to real people had been to her at the VA, the first thing Marina did was ask to have coffee with anyone who had dropped out of becoming a foster parent. It seemed easier to retain current applicants than to recruit brand-new ones, so she wanted to find out: Why had they given up?

After a few coffees, the answers were all the same: potential foster parents shared that they would call asking about the next steps in the licensing process, but the employee answering the phone often didn’t know and didn’t call them back. If the agency was this disorganized when it was recruiting them, the families thought, how would they support them if they had a challenge with a child in their home? This broke trust, and they dropped out.

This insight ended up being good news: it revealed a relatively easy fix that the agency had not yet considered, because it was not talking to the families that had dropped out. Marina shared the feedback with the state team, which quickly responded by making a shared spreadsheet that let everyone in the office see each family’s progress as they worked through a series of required steps toward becoming licensed. This made it easy for anyone answering the phone to tell people where they stood in the process. Was this spreadsheet the ultimate solution? Of course not. A couple years later, the state implemented an online foster parent licensing portal from Binti, a software vendor focused on foster care, that let families log on to see their own progress. But that initial spreadsheet let the team get started and make immediate progress toward improving the foster parent experience. Today, Rhode Island has one of the highest foster parent licensing rates in the country.

Get Outside the Office—Literally

Once you’re inside an organization, you can’t have a truly outside perspective. No matter if you’ve been there for ten minutes or ten years, you can counter this by going outside.

One of Marina’s most impactful experiences was meeting a woman on a bench at the Menlo Park VA Hospital. Marina, who is rather shy, was forced to sit on this bench all afternoon for an exercise in a design thinking workshop. She was instructed to keep her VA role to herself and simply make small talk with anyone who sat down. This woman seemed eager to talk. Due to her husband’s illnesses, she had essentially lived at the hospital for the last year. As she told her story, Marina racked up a growing mental list of all the VA benefits for which this woman qualified: pension, home loan, and college tuition for their oldest son, to name a few. Yet when asked what other benefits they used besides healthcare, the woman’s response stunned her: “What other benefits?”

The realization that it was possible to literally live at the VA for a year and not know about its eighty-two non-healthcare benefit lines was alarming; but it fueled Marina’s vision of a VA that could organize itself around a veteran and their family, instead of continuing to force veterans to organize themselves around the agency.

Going outside the building proved a repeatable success play at the VA. In the face of a mounting crisis in which veterans could not enroll in VA healthcare because over 800,000 paper applications had piled up in a warehouse, Mary Ann Brody, a new member of Marina’s team, sought answers in an unlikely place: the White House correspondence office. There, every letter sent to the president is meticulously saved and cataloged. Mary Ann pulled a letter from a homeless veteran, Dominic, who was asking for help in obtaining VA healthcare. She arranged to meet him and (with his permission) recorded their interaction, asking him to show her how he had tried to get VA healthcare in the past. Busting all stereotypes of homeless veterans, Dominic was young, personable, and tech-savvy—and he expertly narrated the seemingly impossible task of enrolling in VA healthcare, which he had unsuccessfully tried to do twelve times. The application experience was so bad, he likened it to trying to get through a door spiked with IEDs. After demonstrating that there was literally no way for him to enroll, Mary Ann had him try the team’s prototype for a new and simpler application. Dominic was able to complete it on the first try, declaring he would use it “over anything the VA would provide.”

This video took off like wildfire across the VA and the White House. The focus immediately shifted to: “How do we get this new online application into the hands of every veteran?” Within weeks, every VA employee was directed to point veterans at the new online form instead of paper. Most importantly, Dominic was enrolled in healthcare the very next morning. In a testament to its leadership, the VA even put Dominic’s videos online. Faced with his undeniable real-world experience, decision makers were newly motivated to pick up the pace of transformation. Over a million veterans have since enrolled in healthcare, all thanks to Mary Ann getting out of the building and finding Dominic. (You can watch Dominic’s videos on our website.)

Observing from a Distance

You should also find ways to indirectly observe your customers’ behavior. We aren’t proposing one-way mirrors or fancy surveillance systems, though these days there are lots of high-tech solutions to measure where people move in a store, where they linger on a web page, and how much time they spend making a decision. It often doesn’t take anything but a keen eye to spot customer dissatisfaction. In 2001, delivery driver Joe Perrone saw frustrated customers walk right past parked FedEx trucks while carrying packages to their nearest drop-off location. He suggested adding drop slots to trucks. Thanks to his idea, FedEx trucks effectively became mobile drop-off locations.

If you have an OXO Good Grips peeler in your kitchen, it’s because its designer, Sam Farber, went out and observed real people—starting with his wife. A lifelong entrepreneur, he was looking for a way to break into the kitchen goods space when he noticed his wife, who had arthritis, struggling to peel a carrot. With a bit of clay, he built a prototype—which he then refined by watching many volunteers from the American Arthritis Foundation try to use it. The resulting Good Grips peeler works well for people with joint problems, and also works really well for people who don’t have joint problems.

This technique isn’t limited to observing human users: Canadian officials struggled repeatedly to design raccoon-proof garbage bins until they took the time to observe real raccoons breaking into the bins at night. This ultimately led to a new bin design that the raccoons could not overcome.

Grounding your initiative in the context of real stories of real users (not to mention prospective and former customers) will give you more than survey data—it gives you a narrative that is easy to understand, and easy for others to repeat. Your strongest competitive edge will come from immersing yourself among real customers—especially if that isn’t part of your job description.

HOW CAN I USE THIS?