Two years ago the country’s top technology officer pitched President Obama a bold idea. Todd Park had been mulling over how to inject some Silicon Valley zing into the federal bureaucracy. The answer came at a floating monthly breakfast he attended with about 50 of the administration’s tech-savviest operatives—an unofficial group that calls itself the Innovation Cohort. Park had notched some wins for the White House by bringing in heavy hitters from outside government to handle discrete challenges. Why not scale that up?
Park—along with a deputy, a Harvard-educated former minor-league infielder named John Paul Farmer—proposed assembling a band of tech-industry standouts to tackle problems in the executive branch. Recruits would act with surgical precision, with one or two deployed per project and a mandate to achieve it within six months.
It turned out to be an easy sell. The President, after all, had won office promising to move the country past the old ideological fight over the size of government toward a more efficient system. With Obama’s sign-off, the Presidential Innovation Fellows—or PIFs, as they’ve become known around the White House—were born.
That spring Park, Farmer, and the country’s chief information officer, Steven VanRoekel, traveled to the TechCrunch Disrupt conference in New York City to enlist talent. Pacing an empty stage, they described the first projects: giving more patients online access to their medical records, enabling aid workers in Afghanistan to deliver payments by mobile phone, and easing the way for smaller tech companies to compete for lower-cost federal contracts. “We are here to formally launch an effort to disrupt the United States federal government,” Park told the crowd.
Nearly 700 people applied; the White House chose 18, including a Yale Law School lecturer, the director of innovation at Aetna, and a serial entrepreneur who had just sold a payment-processing company to Visa for $190 million. Before the PIFs got started, they went through half a week of training alongside the civil servants they were about to partner with. They learned about the “lean startup”—a Silicon Valley approach calling for experiment-driven product development—and “design thinking,” another model for testing new ideas, developed by the design consultancy IDEO. “As innovators, we always favor more flexibility in developing solutions,” says Garren Givens, a first-round PIF who now directs the program. “The problem with laws is they aren’t always agile.”
That tech procurement reform was included in the first round of projects soon proved prophetic. In October 2013, about halfway through the second round, a massive tech procurement failure spawned the federal insurance-exchange website HealthCare.gov, a bug-ridden, anger-inducing disaster that threatened to sink Obama’s greatest legacy. The fiasco was like a Times Square billboard boldly advertising everything wrong with the old way of doing things: Arcane contracting weeded out startups in favor of entrenched players, who then followed established practices that eschewed the testing the PIFs had learned specifically to prevent a meltdown. “It solidified why it was so important to have a standing SWAT team,” Farmer says.
Obama asked Park to organize the emergency response to salvage the web portal for the new health exchange, and Park in turn tapped four PIFs—two alums and two current fellows—to pitch in on the rescue. One alumnus, Greg Gershman, recalls the triage as “a once-in-a-lifetime experience.” Still, Gershman is frustrated. Another project he worked on as a PIF—an effort called MyUSA, which aims to make it easier for citizens to interact online with the federal government—remains shelved two years after he developed a working prototype; it’s partly a victim of the Paperwork Reduction Act. The 1995 law, intended to keep government from overburdening people and businesses with information requests, is now hamstringing technologists like the PIFs, who want to run pilot tests that solicit public feedback for their innovations before they tweak and expand them, a.k.a. that whole lean-startup thing.
Clay Johnson, the PIF who tackled the small-bore tech procurement reform project, says it is “America’s worst law.” He blames it and other bureaucratic hurdles for thwarting the broad adoption of an online-bidding process he developed to save the feds an estimated $24 billion a year on tech spending, more than NASA’s annual budget. Johnson was once the digital guru of Howard Dean’s 2004 presidential campaign, but he now says the small-vs.-big-government debate that consumes our politics misses the point: The bureaucracy’s inability to modernize is “the government’s climate change,” he says. “If we don’t fix it soon, it will mean the end of government as we know it.”
White House officials might not frame it so darkly, but they acknowledge the challenge. Park points to a pair of fresh initiatives designed to address the barriers that exasperated the first class of PIFs. In April the administration launched a permanent corps of technologists called 18F. The idea is to expand the ranks of those toiling to wrench the federal government into the 21st century and recognize that many of the projects will take more than six months to execute. More than half of the PIFs from the program’s second round have decided to stay on the job by moving to 18F. And in August the White House announced a new arm, dubbed the U.S. Digital Service and helmed by a former Google engineer, to clear a path through cobwebs of outdated policies so that the projects can advance. Park, meanwhile, will leave his post in September to move back west as a White House adviser in Silicon Valley, tasked with recruiting talent into government.
The test begins this month when the third round of PIFs begin. Among their projects: Make the federal trove of data on weather and climate easier for entrepreneurs to use, create a platform to crowdsource the digitization of 12 billion pages of government records stored in the national archives, and coordinate a challenge to identify asteroids that threaten earth.
Obama’s first presidential campaign envisioned an “iPod government” that was as simple, efficient, and user-friendly as the breakthrough Apple device. The HealthCare.gov debacle seemingly dashed that dream, but the PIFs and their sponsors are racing to restore it.
This story is from the September 22, 2014 issue of Fortune.