On April 21, 2021, in a rare moment of unity in Washington D.C., the White House endorsed a bipartisan bill known as the Endless Frontier Act that would commit $100 billion in funding over the next five years into research intended to help maintain US technological leadership over China.

In a country that over the last three years has seen two impeachment trials of a president, an attempt to overturn the results of a presidential election, and a violent riot at the Capitol, this support from both Democrats and Republicans is no small feat. We live in a time, after all, where politicians on the left and right can’t seem to agree on the causes or dangers of climate change (around which there is a clear scientific consensus), or whether face masks can help slow the spread of Covid-19 (they indeed can, according to the peer-reviewed scientific literature), yet concerns about being overtaken by China in science and technology have brought the two political parties together, even if briefly.

The Endless Frontier Act, named after the influential report written in 1945 for President Franklin D. Roosevelt, would put that $100 billion into a new directorate in the National Science Foundation modelled after the storied Defense Advanced Research Projects Agency, the Pentagon department credited with developing the Internet, driverless cars, and drones. The new directorate, according to the proposed legislation, would be focused on key technologies, ranging from artificial intelligence to synthetic biology — all areas where China has excelled in recent years.

The Endless Frontier Act comes amid a global pandemic that as of April 2021 has already claimed 3 million lives, while spurring some of the most accelerated scientific progress in recent memory by way of new vaccines to combat the virus. Particularly in the United States, Operation Warp Speed, the government programme that supported several vaccine candidates now being injected in people’s arms every day, will likely go down in history as an example of extraordinarily successful state-assisted innovation.

The Endless Frontier Act harkens back to an earlier era of ambitious government expansion into the sciences. After the Second World War, the government became the largest supporter of science in the United States, giving it the ability to create entirely new fields of research, such as computer science, and heavily influence the direction of existing fields, such as physics.

Today, however, in many areas of research, whether artificial intelligence or biotechnology, government spending is often overshadowed by private sector funding, whether through venture capital or other forms of investment. Even space exploration, once the exclusive domain of government, is facing competition from private industry.

More than 75 years after the original ‘Endless Frontier’ report, the landscape of government-supported science and technology has shifted drastically, particularly in the United States, and governments have given little thought to how to adapt their approach. Today, not only has the US government largely backed away from some of its most ambitious post-war ideas for sponsoring basic science, but it has also increasingly latched onto a model of innovation — such as the idea of high-risk ‘moonshots’  —  that fundamentally misunderstands the history of government-sponsored innovation and what is required to achieve significant breakthroughs. Some government-sponsored programmes, like Operation Warp Speed, will still be successful, but it’s likely many others without clearly defined goals will fail. Understanding what works and why could be the key to avoiding some of the costliest failures.

The origins of ‘The Endless Frontier’ report date to November 17, 1944, when President Roosevelt wrote to Vannevar Bush, the head of the wartime Office of Scientific Research and Development, asking how the country could take advantage of the military innovations being developed so they could be ‘profitably employed in times of peace.’

The president’s vision was broad — focused on the betterment of the country and its people  — not necessarily on staying ahead of other countries or national security. ‘The information, the techniques, and the research experience developed by the Office of Scientific Research and Development and by the thousands of scientists in the universities and in private industry, should be used in the days of peace ahead for the improvement of the national health, the creation of new enterprises bringing new jobs, and the betterment of the national standard of living,’ the president wrote.

In July 1945, just weeks before the United States dropped atomic bombs (the most significant and secret of those wartime technologies) on Nagasaki and Hiroshima, Bush delivered his final report to the president. In it, he spelled out a vision that encompassed everything from health to employment.  ‘Progress in the war against disease depends upon a flow of new scientific knowledge. New products, new industries, and more jobs require continuous additions to knowledge of the laws of nature, and the application of that knowledge to practical purposes,’ Bush wrote. ‘Similarly, our defense against aggression demands new knowledge so that we can develop new and improved weapons. This essential, new knowledge can be obtained only through basic scientific research.’

Bush’s report laid out a bold plan for the federal government to take a leading role in funding basic research. Scientists would lead this effort, rather than bureaucrats or politicians, ensuring the focus on the expansion of knowledge rather than short-term policy imperatives. Even if its most ambitious recommendations were never fulfilled, Bush’s vision of government patronage for the sciences set the stage for much of the postwar order. ‘The Endless Frontier’ led to a fundamental rethinking of the relationship between the government, science, and technological innovation. ‘For science to serve as a powerful factor in our national welfare, applied research both in Government and in industry must be vigorous,’ Bush wrote.

Following Bush’s advice, the National Science Foundation was established in 1950, signalling the government’s permanent role in furthering research. Less than a decade later, however, another event would have a profound impact on science policy. In October 1957, the Soviet Union launched Sputnik, the world’s first artificial satellite, beating the United States to space. While Sputnik was not a complete shock to the scientific community, which had been watching Soviet progress in rockets, the launch created a political opportunity for critics of President Dwight D. Eisenhower. It also highlighted some of the shortcomings in the US government’s approach to space technology, which lacked a central authority.

A few months later, President Eisenhower authorised the creation of the Advanced Research Projects Agency (it later became known as DARPA, when defense was added to its title), which would consolidate and accelerate the nation’s rocket programmes. The agency was also given a broad remit to ‘to direct such research and development projects being performed within the Department of Defense as the Secretary of Defense may designate.’ In other words, the agency could work in any area of science that the Pentagon’s leadership wanted, cutting through red tape to quickly start research and advance new technologies. And that’s precisely what it did in a surprisingly broad range of fields.

Within a few years of being founded, DARPA began funding computer science at universities and labs across the country, helping to lay the foundations for an entirely new field of research. Within a decade this research led to a nascent proposal for a computer network known as the ARPANET, the predecessor to the modern internet.

Over the next few decades, DARPA managed to churn out a series of notable innovations, some useful primarily to the military, such as stealth aircraft, but others, including voice recognition, driverless cars, and unmanned aircraft, with broad commercial applications. Its pursuit of high-risk, high-payoff research became a model that by the 2000s the government tried to replicate elsewhere, including in the Energy Department, Homeland Security and the Office of National Intelligence. In most cases, those new agencies stumbled.

Even so, the attempts to replicate the DARPA model continue. Most recently, President Biden on April 29 suggested to a joint session of Congress a new DARPA for health. ‘The National Institutes of Health, the N.I.H, I believe, should create a similar advanced research projects agency for health,’ he said. ‘And that would — here’s what it would do: It would have a singular purpose, to develop breakthroughs to prevent, detect and treat diseases like Alzheimer’s, diabetes and cancer.’

More than a decade ago, amid several of the already faltering attempts, I asked one of DARPA’s former directors, Stephen Lukasik, if there was any appropriate field for creating a DARPA-like agency. He mused that one obvious possibility would be an agency to address climate change. Scientifically, it would be the right problem to look at, but, he cautioned, there were already too many entrenched interests in place, such as the fossil fuel industry.

The problem is not just scientific, but economic and political, and those are factors that a science agency is ill-equipped to address. ‘In the case of space we had a national problem with no existing structure,’ he said, explaining DARPA’s initial success. ‘As soon as you say, “What about energy? Healthcare? Homeland security?” You have a huge structure.’

The best example, he suggested half facetiously, would be an agency dedicated to protecting the earth from being struck by an asteroid. ‘We could pull off a DARPA for that, because there isn’t an anti-asteroid infrastructure in place,’ he said. ‘In general, the threat would be universally understood, the impact would be universally disastrous, and it would be scientifically challenging.’

Imagining a mad scramble of government-funded scientists working on technologies to divert an asteroid impact may sound far-fetched (in part, because it is the plot device for several Hollywood films), but so too was the idea of a global pandemic prior to 2020.

When the first cases of the Covid-19, the disease caused by the novel coronavirus, were identified in the United States in January 2020, the government was unprepared on almost every level. The National Strategic Stockpile, which included N-95 respirators, didn’t have anywhere near the inventories needed to deal with a pandemic, despite years of warning from scientists that one was coming. There were scientific stumbles as well, demonstrated by the initially botched attempt by the Centers for Disease Control and Prevention to develop its own Covid-19 test. In another sign of the ill-prepared times, when the pandemic hit the United States, the US Army Medical Research Institute of Infectious Diseases at Fort Detrick, the military lab responsible for defending against biological threats, was closed due to safety violations.

Amazingly, amid an otherwise chaotic response to the pandemic, the Trump Administration initiated Operation Warp Speed, a $10 billion programme to accelerate development and production of vaccines for Covid-19. Even as the United States lagged behind in medical supplies, testing, and other efforts to contain the spread of the virus, the public-private partnership was by any measure a stunning success, leading to several vaccine candidates approved by the Food and Drug Administration in under a year. As a result, the United States, which had one of the worst track records for controlling the spread of Covid-19, quickly became one of the top countries for progress on vaccinations.

That leads to a critical question: Why did Operation Warp Speed succeed and what lessons does that offer for future government-sponsored programmes? One of the key aspects of the operation, similar to other successful government-sponsored research efforts at DARPA, was that it had a clear customer. The government wasn’t just sponsoring the research, it was also the customer for the product, ultimately buying hundreds of millions of doses of the vaccines from the participating companies, while also smoothing the path for vaccine approval through a fast-track mechanism known as emergency use authorisation.

By contrast, progress with vaccinations in Europe has been rocky, in part because there was no central acquisition authority. While the United States has faced hiccups in the distribution and administration of the vaccine, which is largely handled by the state and local authorities, the federal government’s purchase of the vaccines helped accelerate overall progress.

The pandemic has demonstrated that the federal government still can bring resources to bear to help solve an urgent scientific problem rapidly. Without Operation Warp Speed, pharmaceutical companies would still have moved forward, but it’s doubtful that Moderna, which had never previously brought a drug to market, would have produced a vaccine in under a year without the nearly half a billion dollar federal government investment the company received. And while Pfizer didn’t take federal money for development of its vaccine, it did benefit from an advance purchase order under Operation Warp Speed, without which, it likely wouldn’t have moved as quickly as it did. Having a guaranteed market for an innovation, thanks to a government purchase, can be as valuable as research dollars.

In fact, one thing that Operation Warp Speed should teach us is that the ability of the government to direct and accelerate innovation depends in large part on its ability to guarantee a readily available market for technologies. In the case of the Covid-19 vaccine, the government’s investment in development was important, but the guarantee of a large market — i.e. the entire American population — was even more critical.

The success of Operation Warp Speed thus raises a critical question for the Endless Frontier Act. Even if the government funnels $100 billion into new research, who will be the customer for the resulting technologies? Possibly the military, or the private sector, but there’s no guarantee they will be interested or have a specific need for the innovations that come out of this research. Any government research manager can talk for hours about the ‘valley of death’ for innovations that emerge from academia and labs only to die on the vine.

Providing more funding for science and technology, as the Endless Frontier Act proposes to do, is laudable and will have some benefits for the country, but the question is whether the government has the political will and patience for the type of long-term commitment that requires. And without a clear customer for the scientific and technological innovations this new directorate would fund, it’s unlikely to produce the type of breakthroughs that lawmakers typically expect to maintain steady funding over many years.

Perhaps most fundamentally, the idea of making this directorate a ‘new DARPA’ should not inspire confidence in the results, given the series of mostly forgettable ‘DARPAs’ for energy and homeland security. One reason for those lacklustre results is that the new agencies have never enjoyed the independence or funding the original DARPA enjoyed. More fundamental, however, is that those attempts to replicate DARPA never had a clear customer, like the Pentagon, with its now more than $700 billion annual budget and central acquisition authority. Who was going to buy the innovations produced by the energy ARPA? Certainly not the Department of Energy. Private industry possibly, but only if those innovations are immediately profitable, which is usually not the case. Similarly, the Department of Homeland Security, which is made up of myriad agencies, has no central authority for purchasing technology.

If the goal of the new Endless Frontier legislation were simply to reinvigorate the country’s science base, that would be laudable and perhaps sensible. But the gulf between ‘The Endless Frontier’ report of 1945 and the Endless Frontier Act of 2021 is the gulf between Vannevar Bush’s vision of broad-based scientific support that would advance the nation’s welfare, and a bill inspired mainly by competition with China. To bridge that gap will require political leadership that can reimagine a relationship between the state and science that fits the realities of the 21st century.