Russia’s drift away from partnership and cooperation with the West began well before Putin came to power. The main drivers of this development among the Russian bureaucracy and elites were the reaction to the Soviet collapse, the crisis of identity, fragility of Russian sovereignty, and ressentiment against the US-led international order.

In the late 1990s, anti-Americanism emerged as an important element in an emerging Russian identity. This was a polyvalent phenomenon, a ‘love-hate’ relationship that could turn on a dime, from admiration of, and desire to be accepted by, Washington to suspicion and alienation. The key question debated in Moscow was about the place for a new ‘democratic and market-centric’ Russia in the US-led liberal order. The supporters of the pro-Western course contended that Washington would help Russia to find such a place. The sceptics argued, instead, that the US strategy was to marginalise and weaken Russia.

Until 1998, however, the anti-US trend in Russian ruling circles and elites had not yet formed the basis for foreign policy. And there was large scope for both Western diplomacy and institutions to operate in Russia between 1991 and 1994. The US, NATO, and EU could have made Russia ‘an offer it could not resist’. But what kind of offer? It could have been a clear ‘road map’ for the Kremlin to win a respectable place in the international order – while gradually abandoning great power ambitions to be equal to the United States and have its own sphere of influence in Eurasia. The greatest opportunity for the United States to create a stable framework to influence Russia’s course was the Partnership for Peace (PfP), the programme pioneered by the Clinton Administration, and in particular by the Pentagon, in January 1994. Some, especially from today’s unfavourable vantage point, may doubt that this opportunity existed at all. Critics say that Russia’s chaotic internal dynamics and structural ‘path dependency’ played a primary role, severely limiting what the United States and other Western powers could do.

While structural impediments were strong, agency and contingency should not be dismissed. During the first years of Yeltsin’s presidency, Russian leadership was aimed passionately at Russia’s integration with the West, NATO, and other international liberal structures. In March 1992, when Yeltsin proposed to George H.W. Bush that they declare that the United States and Russia were no longer adversaries but allies, Bush was hesitant, feeling that the Russians wanted a special role in Eastern Europe.

Russia’s elites and bureaucracies had high expectations for what the United States could deliver and were prepared to go far to win American trust. There was a desire, even among Yeltsin’s critics, to impress the West with a readiness to build democracy and a market economy. American soft power in Russia was at its peak. Some Russian officials even regarded Bush and Secretary of State James Baker as umpires in their domestic political and bureaucratic rivalries.

In the Spring of 1991, as the Soviet Union was on its last legs, Grigory Yavlinsky and his Harvard associate Graham Allison formulated the challenge for the US government in Russia: a ‘postwar transformation without occupation’. Allison recommended a ‘deep engagement’, backed by the promise of tens of billions of dollars gradually disbursed in stages, in return for systemic and structural changes in Soviet economy and state practices. This was the idea of a big Western ‘map’ for reforming the Soviet Union into a confederation, backed by the US authority and assistance. The Bush administration considered this approach unrealistic. Instead, the White House focused on the peaceful roll-back of Soviet forces from East and Central Europe. Bush decided to ‘manage’ Gorbachev personally, while ‘deep engagement’ was downshifted to a technocratic engagement via IMF-World Bank experts. There was no political will in Washington to provide major assistance to a long-term geopolitical rival.

As history accelerated and the Soviet Union fell apart, US leaders focused on the most urgent issues, such as the future of nuclear weapons and humanitarian assistance. Important strategic choices were postponed or never made. A few in the Bush administration understood that, after the Soviet collapse, Ukraine could become a bone of contention between the West and an eventually resurgent Russia. Dick Cheney argued that Ukraine and the Baltic States might be a buffer zone against a rogue Russia. Baker pursued a more nuanced strategy: to contain Russian nationalism, but to engage and include Russia in the Western security architecture. Bush and Brent Scowcroft were sceptical but went along with Baker’s reasoning. Then came the change of American teams as Bush lost the presidential election to Bill Clinton in November 1992.

Clinton learned from Bush that he should act as ‘an air-traffic controller’ for the Russian President Yeltsin. This personal engagement was paralleled by a close relationship between Strobe Talbott, ambassador-at-large to the new independent states, and Russian foreign minister Andrei Kozyrev, between Al Gore, the US vice-president and prime minister Victor Chernomyrdin and a few other similar ‘partnerships’. Yet, the administration did not do much to help the core group of Russian reformers. Instead, Clinton decided to manage Yeltsin on a personal basis. The ‘Russia First’ policy of the administration became a ‘Yeltsin First’ policy. It was the most convenient, but perhaps not the best possible way of using enormous Western leverage on Russia to steer its elites towards reforms. Soon the limitations of such an approach became clear. The Russian leader jealously guarded his ‘monopoly’ on the US leadership and reacted with hostility when the Americans approached other politicians. This narrowed US capacity to influence other Russian politicians, whom Yeltsin viewed as rivals. Some Russian reformist veterans later reminisced that the ‘bromance’ between Clinton and Yeltsin was an obstacle to serious discussion of the US-Russian relationship and the issues standing in the way of a substantive partnership.

In early 1993, the Clinton Administration took Yeltsin’s side in his struggle against his multiplying enemies. The latter were lumped together by the Yeltsin camp and Western media as a ‘red brown’ coalition of communists and nationalists. The Supreme Soviet of the Russian Federation became, in Western eyes, a dangerous institution, a concentration of people who yearned to restore the Soviet Union and destroy ‘democratic reforms’. At the same time the much-heralded financial package to the struggling Russian government never went through, creating serious doubts in Russia about American goodwill. Some Russian reformers compared Clinton’s approach to ‘spinach-feeding’: Russians were told what to do by Western ‘teachers’, just like Latin American countries a decade earlier. There was no Western financial package to back those ‘lessons’ and no forgiveness of Russian debts (in contrasts to Poland).

Meanwhile, the main social-political base of pro-US sentiment in Russian society, the educated elites in Moscow, St. Petersburg, and other Russian cities, were weakened and demoralised by a traumatic transition to a market economy. As the year went on, the constitutional crisis flared up between Yeltsin and the Russian Supreme Soviet. American documents show that the United States was on Yeltsin’s side from the start, and encouraged him to get rid of the first Russian parliament in a peaceful way. Emboldened by this support, Yeltsin acted imprudently, crudely, and without adequate political preparations. This resulted in the confrontation in September that took on a life of its own and terminated in early October in an explosion of violence. The best way for ‘Russian democracy’ would have been to support mediation and the role of the Russian Supreme Court. Instead, because the events spiralled out of control, and Western leaders feared a Russian civil war, they stuck with Yeltsin, no matter what he did.

The outcome of the ‘October crisis’ of 1993 looked acceptable at the time but contained numerous longer-term dangers for Russia’s development in a liberal democratic direction. In the West, the biggest surprise at the time was the victory of Vladimir Zhirinovsky’s Liberal Democratic Party of Russia at the elections to the new Russian parliament that replaced the destroyed Supreme Soviet. The Zhirinovsky factor, however, became a decoy, which distracted attention from less visible but damaging consequences of the September-October crisis.

Yeltsin pushed through a new constitution which was legitimised by the national referendum. This constitution formally laid out checks and balances, yet in practice weakened them, giving a near monopoly on power to the Russian president. The authority of the Supreme Court was gravely undermined, and Russian society learned a lesson of political cynicism: power came from the barrel of a tank. The newly elected parliament included two chambers: the Council of Federation and the Duma that consisted of many autonomous and energetic local politicians. Yet the subsequent years demonstrated their inability to check the power of the president and express the interests of their constituents. Soon the Russian parliament came to be regarded by most Russian people as nothing but ‘a bunch of chatterers’. Increasingly, members of parliament came to depend on favours from the presidential apparatus. Those factors would later serve the foundation for a Putinist regime: only the personal will of the president now determined if Russia would move towards the West or away from it.

From the very start of Russia’s independent foreign policy at the end of 1991, Yeltsin and his reformist entourage wanted to make Russia a member of NATO. At the same time, Yeltsin viewed Russia as another superpower, which would develop a special partnership with the United States. This created all kinds of problems for Washington. Given the legacy of confrontation with Moscow, American leadership and exceptionalism, the sensibilities of smaller member-states, the Eurasian scope of Russian power, and Eastern European sensibilities, Russia’s membership was out of the question. It did not fit into any imagined future scenarios for the main Western security organisation. Yet Russia outside NATO could be a big problem as well. The Bush team left Washington without any strategies on how to square this circle.

An alternative path was proposed by the leaders of the Pentagon, Les Aspin and William Perry. Their 1993 plan, The Partnership for Peace (PfP) created a common security structure in partnership with NATO, consisting of the members of the former Warsaw Pact (minus East Germany) and post-Soviet states, including Russia and Ukraine. The plan aimed at avoiding the divisive effects of NATO expansion – inevitable given historical past and mutual grievances between Russia, Ukraine, and other Russia’s neighbours. This also was a step away from the ‘Ukraine as a buffer against Russia’ envisaged by Dick Cheney, Paul Wolfowitz, and others in the Pentagon until 1992.

It has not been fully clarified to this day why Clinton shifted from his initial support of this scheme to the ‘not whether but when’ course of NATO expansion. Briefly, the PfP and NATO enlargement were considered in the White House in parallel, but then the latter took off, while the former faded into obscurity. The existing literature puts the blame mostly on Russia’s domestic development: the fighting between the Russian parliament and Yeltsin in October 1993, the victory of Vladimir Zhirinovsky, and the brutal invasion of Chechnya in December 1994. These arguments, however, do not explain the causality chain behind Clinton’s choice. What is clear, however, is that the Poles, Balts, and other proponents of NATO expansion used Russian developments to tip the balance of discussion in Washington. Still, Clinton’s choice was crucial. Had the president backed the PfP option, the United States could have taken care of Eastern European concerns. There were no credible threats to Eastern European security from Russia at the time.

Then Russian foreign minister Andrei Kozyrev provides a different explanation for the origins of NATO enlargement. He claims that the main roadblock was the discussion of Russian special status as a privileged partner within the PfP. He recognised that it involved paying a political price Clinton was not prepared to pay. The Secretary of State Warren Christopher, according to Kozyrev’s memoirs, ‘was haunted by the fear of being accused by the Eastern European states and their expatriate populations in America of caving in to Moscow’s pressure at their expense’. According to the Russian perspective, American domestic politics prevailed over a carefully constructed strategy. The PfP had the support of many in the bureaucracies, but too many millions of voters favoured giving preferential security to Poland, Hungary, and the Czech Republic – and not to Russia. The NATO enlargement, unlike the PfP scheme, was promoted feverishly by Eastern Europeans, the American Republican right, and the Democratic liberal left. Clinton followed his political instincts: he knew he had won elections despite his lack of foreign policy strategies.

William Perry, the American secretary of defense, saw the victory of the Eastern Europeans in the NATO vs PfP debate as the taproot of US-Russia alienation. He recalled: ‘Russia expressed its objections to the proposed changes on its borders, but its views were ignored. As a result, Russia began to withdraw from its cooperative programs with NATO. The combination of the West failing to act during Russia’s financial crisis and ignoring their strongly held views on NATO expansion, reinforced a prevailing Russian belief that we didn’t take them seriously. Indeed, many in the West saw Russia only as the loser of the Cold War, not worthy of our respect.’

The decision to expand NATO was a product of complex political factors. Still, the enlargement derived more from contingency and ideology than a strategy. It was tempting to fill the vacuum as the last ex-Soviet (now Russian) troops were leaving European countries. The new course was an immediate success because it rested on the eagerness of Eastern European states to join and the inability of Russia to resist the process physically or conceptually. NATO’s ‘open door’ acquired a new ironclad justification 30 years later, in an era of Russian aggression. The champions of NATO enlargement argued in 1994 that Russia would do nothing to stop it; now when Putin responded with aggression, they say this justifies the course taken 30 years ago.

Yet the critics of this option at the time may say ‘we told you so’, that the current predicament is in part the price for the NATO ‘open door’ – with the exception of Russia. The NATO ‘open door’ made room for risky divisions and delineations that the PfP would have taken care of. The inclusion of Eastern Europeans and the denial of a security sphere to Russia eliminated the chance for the US to make an offer that Moscow could not resist, while leaving Russia in the status of the European security architecture’s loose cannon. The Russia-NATO Founding Act of 1997 was a well-meaning document, but it failed to square the circle. It left Russia without clear rights, responsibilities and incentives to cooperate, but not as part of the alliance and without any hope of joining. In the absence of a ‘map’ towards partnership and alliance, particularly after 1997, the balance in Russian domestic politics changed in favour of the sceptics, like Evgeny Primakov, who viewed Russia’s future as a power in a multipolar world, not in the US-led liberal order. The departure of Andrei Kozyrev and the ascent of Primakov to the levers of Russian foreign policy meant a comeback for those who believed that the West would always keep Russia in its crosshairs.

The enlargement of NATO into Eastern Europe and the Baltics created a potential intermediary zone between NATO and Russia, most importantly including Ukraine. That was a problem that George F. Kennan and other critics had in mind when they opposed this policy. The ‘open door’ course found its ideological justification in history and democratic ideology, but the strategic reality differed from lofty intentions. The American zone of security commitments and responsibilities included the lands where historical grievances and insecurities pointed against Russia, and thus denied the US room to be a fair mediator in Eastern Europe and the post-Soviet space. Gradually, perhaps inevitably, the new Eastern boundaries of NATO became a new frontier that had a dynamic of its own. What was not acknowledged fully, despite the heated debate, was that the inclusion of Poland and the Baltics over Russia would in time empower a vision of history in Eastern Europe, create a self-fulfilling cycle of mutual historical grievances, and increase temptation for Poland to lobby for decoupling Ukraine and Russia and include the former into NATO – to enhance the buffer against Poland’s historic enemy.

The momentum for enlargement grew out of sync with security considerations and a pragmatic strategy. It became an ideological mission. Ultimately, the ‘open policy’ decreased, rather than enhanced, strategic freedom for the White House, the State Department, and the Pentagon. The frontier between NATO and Russia was not meant to create a security dilemma, but it did, at the cost of other strategic priorities.

In December 1992, Kozyrev made a ‘mock’ speech at the OSCE meeting in Stockholm: he borrowed the line from the ‘moderate’ Russian opposition. He also published an article in which he presented a choice to Western partners: partnership with Russia or a Cold Peace. If a chance for Russia’s inclusion was missed, he argued, Russia would remain unanchored and drifting in confrontational direction. Kozyrev also recalls his message at his first meeting with the newly installed secretary of state, Warren Christopher: the US should be careful to treat Russia as a great power; if not, there would be a nationalist blowback. ‘The country’s public opinion would trade the Soviet-style confrontation with the West for a partnership but were not willing to abandon the status’ of a great power with a role in world affairs. According to the Russian foreign minister, he felt that US leaders ‘either missed the essence of what I had to say or did not care’.

This was probably the preparation for what would ultimately become a ‘blame game’ over an increasingly likely confrontation between Russia and the West. Years later, when Putin took an openly anti-US course, Kozyrev and other reformers were still trying to debate among themselves why their partnership with the West ended in a failure. They blamed Russian conditions, but also Western lack of trust in a democratic Russia. Yeltsin’s main liberal strategist in 1990-93, Gennady Burbulis, an ardent promoter of Russia’s integration into the West, including NATO membership, told this author that the collective West made a mistake by continuing to view a democratic Russia as a potentially dangerous competitor. Kozyrev’s memoirs are full of bitterness about this lost opportunity, particularly during the first two years of the Clinton presidency. His characterisations of Clinton and Talbott are unusually harsh. Without a visionary Western leadership, he said, ‘any Russian leap’ towards integration with the West ‘was doomed to fade out’. The US leadership, however, continued to kick the can of the ‘Russian problem’ down the road.

The Russian-Ukrainian war may look like ‘a good war’ – where the United States only uses treasure and arms, not blood – that counters an aggressive Russia which expends munitions and men. But the price of the failed inclusion of Russia into the Western security architecture will keep growing. The arms race is already back in Eastern and Northern Europe – from Finland to Poland – to back-up the US security umbrella. Washington ended up in a position of indispensable security guarantor of this enormous area against a weakened, but still dangerous country, possibly for decades to come. It is not clear if Washington will be able to hold ‘this fort’ politically for long, considering the rising volatility of American politics and the growing challenges of competition with an ascending China.

This essay reflects the proceedings of The Failure of the Post-Cold War Global Order event, hosted by the Henry A. Kissinger Center for Global Affairs at Johns Hopkins SAIS and the University of Mainz.