Why are the energy policies of major democracies now seemingly characterised by confusion and muddle? The challenges involved are complex and have become increasingly so in consequence of climate change policies. Complex problems do not, however, necessarily require complex policy responses, as Adam Smith demonstrated long ago. His conclusion, for example, that ‘monopoly[…] is a great enemy to good management’ implies a simple policy of avoiding both the creation of and dependence on monopolies wherever realistically feasible. We are therefore led to another question: why, in the case of energy in general and of electricity supply in particular, might complexity pose a particular obstacle to coherent policymaking in democracies?

In practice and for some obvious reasons, the energy sector tends to attract the attention of politicians of all stripes. Energy is a basic factor in economic development; much of the growth in labour productivity since the eighteenth century is associated with the harnessing of concentrated sources of energy in assistance of human effort. Additionally, power over the disposition of physical power comes with power over human populations and the latter is enhanced by the ubiquitous use of electric power, which is consumed in virtually all homes and commercial and industrial activities.

On the authoritarian side of the political fence, Lenin pronounced that ’Communism is Soviet power plus the electrification of the whole country’, signalling energy policy to be a very high priority. Interestingly, in the same speech (in December 1920) he also said that ‘Henceforth, less politics will be the best politics’. This could be interpreted as no more than a familiar, communist utopianism, but there was more to it than that.

Beyond the overarching aim, political power, and identification of a chief priority (electrification), his point was that politics was better off kept out of the weeds of economic policy, where a complex tangle of how-to questions has to be addressed. Immediately before the ‘less politics’ sentiment, he said: ‘We have, no doubt, learnt politics; here we stand as firm as a rock. But things are bad as far as economic matters are concerned.’ Lenin was talking about the latter, about de-politicising the process of answering the how-to questions.  

In today’s Western democracies, in the context of addressing the climate change challenges, we now hear advocacy for the speedy ‘electrification of everything’. Yet, notwithstanding a shared intensity of interest in economic development, the energy strategy of Russia does appear to have been rather less muddled than that of the West. Why is that?

There is a simple division of labour/responsibility in Lenin’s thinking that foreshadows the development of energy policy in the UK in the 1980s, based on a separation of the high politics, focused on overarching gains, enduring priorities and the institutional architecture of policy development, from the how-to questions that are better delegated to specialists capable of assembling and applying the requisite, collective know-how. With hindsight, however, it can be seen that this separation has unravelled in the UK and in other democracies that have attempted to do the same. That suggests another ‘why’ question: why did that happen?

It is a characteristic of good policy strategies that they simplify responses to challenges, but strategic simplification is the step that democratic politicians appear to have difficulty in making. Instead of simplification, what we observe in practice is what is sometimes referred to as ‘retail politics’.

This conceptually frames the politicians’ challenge as dealing with a multitude of groups with conflicting ‘partial’ (meaning partisan) interests – whether to do with financial gain, promulgation of an ideology, expanding the influence of a bureaucracy, or whatever — to each of which it is desirable that some beneficial, identifiable outcome/deliverable be offered, usually proportioned to each group’s salience to the relevant political decision makers at a given point in time.

It is a view that works well for the managers of supermarkets – fluctuations in consumer tastes and spending patterns can be addressed by variations in the product mix on offer. It works much less well for the developers of a national energy policy. Energy policy is a singular thing, a system rather than a collection of products. Moreover, the infrastructure of a shared energy system is not quickly and easily adjustable to the shifting political salience of different interest groups; the timescales involved on the two sides of the equation differ.

A more insightful analogy for energy policy development is software programming, where the task is to construct sets of ‘rules’ (the coding) whose individual elements are designed to work together. For these ‘rules’, loss of coherence, for example, bugs, clumsy coding, can substantially harm end performance. Using this analogy, the problem might be characterised as a desire of politicians to write specific ‘fixes’ into the code to ‘deliver’ identifiable benefits to salient partial interests, without taking any great heed of the induced effects the fixes can have on the performance of the programme as a whole.

The broad general problem can be illustrated by reference to what is frequently referred to as the ‘energy trilemma’. It identifies three, high-level aims for energy policy and points to the inevitable trade-offs involved in their pursuit.  The three are: affordability/consumer protection, reducing environmental harms and security of supply.

Affordability is of great interest to democratic politicians chiefly because of the range of its impact on voting behaviour – energy supply is significant for more or less everyone – and it finds its day-to-day champions in consumer and, because it affects their costs, commercial pressure groups, energy intensive industrial users of energy being prominent among the latter.

Reducing environmental harms is nowadays largely, but not exclusively, focused on decarbonisation. It, too, affects voting behaviour, though only for smaller sub-sets of the electorate. The main point of issue is not de-carbonisation as such, but rather the calls for immediate and costly actions in pursuit of it, typified by a statement such as ‘we have N years to save the planet’, where N is a small number. Its most active advocates tend to be motivated by ideas and a sense of moral purpose rather than by financial interests, but their voices are reinforced by business interests who stand to benefit from greater urgency. Instilling a sense of urgency among those asked to pay is a familiar marketing ploy of ‘sellers’, of ideas as well as of goods and services, to achieve higher sales and/or higher prices than would otherwise be the case.

The notion of security of supply is less precisely defined; it can encompass a range of issues, from maintaining energy systems in minute-to-minute balance so as to avoid black-outs (an activity known as systems operations), to much longer-term matters of a geopolitical nature, such as those that came to the fore in the wake of the invasion of Ukraine, most specifically, the dependence of Western Europe on politically monopolised supplies of gas, oil and coal from Russia. It is characterised by an absence of any interest group that advocates for it on a sustained basis. Hence its political salience is more events-driven – that is, triggered by some or other manifest supply failure – and the attention given to it is typically more sporadic.

The problem is that the political salience of each limb of the trilemma tends to vary over time with a frequency that is higher than energy system infrastructure investment can accommodate at moderate cost. Crucially, when political salience shifts, ‘retail politicians’ have a tendency to apply short-term fixes or patches to the policy design, which can undermine the functional integrity and effectiveness of the policy as a whole.

A particular problem in energy systems is that some fixes can resemble bugs in a software programme that can cause the programme to crash when a particular set of conditions is satisfied. The title of a book devoted to analysing banking supervision ahead of the 2008 financial crash captures the problem well: Fragile by Design by Charles W. Calomiris and Stephen H. Haber. The fragility – an extreme vulnerability to unanticipated shocks – is not intended by the designers, but it is nevertheless a feature of the design.

In 1998, California implemented a major reform of its electricity sector, following the earlier example of England and Wales in vertically separating the generation and retailing of electricity from the transmission of electricity through its high voltage grids and, simultaneously, establishing a competitive electricity generation market. The arrangements, which included a relatively crudely constructed retail price cap (a fix directed at protecting consumers from potential price spikes), worked in a satisfactory fashion for a couple of years. But high demand in the summer peak period of 2000 led to a combination of black-outs, brown-outs (drops in voltage that impair and, in some cases damage, electrical appliances) and sharp upward spikes in wholesale prices. As a result of the squeeze in retail-wholesale margins, there were utility bankruptcies and the reforms collapsed, leaving taxpayers and consumers with a financial burden running into tens of billions of dollars in money of the day.

Broadly, it might be said that the short-term, political prominence of ‘affordability’ led to a policy design that gave insufficient weight to security of supply considerations, even of a relatively near-term and local nature, no geopolitical factors being relevant in this case.

In Germany, the decisions to phase out reliance on nuclear power were associated with an increase in the political clout of those who identify as Greens. Nuclear power can be a divisive issue for Greens: it is a low carbon technology, but it comes with risks of other, serious environmental harms — for example, accidental discharges and leaks of radioactive substances, whether from operating reactors or from the storage of nuclear waste. An anti-nuclear stance was particularly strong among German Greens. Temporarily empowered by election results that substantially increased their votes in the legislature, they negotiated a fix to the existing system design that favoured their own priorities. It was a victory of a partial interest over a more general interest that could reasonably have been expected to have favoured giving higher weights to affordability and to security of supply. The invasion of Ukraine exposed some major consequences of that imbalance, including a consequentially greater dependence of Germany on (monopolised) Russian supplies of fossil fuels.

The British example is somewhat different, concerning a much smaller-scale, politically motivated intervention in 2009 aimed at putting limits on differentials in retail energy prices across regions of the country, not a price-cap on the average price across all regions (that would come later). In this case, there was no significant lobby group in favour of the measure. The political salience came from a recently appointed Secretary of State for Energy and Climate Change, Ed Miliband, a politician on the rise looking for an attractive sounding cause, like ‘no discrimination between regions’, to bolster his reputation.

As anticipated by leading energy economists at the time, the fix turned out to be bad for affordability, it created upward pressure on retail prices, and was later abandoned, but, alas, regulatory processes are characterised by hysteresis, meaning the dependence of the condition of a system on its previous history. Until around the end of 2007, the division of labour in energy policy envisaged at the time of the creation of independent regulators in the 1980s had been well sustained. Political pressures on the energy regulator Ofgem to go beyond its allocated statuary duties, to allow politicians to make immediate ‘retail offers’, were moderate and, when they did occur, were met with firm resistance. However, the regional pricing fix was the first significant breach in the separating wall between a more volatile retail politics and the more considered, longer-term approach to decisions allocated to the regulator, and it was a breach that widened quickly in the years to follow.

What is to be learnt from all this? The most important lesson is probably just that energy policy has many, far-reaching effects and that the political salience of each of the different effects can be quite volatile in democratic systems. That is, in part, linked to electoral cycles, but only in part. Each new minister responsible for energy tends to arrive at the desk with a view to making a mark on things, often with some pet schemes of their own and an almost invariable predilection for turning a blind eye to the consequences of their actions.

That results in higgledy-piggledy policy shifts that are difficult to predict, and which both cumulatively impair policy coherence and effectiveness and, more specifically, chill incentives for long-term, unsubsidised investments in an energy system, because businesses are less able to form expectations about the future policy environments in which they will operate.

Authoritarian leaders appear to fare better in developing the temporally stable policy simplifications that go under the name of ‘strategy’ and that are advisable in the face of uncertain complexity. These leaders can suppress at least some of the volatility in political salience that characterises democratic systems. Their major weaknesses are somewhat different in nature. For every complex challenge there is potentially a range of simple, but bad/wrong responses and, in the absence of vigorous public debate, sorting the wheat from the chaff is less likely. Once chosen there is also a tendency to stick with a poor policy for longer, since critical views can also be suppressed.

What, then, is to be done? When it comes to energy policy, less retail politics is better politics, but achieving that does not necessitate suppression of political competition. Retail politics is simply one mode of competition and other modes are available. Businesses compete with each other via strategic differentiation, creating market environments characterised by strategic diversity, and so too can political parties. More political competition in policy/strategy development, not less, tends to provide stronger incentives to discover and implement better policy options. A shift to that type of competitive discovery process, focused on choices between alternative energy strategies, not between me-too policies accompanied by opportunistic and volatile differentiation of their bells and whistles (retail offers), would be an appropriate response to the new context created by the war in Ukraine.