Why is it difficult for markets to work in the electricity sector ?

Authors: Manish Agarwal and Yogesh Upadhyaya

“Market forces will resolve all issues in the electricity sector.” We hear this often. We believe that unless you work through the details, this is a lazily held view. Electricity supply is very different from products like ball bearings or ice cream — where the markets can become competitive much more easily. Electricity has many structural barriers to competition. In this article, we will point out why it is infuriatingly difficult to create a competitive market in electricity. Some of these barriers are specific to India and others are caused by the nature of electricity itself.

A few months ago, Akshay Jaitly and Ajay Shah published a paper, “The lowest hanging fruit on the coconut tree: India’s climate transition through the price system in the power sector.” Unlike many other commentators, the authors have worked hard at building a case for a price driven electricity sector. In the paper, the authors write

“All firms in the electricity sector need to be creatures of the market economy, which constantly reshape technology and business models in response to prices.”

“The problem is at the core: a government-controlled centrally planned system.”

“This involves going with the grain of the price system i.e. stepping away from the command and control system”

Jaitely and Shah have argued that a market based electricity sector combined with a Carbon tax is the best way to decarbonise the Indian electricity sector.

In this article, we will show why it is extremely difficult for firms (and customers) to be creatures of the market economy. We will also show that a lot of problems of the electricity sector are not because it is government controlled, but because elements of the sector are highly prone to becoming a natural monopoly. Even parts that could be set up as a competitive market will be susceptible to market failures, with the risk and impact both being higher than in other sectors. These reasons make it extremely difficult to leave fuel choice and capacity additions to market forces. Privatization can lead to operating efficiency, as experience in India has shown, but even that needs a lot of careful and hard work. Electricity sector is different from ball bearings (their example). Markets can work. But these markets will not work just because the governments ‘leave the sector alone’. On the contrary, they will work only if governments help design these markets carefully and with great humility and monitor them regularly to see if the markets are achieving the desired objective. We should have realistic expectations from what the markets can achieve.

Our core arguments

1. Reliable and affordable electricity is key to development. Citizens cannot go about their daily lives without transportation, refrigeration and many other products and services which depend significantly on electricity. Modern businesses cannot survive without reliable and affordable electricity. Income levels and quality of life of Indians cannot be improved without reliable and affordable electricity. Hence, decarbonisation of electricity cannot be the only objective of the electricity sector policy. Any sector reform program (including reliance on competition and market forces) has to be tested on its ability to provide cheap and reliable electricity.
2. Most of the challenges of the electricity sector in India are in distribution. Most distribution companies have high transmission and commercial losses. It is extremely difficult to create competition in distribution. Privatization with enlightened and strong political and bureaucratic support is what has worked.
3. Market means producers and consumers being free to choose each other. This is difficult to do in the electricity sector because demand and supply is bid for and matched days in advance and this makes it easier to manipulate the market. More importantly, in India cross subsidy makes creation of markets almost impossible. Finally, to create meaningful markets, supply from existing producers would need to be allotted to new players. All these make the electricity markets synthetic and unlike most other markets.
4. If somehow a market for electricity were to be created, the price signals would have no influence on most electricity suppliers. The dominant coal producer in the country, Coal India Limited (CIL), does not respond to price signals. India has a high share of hydroelectricity and many dams have dual objectives of irrigation and electricity generation with the former taking priority. It is impossible to set up nuclear power generation plants based purely on market signals without any government involvement. Renewable energy has thrived because it was kept outside the power market, and was required to be dispatched whenever available.
5. Leaving fuel choice and capacity addition to market forces can be disastrous for India. If private investors do not invest in fossil fuels, and electricity storage solutions do not become cost-competitive at large scale, India could fall foul of its objective of reliable and cheap energy. Achieving the objectives of cheap & reliable power and decarbonisation will need a calibrated, dynamic balance; the Government will necessarily need to be actively involved in this.

Our argument in detail

Before we begin, it is important to state what this article is not about. This article is not about maintaining the status quo. This is not an argument for continuing with public sector dominance of the power sector, and the regular bail outs of Distribution Companies (discoms). We very much welcome private sector investment and associated efficiencies; but want to caveat the challenges in harnessing those efficiencies, and point out why “let Govt get out of the way” is neither feasible nor consistent with the energy sector goals of the country.

Both the authors share the frustration of Jaitely and Shah with the electricity sector. One of us (Yogesh), started his career advising the State Electricity Board (SEB) on privatization related issues. He then switched to rating SEBs and other entities in the electricity sector. Five years of this and he was so frustrated with lack of progress in the sector, that he went out and co-founded a software company. The other (Manish) had a similar start but had the gumption to hold out for much longer. He not only advised state entities on privatization of generation but also actively worked in privatizations in Orissa and other discoms. He has advised the Central Electricity Regulatory Commission in its early period, and spent three years in Andhra Pradesh Electricity Regulatory Commission, in designing regulations and supporting tariff making. He has spent a fair part of his career working with policy makers and regulators to enable more private sector investment in the electricity sector.

Both the authors also share the love that Jaitely and Shah have for market based solutions. One of us (Yogesh) ran a Fintech company for ten years and can personally testify to the benefits of unfettered innovation. In his writings, Yogesh has spoken admiringly about Friedrich A. Hayek’s “Use of knowledge in society.”. The other (Manish), in his decades in consulting governments, has many times advocated for and seen the benefit of the energy of the private sector. These benefits are reflected in data. For example, private sector power generation companies set new benchmarks in plant availability (from 68.5% to over 95%), coal consumption efficiency (reducing Heat Rate from over 3000 kcal / kg to below 2000 kcal / kg) and construction time (reduced from 45 months to less than 30 months). Many public sector generation companies now deliver these benchmarks. Subsequently, in implementing the reforms envisaged in Electricity Act 2003, Manish was actively involved in design of power markets, enabling open access, and reducing cross subsidies.

We begin our detailed working with a very basic but often forgotten observation. Decarbonisation cannot be the only aim of an energy or electricity policy for India. Electricity is critical infrastructure and it needs to be reliable and cheap.

Cheap and reliable energy is critical for development

a. India needs affordable energy to increase income levels

Affordable energy is critical for development. History shows that economic growth accompanies periods when energy is available cheaply. As Vacalav Smil points out in his book “How the world really works” while talking about the oil crisis of 70s,

“..and this ended the era of rapid economic expansion that had been energized by cheap oil. From 1950 to 1973 the Western European economic product had nearly tripled and the US GDP had more than doubled in that single generation.”

Cheap energy is crucial for businesses. If energy is expensive, businesses have less money for investing in growth and for paying their employees. Cheaper energy is a key part of Real Ease of Doing Business as we have discussed here. Citizens and the economy are also very sensitive to the price of energy. High energy costs can severely affect households. High energy costs can also wipe out the surpluses of farmers and of businesses. High ball bearing prices do not have anywhere near the same impact.

b. Reliable electricity is critical to modern life

Energy should not only be cheap but also reliable, i.e. available on demand. Modern life is not possible if energy is not available reliably. We would not be able to perform basic functions like cooking at home and commuting to our work place. Logistics chains would not be able to supply all the goods that we consume.

Given the centrality of energy in society and economy, cheapness and reliability should definitely be the goals of the society’s energy policy.

Electricity is the most convenient form of consumable energy. Use of electricity is effortless — you just have to flick a switch. It is clean at the point of use. No fuel needs to be stored. No stoves need to be cleaned periodically. This is why all modern societies have an increasing electrification. Given the criticality of electricity to society, the electricity grid needs to be reliable. Here is Smil again,

“A very high reliability of electricity supply — grid managers talk about the desirability of reaching six nines: with 99.99999 percent reliability there are only 32 seconds of interrupted supply in a year! — is imperative in a society where electricity powers everything from lights (be they in hospitals, along runways, or to indicate emergency escapes) to heart lung machines and myriad industrial processes.”

A tragic and dramatic demonstration of the importance of electricity came about in the last two years when patients across the country died from Covid when power cuts made it impossible for them to get Oxygen. A google search of ‘power cut covid death’ throws up pages and pages of results. India should also be targeting the reliability targeted in western grids.

c. Governments will intervene if the market makes electricity prices go high

If electricity is not cheap and reliable the society suffers. If ice cream or ball bearing supplies are not cheap and reliable, it is a minor inconvenience. In that sense, electricity is similar to any other infrastructure sector like roads, telecom and ports. Like any other infrastructure industry, society should be investing for the demand we would see tomorrow and not today but markets may not always ensure that. This is because, for producers, the benefits of shortages and the costs of over capacity may be symmetrical but for a society the cost of shortages may be much more than the cost of overcapacity. We will look at this issue in detail in our section on markets.

Having looked at the criticality of electricity, let us turn our attention next to distribution. This is the area which is most in need of reform. This is also the area where the power of markets does not work.

Power sector cannot have “all players as creatures of the market”

The electricity sector can be understood as three broad segments. a. The transmission and distribution wires (along with the transformers, capacitors, meters, etc); b. the power plants that generate the electricity that flows over these wires and into our homes, malls, farms and factories and c. customers.

a. Viable discoms are prerequisite to creating a market

The wires business (transmission and distribution) is a natural monopoly. It does not make sense to have multiple grids in a region. For example, anyone setting up a second transmission wire from a distant power plant to the grid would always be outcompeted by the existing player.

A large part of the challenge in the electricity sector — probably the largest — is that the tariff for many categories of customers is lower than the cost of supply. This low tariff is cross subsidized by other categories of customers, typically business customers. The high tariff for businesses incentivises them to steal electricity, often with the help of discom employees. The AT&C losses across the country were 20% and the combined losses of distribution companies was Rs 75,000 crore in 2019–20. Note that most of the power generators make profits but get into financial trouble because the financially strained distribution companies do not pay them on time.

Improving the operations of a distribution company means

• Accurately measuring the electricity consumption for the customer categories that are subsidized.
• Raising tariffs for the subsidized customer categories.
• In case, the state decides to subsidize a certain category of customers, making sure that it pays the subsidy to the distribution company in time and in full.
• Stopping theft of power.
• Reducing technical losses.

c. Discom viability depends on regulation, political will, fiscal space, and perhaps privatisation

The first thing to note is that these improvements, if they happen, will not be driven by competition but by the regulator and by the distribution company operator. More importantly, all the above require a tremendous amount of political will. A big risk for any private entity is whether the political will to support such reforms will continue in face of protests, electoral reverses and change in government. Added to the risks is the challenge that the return of the distribution company is capped by regulation. This incentivises private operators to indulge in illegal practices such as gold plating investments (that is inflating the cost of equipment to get a higher return) potentially negating the benefits of increased efficiency.

The empirical evidence for benefits of privatization of distribution is mixed. The privatization of three distribution companies in Delhi can be termed a success. In Delhi, for example, the AT&C losses came down from well over 60% prior to privatization to below 40% by the fifth year and below 20% by the tenth year of privatization. In Orissa, the privatization of distribution had to be reversed. There is evidence to show that a carefully managed privatization of distribution with backing of political will results in better operating efficiency. However, these gains do not come from ‘market forces’ but come from the working together of an enlightened regulator and an efficient operator and government. Success also requires a strong and long term backing of political reforms.

Private operators need a very high level of confidence that when they start detecting and preventing theft of power, the political system will back them up fully, and protect them from local backlash. Achieving this was difficult in Delhi, and more so in Bhivandi (a handloom dominated area outside Mumbai, where Torrent, a private company, has been distributing and supplying electricity for many years). The success in Delhi and Bhivandi have not led to many more States going down this path. The reason, perhaps, is that the benefits do not seem to outweigh the political costs.

If competition is not possible in distribution then what do people mean by market forces in the electricity sector? It is customers and suppliers having a choice to trade with anyone. Let us look at that next.

Electricity markets are synthetic; need careful monitoring

There are two levels at which competition works in the power sector (not mutually exclusive). One is known as Bulk Supply competition, where generators and large purchasers (primarily Discoms), transact in a Power Market. The other is known as Retail Supply competition, where multiple suppliers compete to supply to end consumers, using the wires of the discom.

a. Competition cannot be effective if price regulation prevents cost-reflective tariffs

Let us talk about Retail Supply competition. This is the ideal world that is imagined by those in favor of unleashing market forces. Monopoly distribution companies that are regulated (and contribute a very small proportion of the overall cost); and several suppliers (generators and traders), providing tailored packages to different types of consumers. Customer choice, as in case of telecom and direct-to-home entertainment, is expected to be replicated in the power sector. Setting up customer choice is extraordinarily difficult. Let us discuss some of the difficulties.

The first difficulty is that to set up a working market requires the elimination of cross subsidies. What are cross subsidies? Well, certain consumer classes (typically, industries and commercial offices / malls) are charged a tariff much higher than cost, so that the surplus can be used to keep tariffs below cost for other consumers. In India, industry and commercial businesses pay much higher tariffs than agricultural customers and many households. This is when the cost to supply to industries is a lot less than the cost to supply electricity to farms and to houses. This is because of the long low tension distribution lines needed to supply to farms and houses.

Why does cross subsidy become a hurdle to Retail competition? Clearly every supplier would want to supply to industrial and commercial customers. The supplier left with subsidized customers will have an extremely unviable business. To compensate, the regulator could allow a cross subsidy surcharge to be levied on suppliers who are serving businesses. This, along with other non-tariff measures, make it unattractive for alternate suppliers to compete with the discom. In any case, these charges distort the market and reduce the power of prices.

The Electricity Act (Amendment) Bill has proposed a cross-subsidy fund, as a means to enable retail competition. We have discussed our concerns with the Amendment Bill here . Taking subsidy delivery away from the Discom is a necessary step. Its success will be critically dependent on the fiscal capacity of the State and the detailed design of the cross subsidy fund.

b. State Governments do not have the fiscal space to allow prices to reflect costs

How difficult is it to eliminate Cross Subsidies? It is so difficult that State Governments insisted on the Electricity Act to be amended, so that the provision to eliminate cross subsidies can be deleted. On all India average basis, industrial and commercial consumers pay Rs 7.4 per unit and Rs 8.1 per unit, while domestic and agricultural consumers are charged Rs Rs 4.5 per unit and Rs 0.75 per unit. The difference is wider in several States.

If somehow, a state government was persuaded to eliminate cross subsidy and promise the distribution companies that they would pay them the subsidy from their budget, then we run into a different set of problems. It would be very difficult for any reputable business to trust a government to make these payments in a timely manner.

We earlier mentioned that we like the idea of Direct Benefit Transfers (as Electricity Coupons, if there is concern about the money being used for less productive purposes). We also like the idea of Cross Subsidy removal, because it would make the Indian industry more competitive. But that also means more burden would be on the State Govt to subsidise needy categories (remember, affordable power is necessary for economic growth). Imagine the fiscal cost of removing cross subsidies, where State Govts have extremely limited ability to raise taxes. Imagine the political cost of removing cross subsidies, and making industrial tariffs lower than agricultural and household tariffs because it costs less to supply to industry than to households and agriculture.

c. Consumers cannot respond to price signals if prices are not deregulated

Finally, we note that if customers are subsidized, then those customers would not respond to price signals. A farmer that is getting free power has very little incentive to not waste it by leaving the water-pump on all night, for example. Additionally, the cost of power varies by time of the day and time of the year. For a power market to function, this will need to reflect in consumer end tariffs. Implementing Time of Use tariffs requires advanced meters to be installed at the consumer end. While a program to do so is underway, it will take a long time. Considering the large population of consumers that have not paid flat meter based tariffs, implementing time-of-use based pricing poses a challenge for regulators and discoms.

d. Even in developed countries, power markets have been gamed

If governments were persuaded to remove cross subsidies and provide timely subsidies, we would face another set of difficulties in creating markets in electricity. Electricity markets are synthetic, and prone to gaming; so difficult to set up and regulate. Why is this?

In a grid, the demand and supply has to be exactly matched at every instance. If the demand and supply are not in balance, the grid can collapse and even damage the equipment. In other words, a shortage in one area can lead to widespread non availability of the grid.

This means that the only way to ensure reliability is to have production capacity and affordable storage in excess of peak demand. Again this is not true for other industries. A production shortfall in ball bearings can be easily managed because of inventories that exist through the supply chain.

To ensure grid stability, a System Operator determines when and which suppliers would meet demand. Similar regulatory bodies also create the market for electricity. The System Operator monitors its functioning. This market is created hours and sometimes even days in advance. If a significant supplier fails to meet the demand of a customer, not only is the supplier in breach of contract of the contracted customers, but also the supplier can make the whole grid unstable. This is not the case in any other product or service. This grid manager can be a government body or could be a cooperative but no sizable grid across the world works without such an active hour by hour centralized manager. In this sense, ‘centralized command and control’ is built into the structure of the sector.

This characteristic makes electricity markets very open to manipulation. This is precisely what happened in the California electricity crisis of 2000 and 2001. Electricity prices rose sharply during times of high demand. The situation was manipulated by Enron. Company officials were caught on tape cajoling power plant operators to shut their plants so as to raise the prices even further. Apparently, Enron had many different strategies for price manipulation — with such innovative names as Deathstar, Black Widow, Big Foot, Cong Catcher, Forney’s Perpetual Loop, and Red Congo. Customers had to pay Billions of Dollars in surcharges.

e. Reallocation of contracts, to reduce market dominance, will be complex and long drawn, to avoid charges of favoritism

This is not all. If markets have to be set up for electricity, existing depreciated assets have to be allocated to all suppliers of electricity fairly. If, as an incumbent, I own generation assets that have been paid for, and have contracts with power producers who run depreciated plants, then I would have an advantage over any new comers. These assets have to be distributed over all newcomers for any competition to be meaningful. However, this redistribution would be done in a process designed in the backrooms of government and corporations. Hardly a description of competitive markets.

Overall, given the complexities, making a statement about the efficacy of electricity markets is meaningless unless you have looked at the detailed rules that govern the set up and the running of the markets.

In summary, markets cannot work without removal of cross subsidy and a believable commitment from the state government on payment of subsidy. Also, it is meaningless to speak of the efficacy of these markets unless you look at the detailed setup. If somehow, we are able to set up very good and robust markets, we come to another problem. Most of the cost of supply is in generation and most power generators cannot meaningfully respond to price signals.

Most power producers in India cannot respond to price signals

Most of the generation capacity in the country cannot respond effectively to price signals. Coal fuelled thermal power plants are responsible for approximately three fourths of electricity generated in the country. For the older depreciated plants, fuel costs constitute most of the costs. More than half of these plants get coal from Coal India Limited (CIL). CIL is almost a monopoly and does not respond to price signals. If it did, it would have produced much more coal and not forced users in the country to import more than 200 MT every year.

Other coal plants import coal. Price signals from the electricity market cannot impact the imported coal price too much. Incidentally, creating a competitive market for coal that delivers cheap coal is not a trivial task at all.

India generates a lot of electricity from Hydroelectric power plants. These plants have a dual purpose of serving irrigation needs and generating electricity. In most plants, irrigation takes a higher priority. For example, a hydroelectric power plant cannot save water for summer when the demand is maximum, if its irrigation consumers require the water earlier. Hydroelectric power plants are not free to respond to price signals.

Nuclear power plants are considered by many to be an essential part of the decarbonisation strategy. We will go into the benefits and risks of Nuclear energy in a subsequent piece but we note here that given the dual use nature of nuclear energy and given the high initial capital costs, it is impossible to build nuclear power plants based on market signals and without intimate involvement of the government. Please see box on France and Nuclear energy for more on this issue.

Ironically, the Variable Renewable Energy Sources (Wind and solar) can be integrated in grids only if we ignore price signals. A lot of the Variable Renewable Energy projects were possible only because they were allowed high tariffs. Even today, a VRE project would not be viable without a ‘must run’ status — an obligation of the DISCOM to buy electricity from these plants whenever the sun is shining or the wind is blowing.

Jaitely and Shah have spoken a lot of Carbon Tax being the only policy lever. As an aside, we note that government extraction from taxes, levies and dividends from the fossil fuel sector is already very high. In 20–21, fossil fuels contributed more than a third of the central government revenues (without borrowings). This is a significant Real Ease of Business issue as we have discussed.

We have discussed till now why reliable and cheap electricity should be a policy goal. We then detailed how it is impossible to create a market in distribution — the place where most of the problems of the sector lie. We then discussed challenges of creating markets in electricity supply. And we just said that even if electricity markets were to be created, the price signals from them would not impact most of our domestic suppliers. There is another challenge with fuel choice. Electricity markets are not very good at allocative efficiency.

Markets cannot deliver the balance between energy security and decarbonisation requirements of the country

Energy prices have shot up across Europe and the world since the beginning of the Russia Ukraine war. Energy bills for households have gone up many fold across Europe and businesses are in trouble even in industrial power houses like Germany.

As we discussed in one of the first sections, electricity and energy in general is crucial to economies and disruption in supply or increase in prices can cause severe distress to the society. A cheap source of energy may be fine for many years but if the supply and price is not in your control it is a looming risk, as Germany and Europe have discovered. Markets are incentivised to optimize short term costs and not reduce long term risks. Let us take an example.

In theory, it sounds simple that a Carbon tax tilts the playing field in favor of clean energy, and it is then left to the market (i.e investors) to decide on capacity addition (fuel, location, size). In practise, this can be disastrous for the country, and politicians will not (and should not) permit this. Can coal based capacity addition (the cheapest source of power, and essential for reliability) be left to private investors ?

a. If electricity storage costs do not decrease rapidly, coal plants will be extremely valuable, and, in a competitive market, be able to demand high prices. In practice, investors know that Governments will not permit them to charge high prices (as we have seen with windfall gains tax in Europe and price caps in India, recently).

b. On the other hand, if electricity storage costs do reduce, the private investment in coal plants could become worthless. Thus, with risk of downside, and unlikely upside, private investors will not invest in coal plants in absence of long term assured offtake commitments from discoms

c. The biggest hurdle to private investment in coal is that financial investors are not willing to invest in fossil fuels, following the ESG and net-zero commitment requirements.

What would happen if the market forces do not lead to coal based generation capacity addition, and electricity storage does not become affordable ? The Government cannot afford such a situation to arise, and therefore needs to actively, and dynamically, take decisions to avoid a situation of expensive and/or unreliable power.

Other options for reliable power are hydro and nuclear. The government is the most important decision maker in setting up Hydro and Nuclear power plants, it is clear that markets cannot have any role in deciding the generation mix in the country. This is the decision of the government and is perhaps the most important one that a government can take.

Summary

We have discussed that ‘markets can take care of our problems in the electricity sector’ is a very misleading statement.

• The objective of reliable and cheap energy should be at the heart of any reform and markets, left to themselves, do not deliver on this objective.
• The biggest reform is needed in distribution, and competition is not possible there.
• Making distribution viable would require the political will and fiscal capacity to address subsidies and cross subsidies, and institutional strengthening regulatory authorities and discoms to deliver efficiency improvement. None of these are outcomes of a market; these are prerequisites
• If these are done, creating a functional electricity market will still be a formidable challenge. It would require allocating existing generation assets (or privatising them), and strong institutional mechanisms to enforce rules and prevent gaming. By nature, electricity markets are ‘constructed markets’. Very different from markets in most other products and services.
• Even if somehow an electricity market were to be created, most generators would not be able to respond to price signals. Coal supply is a monopoly, hydroelectric plants run according to irrigation needs and not power needs, and renewable energy plants need to be dispatched whenever they are running.
• Similarly, at the consumer end. If the market leads to shortage and/or price increases, Governments will intervene. The cost of energy shortage to the economy is very high; unlike ball bearings.
• Finally, capacity addition (in terms of choice of fuel, location, size) cannot be left to market forces. Synthetic markets are not good at delivering reliability, nor in discovering the appropriate cost of reliability. That private investors do not want to invest in fossil fuels (which are the cheapest source of reliable power at present) only compounds this challenge. Considering that the cost of energy shortage is significantly higher than the cost of under-utilisation of assets, Governments will need to remain actively involved in the balance between energy security and decarbonisation.

Twenty years of having a power market have shown that several other steps are needed to reform the electricity sector. These steps are well known. None of them are easy; they require coordination between States and Center, and across departments. But none of them can be wished away by “unleashing market forces”.

We thank Neeraj and Santosh for their inputs

You can follow AskHow India (@AskHowIndia) or Follow Manish Agarwal (@manishbgagarwal) or find him on LinkedIn

Yogesh Upadhyaya can be found on @YogeshUpadh on twitter or LinkedIn or on https://t.me/YogeshUpadhyaya