The History and Evolution of the U.S. Electricity Market
If we are to select a single picture that could represent modern civilization of mankind, perhaps nothing would be more appropriate than NASA’s satellite image of Earth at night. Beneath the night sky, connecting these clusters of light is the transmission grid built on Earth by mankind over the last century. Operating such a large and complex power grid must not only consider safety and stability, but also take into account socio-economic benefits. How is it done? I hope that through this article, everyone can have a preliminary understanding of one of the world’s most mature power markets – the North American power market.
The story starts with the battle of the currents between Thomas Edison and Nicola Tesla in the late 19th century. According to legend, in order to prove that DC power was superior to AC power, Edison was willing to go so far as to electrocute an elephant using AC power. However, in the end, AC power won out because of its ease of transformation and lower transmission losses over long distances. It is precisely the feature of AC power being easy to transmit over long distances that determined the economies of scale in power transmission and distribution (increase in scale leads to increase in economic benefits, and long-term average total cost decreases as output increases). The power industry also gradually developed into a natural monopoly in the course of its development. In 1973, the fourth Middle East war broke out and the Organization of the Petroleum Exporting Countries (OPEC), led by Arab countries, announced an oil embargo, which led to an energy crisis. In the US, the continued growth in population and electricity demand also made the power industry aware of the urgent need to break away from dependence on oil and natural gas. A large number of coal-fired, nuclear and small-scale independent power plants that rely on non-fossil energy sources (geothermal, wind, solar, etc.) emerged. To cope with the energy crisis, the US House of Representatives passed legislation in 1977 to reorganize the Federal Power Commission (FPC). At the same time, the Federal Energy Regulatory Commission (FERC) was established. In order to ensure that these newly established independent power plants can use the transmission grid fairly and compete with traditional power companies, FERC has issued a series of orders since 1992, and the Order No. 888 in 1996 marked the beginning of the liberalization of the electricity markets. The main goal of this reform was to separate the transmission sector from the natural monopoly, establish a wholesale electricity market, and promote free competition in the generation and retailing sectors. As of now, there are 7 Independent System Operators (ISOs) or Regional Transmission Organizations (RTOs) in the United States. These are the Midcontinent Independent System Operator (MISO), PJM Interconnection, Southwest Power Pool (SPP), New England Independent System Operator (NE-ISO), New York Independent System Operator (NYISO), California Independent System Operator (CAISO) and Electric Reliability Council of Texas (ERCOT). With the exception of ERCOT, the other 6 markets are all regulated by FERC.
Parallel to FERC’s efforts to liberalize the power market, the economic theories of electricity market also progressed. The Chicago school of neoliberalism has had a great impact on the initiation of power market reform. This includes Milton Friedman (winner of the 1976 Nobel Prize in Economics), who advocated minimizing the role of government and allowing the free market to operate, and George Stigler (winner of the 1982 Nobel Prize in Economics) who questioned whether natural monopoly is consistent with social welfare maximization. Their ideas influenced a group of Latin American economists who were educated at the University of Chicago (the “Chicago boys”), who in 1982 attempted a power market reform in Chile and achieved great success. Chile thus became the first country in the world to undertake power market liberalization. In 1983, Professor Paul Joskow of the Massachusetts Institute of Technology (MIT) pointed out that the competition can be introduced to generation and retailing segments, and only the transmission and distribution network has natural monopoly characteristics that can be regulated. In 1988, four professors from the Massachusetts Institute of Technology and Boston University, Fred Schweppe, Michael Caramanis, Richard Tabors and Roger Bohn, jointly published a book on the principles of spot pricing of electricity, discussing in detail the relationship between electricity spot prices and the operation of the power grid. In 1992, Professor William Hogan of Harvard University published an article discussing the design of financial transmission rights. There are many other scholars who have contributed to power market theory, such as Jean Tirole (winner of the 2014 Nobel Prize in Economics), Ross Baldick (UT-Austin), George Gross (UIUC), Frank Wolak (Stanford), Marija Ilić (MIT), Daniel Kirschen (UW), Mohammad Schahidehpour (IIT) and so on.
Electricity Market Design and Challenges
Despite having systematic economic theory as guidance, the process of reform in the US electricity market still encountered many unforeseen problems. These include flaws in policy and market design, as well as challenges from technology and modeling. The California power crisis in 2000 was a painful lesson. As the public’s demand for environmental protection and the government’s incentives for new energy subsidies increased, a large amount of renewable energy power plants was integrated into the power grid, posing greater challenges to the stability of the power grid and the operation of the power market. California has had a problem starting in 2012 known as the “duck curve,” which reflects issues with the penetration of renewable energy into the grid. At the end of this section, we will introduce the financial risks in electricity markets through a recent case of a default in the PJM financial transmission rights market.
Kirchhoff’s Laws and the Economics of Electricity Markets
Kirchhoff was a 19th century German physicist. Kirchhoff’s laws are a set of basic equations about circuits: the sum of the currents flowing into a circuit node is equal to the sum of the currents flowing out of the node (the law of conservation of charge); the algebraic sum of the voltage drops across all components in any closed loop is zero (the law of conservation of energy). The Kirchhoff equations have a very close relationship with the nodal pricing theory in the electricity market. In 1988, four professors working at Massachusetts Institute of Technology and Boston University published a book on the spot pricing of electricity, entitled “Spot Pricing of Electricity”. This book contains very detailed mathematical derivations. The authors proposed that the spot price of electricity is determined by the marginal cost of generation, while also taking into account constraints such as energy supply and demand balance, generator parameters, capacity of transmission lines, and Kirchhoff’s laws. The price of electricity at each node in the power market can be broken down into three parts: energy price (reflecting the marginal cost of generation), congestion price (reflecting regional price differences caused by transmission line capacity constraints), and loss price (reflecting the electrical resistance loss of current on transmission lines). The book perfectly blended concepts in mathematical, economics, physics and engineering. What’s more noteworthy is the diverse educational backgrounds of the authors: Fred Schweppe (electrical engineering and computer science), Michael Caramanis (chemical engineering, applied physics, and control theory), Richard Tabors (biology, social science, geography, and economics), Roger Bohn (applied mathematics and management economics) – engineers who don’t understand physics are not good economists!
Regional Transmission Organizations and Independent System Operators
The power grid and electricity market are so large and complex, so how is it managed? At the federal level, there are the North American Electric Reliability Corporation (NERC) and the Federal Energy Regulatory Commission (FERC), which are responsible for the reliability of the North American power system and the supervision of interstate power sales/wholesale electricity prices, respectively. In each state, there are public utility regulatory commissions (PUCs) that oversee the state’s electricity market, power allocation, and rate approvals. There are seven non-profit Regional Transmission Organizations (RTOs) or Independent System Operators (ISOs) that are affiliated with Eastern Grid, Western Grid, and Texas Grid, and are connected by high-voltage direct current transmission lines. In addition, there are some for-profit businesses that own assets, responsible for constructing, operating, and maintaining transmission lines.
The Market Segmentation
The electricity market can be segmented according to the product being traded into energy markets, ancillary services markets, forward capacity markets, and financial transmission rights markets. Energy market, as the name implies, is a market used to match supply and demand for electricity. After the market clears, the price of electricity for each node is obtained. In order to ensure the safe operation of the power system, frequency stability, ancillary services are necessary. Eligible units can participate in the ancillary services market and provide necessary support following the system operator’s dispatch instructions. The establishment of a forward capacity market is to ensure the adequacy of generation resources, to maintain the safe operation of the system when future customer demand for electricity increases. Financial transmission rights have a close relationship with the transmission network, and are financial tools established for market participants to hedge price fluctuation risks. It should be noted that the designs of the seven markets of NYISO, NE-ISO, PJM, MISO, CAISO, SPP, ERCOT are different, not every ISO/RTO has a forward capacity market.
Optimization Across Day-Ahead and Real-Time Markets
One of the most important functions of independent system operators is to maximize social welfare while ensuring the safe and stable operation of the power grid. In the language of operations research, it means minimizing generation costs and pollution emissions while satisfying a series of constraints previously mentioned, even in the event of an unexpected failure of any critical transmission device (in reference to the N-1 Contingency standard established by NERC). The optimization process involved in the spot market mainly involves two aspects: the combination optimization of units based on security constraints in the day-ahead market (Security Constrained Unit Commitment) and the economic dispatch based on security constraints in the real-time market (Security Constrained Economic Dispatch). These two optimization processes are similar, both calculate the optimal generator status (for example, 1 is on and 0 is off). Because it also involves some non-integer parameters, this type of optimization is classified as a mixed integer programming problem. ISO needs to simulate every kind of unexpected failure scenario, so the computation is intense. Dr. Yonghong Chen of MISO has done a lot of work on this subject.
The California Power Crisis (2000-2001)
California, as one of the pioneers of electricity market reform in the United States, began operating under a market-oriented model in 1998. However, less than three years into operation, California experienced a severe power crisis. A combination of high temperatures and drought in the summer and cold waves in the winter caused power shortages, even blackouts. Electricity prices rose 800% from April 2000 to December 2000, Pacific Gas and Electric Company (PG&E) went bankrupt, and Southern California Edison (SCE) was also on the brink of bankruptcy, with economic losses of up to $40 billion to $45 billion. In retrospect of this crisis, in addition to natural disasters, there was human error: California’s power reform was too radical, with design errors that resulted in public power companies selling off a large number of power plants in accordance with government instructions; and delays in the process of building new power plants; Companies like Enron deliberately shut down some generators during power shortages to manipulate electricity prices for profit. Fortunately, after this crisis, both federal regulatory agencies and regional operating agencies did not give up, but actively learned from the experience, corrected mistakes, and continued to promote market-oriented reform.
Renewable Energy Tax Credits
The large-scale integration of renewable energy not only meets and reflects the public’s demands for environmental protection and technological progress, but also closely related to government tax incentives and subsidies. Here, we will briefly introduce a few of them. At the federal level, the United States has Production Tax Credit, Investment Tax Credit, Accelerated Depreciation and Loan Guarantee specifically for renewable energy construction investments; at the state level, governments provide cash incentives and discounts (Renewable Energy Certificates) for renewable energy projects. These financial and tax policies have greatly stimulated the integration of renewable energy such as wind and solar power.
The “Duck Curve”
When it comes to the grid integration of renewable energy, California’s “duck curve” cannot be ignored. The “duck curve” refers to the rapid growth of photovoltaic installed capacity, which leads to a net load (electricity demand minus renewable energy generation) on the grid that dips during the traditional peak electricity demand period. When the sun sets in the evening, the net load on the power grid rapidly increases, and the electricity demand and net load for the entire day form a pattern similar to a duck. California has the largest installed solar capacity among all ISOs, so this phenomenon is particularly obvious. Currently, one of the more feasible solutions is large-scale energy storage equipment, such as Tesla’s battery farm. To address this issue, CAISO also introduced the Flexible ramping service in November 2016.
The “GreenHat” and the Financial Risks in the Electricity Market
As previously mentioned, financial transmission rights were initially established as a financial instrument for market participants to hedge price fluctuation risks, but speculative behavior has also emerged. In June 2018, PJM suddenly discovered serious flaws in their regulations regarding trading collateral: two former traders of JPMorgan, Andrew Kittell and John Bartholomew, bought a large number of long-term financial transmission rights contracts through GreenHat, then sold the appreciated portion of the contracts and defaulted on the funds that needed to be paid for the losses. In this way, like a snowball, they raised the investment portfolio to 890 million MWh. And the collateral they paid for this was only 600,000 US dollars. Since 2016, the mark-to-market value of the contracts they held began to decline, GreenHat promised to make up the gap with the income from selling the contracts, but PJM did not conduct a thorough review. After selling the contracts, not only did GreenHat fail to fulfill its promise but doubled down and bought more financial transmission rights contracts, resulting in additional losses. Thus, due to GreenHat’s default, the lack of PJM’s risk management, a $10 million trading loss in 2017 became a huge loss valued between $185 million and $430 million (these losses were ultimately borne by other market participants). The chief financial officer of PJM resigned after this incident was exposed, and the chief executive officer of PJM also revealed plans to retire early, but discussions and reforms regarding this incident are still ongoing.