Exploring how ESOO has fared as a blackout barometer

The Australian Energy Market Operator’s (AEMO) Electricity Statement of Opportunities (ESOO) provides us with a detailed but nuanced look at demand, generation, and reliability projections for the decade ahead.

With the 2024 edition scheduled to be published next week, I’ve undertaken a small project over the past couple of days to review the full history of the report. I’ve done this to share some insights into what has changed over the years, and what hasn’t.

What has changed: the methodology and the purpose

The ESOO has been published annually since the start of the National Electricity Market (NEM) in 1998. Its original purpose was to inform market stakeholders about the adequacy of supply to meet demand in coming years – hence the ‘statement of opportunities’ moniker.

Electronic versions from 2010 onwards can be found in the AEMO archives. We have a dusty collection of older physical copies on our bookshelves in the Global-Roam office. And conveniently, these older editions included a CD-ROM (which still work!) containing the spreadsheet model used for the reliability outlook projections.

As you would expect, the ESOO modelling methodology has come a long way in 26 years.

Amongst other complexities, the energy transition has necessitated the electricity system to: 1) have a greater dependency on weather, and 2) be designed to handle a wider range of contingency events. These new realities forced the AEMO to move from deterministic modelling, towards probabilistic methods in the mid-2010s.

The current methodology is based on time-sequential modelling over each financial year at a one-hour granularity, using Monte Carlo simulations of security-constrained optimal dispatch. The results from these simulations are probability-weighted and assessed against the reliability standard.

The reliability standard in the NEM has always been 99.998%. Simply stated, this means that only 0.002% of demand should be allowed to be unserved by supply each financial year (the equivalent of approximately 10.5 minutes of outages for the average consumer).

However, two policy developments in the late-2010s moved the goalposts, and muddied the purpose of the ESOO:

• In October 2018, the since-disbanded COAG agreed to implement the Retailer Reliability Obligation (RRO). Read Ben Skinner’s excellent summary here if you want to understand all of the gory details of what it is, how it came to be, and the compliance obligations it entails.
• In November 2019, the since-disbanded Energy Security Board recommended energy ministers introduce the Interim Reliability Measure (IRM) – a much stricter threshold for unserved energy, of 0.0006%. This then became a trigger for the RRO in place of the original reliability standard.

These two changes meant that the results from the reliability outlook became even more sensitive, and began to act as an initiator for real-world compliance obligations.

And finally, it’s also worth noting the increasing use of the RERT mechanism over a similar timeframe. Last year we saw RERT tenders initiated almost immediately after the release of the ESOO.

What hasn’t changed: mis-forecasts and modelled supply shortfalls

There are a few common threads that have become apparent to me from skimming through 20-plus editions of the ESOO over the past couple of days.

Firstly, it is clear that demand has been over-forecasted in some historical instances and, albeit less frequently, under-forecasted in others.

I would argue that these prior mis-forecasts are less to do with a lack model sophistication, and more to do with the input assumptions regarding the underlying drivers, made up of:

• Under-predicted factors (i.e. not always anticipated, but did eventuate) – such as the uptake of rooftop solar, increasing energy efficiency, an LNG industry in Queensland, the timing of Hazelwood’s closure, etc.
• Over-predicted factors (i.e. anticipated, but did not eventuate) – such as policy continuity, a long-term emissions trading scheme, the timing of Eraring’s closure, etc.

So all models are wrong, but some are useful – that’s probably not particularly enlightening for long-time readers of the WattClarity blog.

My second and more important observation is that the ESOO has a long history of projecting supply shortfalls. The first image in the series of three above illustrates that this was true even two decades ago. And despite these projections, the NEM has, largely, maintained high levels of reliability since inception.

So does that mean that the AEMO are really awful at forecasting, and always have been? No.

The report is meant to inform the market, so it is an intentional feedback loop. If it informs the market of how to act, then it has achieved its primary purpose – even if that means that what then unfolds is different than the forecast.

This in itself therefore almost guarantees that the reliability outlook will be ‘inaccurate’.

So then, how has the ESOO fared as a blackout barometer?

Somehow the release of the ESOO has triggered an annual tradition of journalists and social media commentators misinterpreting its results as a ‘blackout predictor’ of sorts. It is a tradition that is almost as old as the ESOO itself.

If you were to believe that modelled supply shortfalls = incoming blackouts, then you might have already burned through your candle supply waiting in vain.

In the next five charts I display the number of years until a supply shortfall for each region – based on the central/neutral/medium reliability outlook in each ESOO over the past 20 years.

And now for reference, below is the most recent data that I could find showing actual unserved energy levels in the NEM over a similar time range.

Keep in mind that the ESOO is not the only lever that the AEMO has to manage reliability.

But of course, the link between blackouts and the NEM’s reliability standard being breached is a bit of a silly oversimplification in the first place. As others such as Dylan McConnell have written about, distribution-level outages are a much greater source of blackout incidences for the everyday electricity consumer.

Key takeaways

It is not a recent phenomenon that the ESOO has projected supply shortfalls, and yet we’ve seen very few historically. There should be some lessons from this, so I’ll finish with two key points:

• Like many of the AEMO’s forecasts, the reliability outlook is intended to elicit a market response. The accuracy of a rain forecast is independent of whether you decide to bring an umbrella when you leave the house. Forecasts like the reliability outlook in the ESOO are not divorced from the outcome in such a way. This is an important reason why we always need to think about that other dimension of time.
• And further, it’s in the AEMO’s self-interest to avoid a supply shortfall. That’s one of the main reasons why the AEMO exists – and you can bet good money that they will shoulder the blame if a supply shortfall eventuates. To quote a former NSW premier: “Always back the horse called self-interest, because you’ll at least know it’s always trying.”

My intention here is not to downplay the urgent and obvious need for new supply, but rather to highlight that the ESOO holds up as a very poor leading indicator of blackouts … despite what the mainstream media and social media commentators might try to imply.

Author: Dan Lee, Market Analyst, Global-Roam

Popular content

Queensland solar farm levels up with environmental approval

19 September 2024 The 300 MW Cambridge Solar Farm and battery energy storage system proposed for north Queensland has received an environmental tick of approval by the...