PART I In Search of a Logical Retail Market Structure: Is the NYREV DSO the Answer?

James McGinniss
May 25, 2018 · 15 min read

Renewables and other distributed assets are forcing us to entirely rethink how we approach grid architecture and market structures. Their variable nature and ability to collocate loads to generation at an impressively granular level is not only revolutionary, but requires us to extend the liberalization of energy markets to the retail (distribution) level in order to make our system overall more dynamic. This, by definition, will lead to a disruption of the business models of legacy institutions, especially that of the utility — just how PURPA did at the wholesale level over time. The changes will be gradual, but they will indeed be substantial.

Currently, the most ambitious market restructuring program being undertaken is New York’s Reforming the Energy Vision, or NYREV, which has a stated outcome of creating a fully functioning retail market, much like the wholesale markets of today. While the details will be explored shortly, the aim is to achieve this through a Distributed System Operator (DSO) run by the utility, an analog to the Independent System Operator (ISO) of wholesale markets. Furthermore, while NY is the first state to attempt this, the predominant thinking in energy wonk circles is that the DSO model will be adopted more and more over time, much — again — how wholesale markets developed region to region.

The problem is, though, that retail markets are decidedly different than wholesale markets, with different players and legacy institutions sitting at the table, different problems to address, and different technological constraints. Thus, a one-to-one projection of wholesale markets onto retail markets, despite how successful wholesale markets have indeed been, is unlikely. What’s worse, the trajectory of the NYREV DSO is not at all that of an ISO, which must be addressed in the conversation. The DSO model as it is currently set up, due to the role of the utility, is deeply problematic for the following reasons: inadequate market segmentation, flawed incentive structures, and a lack of consideration surrounding the implications of franchise rights.

We Need a Retail Level Marketplace

With all the excited talk about prosumers (retail costumers who also produce power) and distributed generation, currently the only real market mechanism on the distribution grid is a scratch on the surface of what is necessary for properly integrating renewables. It’s a good start, but Net Energy Metering (NEM) is a blunt instrument that will not stand on its own two legs for long. As more rooftop solar comes online, compensating prosumers with a one-to-one retail credit, irrespective of the time of day (i.e. not matching wholesale fluctuations), becomes unsustainable.

This is because variability on the electric grid due to high renewables penetration is an immediate concern that will only increase in severity, especially if we want the skyhigh growth rates to continue in our efforts to decarbonize the electric grid. In fact, two market issues have already presented themselves, even at relatively low penetration rates. These issues have been covered in depth and, at this point, should be obvious. But, a brief recap is in order.

The first is the CAISO Duck Curve, which causes a very steep ramp-rate — the rate at which generators must come online to meet increasing demand. That is, solar panels stop producing right when the grid experiences peak demand as everyone gets home from work between 5–7PM. This is why we need Time of Use (TOU) rates — retail rates that vary based on the time of day —for both buying and selling, instead of simple NEM. Tiered TOU rates are only an intermediary step toward real-time retail markets. TOU rates in turn will, among other market mechanisms like Demand Response, incentivize collocated battery storage installations to shift solar energy supply to the times that it is most needed. In essence, it will be market signals, and not simply cheaper batteries, that will help flatten out the Duck Curve.

The now infamous Duck Curve. The effect in reality has been worse than what was originally projected.

The second is zero or negative marginal cost assets (solar and wind, respectively) disrupting wholesale market functions. Wholesale markets were designed to have generators bid in at their break-even (marginal) cost of operating, but with the marginal cost of renewables being zero (due in part to solar’s Investment Tax Credit) or negative (due to wind’s Production Tax Credit), their bid is always accepted. This has consequences for how the grid operates as more reliable generators are pushed out of the bid stack, and means that we must find a way to reward reliability measures, such as black start capabilities, frequency regulation, and demand response at the wholesale (and eventually retail) level.

The key here is granularity; coordination of data from these smaller, more variable, distributed (many more nodes in many more places) resources becomes the problem of building the future grid. Distributed generation means smaller, more flexible generators, instead of the large centralized power plants that we have always built. Most new generation being built will be 1kW-300MW, not the 300MW-1GW+ behemoth plants of old. Currently, wholesale markets in ERCOT (Texas) settle on 15 minute intervals with power producers only 1MW and larger. Yet batteries respond in subseconds and can be the size of a microwave. This is an entirely different grid we’re starting to build.

More variable renewables on the grid, especially at the distribution level, leads to a need for energy storage, EVs, and other dynamically trading assets. And dynamic assets require dynamic markets. Thus, distribution level markets are necessary to coordinate all of this information, allowing capital to be deployed efficiently and at scale.

NYREV: An Introduction to the DSO

If you’re not yet convinced that we need a retail level marketplace, then take NY regulators’ word for it. NYREV is a state-wide regulatory and incentive program aimed at building “the grid of the future” and is attempting to address these problems of renewables integration in what is one of the boldest experiments we’ve ever run on our electric grid. The stated hope is to bring fully functioning transactional markets to the distribution level, essentially mandating utilities to liberalize retail markets. This will come first in the form of tariffs (Stage 2 in the image below), before a fully functioning retail market (Stage 3), mediated by a DSO.

ConEdison NYREV Roadmap for their Distributed System Platform Plan, or DSP

The NYREV tariff approach, set to begin taking effect in 2018, will go the way of NEM; it will get us started, but it will not keep us going. So what, then, does a DSO really look like? Currently, NY utilities are mandated to establish DSP’s, or Distributed System Platform Providers, platforms for setting market prices and providing information to distribution grid project developers. This is the NYREV version of a DSO (DSP=DSO). These signals will come in the form of tariff structures, called the Value of Distributed Energy Resources (VDERs) that reward projects for performance in five different ways: with environmental (carbon reduction), capacity (kw), energy (kwh), demand reduction, and locational (demand reduction on troubled distribution feeders) credits. Utilities will post “heat maps” of problem feeder areas and release capacity hosting analyses of feeders for developers.

These plans are clearly far more granular than NEM, but are still widely thought to not go far enough; they are an obvious intermediary step towards a fully functioning distribution level market, as detailed in the roadmap image earlier in the article. The idea is that tariffs will jump start behind-the-meter DG projects before the utility can start running fully functional, more real-time markets. As this is a grand experiment, the utilities are still coordinating these plans, and must, to make sure that the grid remains reliable and that projects are being properly deployed. However, as we hash out as a collective of parties what sorts of projects work best at the distribution level, and how they should be rewarded, we must ask ourselves what role the utilities should be playing. But before we can do that, let’s take a look at how wholesale markets work.

Wholesale Markets and the Role of the ISO

Wholesale markets are segmented into a few distinct categories: the ISO (reliability and market settlement), sellers (power producers/plants), buyers (distributors — what we call utilities — and large enough single users such as manufacturers), and Transmission and Distribution Service Providers (TDSP’s — I know, it’s confusing, but they have nothing to do with the DSP referenced by NYREV). Transmission Service Providers generally build and maintain wires at the wholesale level, while Distribution Service Providers act more towards the retail level — the “telephone” wires you’re used to seeing — and are frequently the utility themselves.

Wholesale, high-voltage infrastructure vs. local, retail distribution infrastructure. Wholesale infrastructure is generally built, maintained, and owned by third parties, whereas distribution infrastructure is mostly still owned by utilities.

In wholesale markets, reliability and bid matching is performed by this pseudo-governmental body, the ISO. The ISO/RTO (ISO’s are sometimes called Regional Transmission Operators) is a benevolent actor on the wholesale grid, by state or region, matching bids for demand from utilities and industries with bids from generators, and are mandated to set the lowest possible price for the bid stack. Basically, if the ISO receives 35 GW worth of bids for demand, they pick the 35GW of cheapest offers from generators. In fact, the ISO has a mandate from the Public Utilities/Services Commission (PUC or PSC) and the Federal Energy Regulatory Commission (FERC) to settle a bid stack at the lowest possible price. Because ISO’s and RTO’s are not investor owned and are overseen by regulatory bodies, we can trust them to do this. They have no incentive other than to keep the grid running reliably at the lowest possible price. They are a benevolent watchdog.

ERCOT bid stack, which settles every 15 minutes. You can see that as demand grows, the more expensive resources are included in the bid stack. The vertical line is where the market settles, and all resources to the left of it are paid the price of the resource it settles at. Note that “other” is renewables at zero or negative marginal cost.

Transmission Service Providers own the infrastructure that the wholesale energy is traded on, acting as toll men. In order to build infrastructure and get it funded, a lengthy bidding and approval process is overseen by the PUC. Private generators transport their power on these transmission lines, and the utilities then distribute this power to their customers on their own distribution network.

This ISO market model has proven to function very well indeed. Thus, keep in mind the following important facts about wholesale markets: the ISO does not own generators, does not get paid for selling power, and does not own any power lines. Furthermore, unlike the distribution grid, the wholesale market is not bidirectional. That is, the idea of “prosumers” is far less prevalent on the transmission network.

The DSO: What Kind of Market Are We Actually Trying to Build Here?

A DSO should act exactly as the ISO/RTO does at the wholesale level, but legacy institutions are not currently letting that happen. Utilities are investor owned with the following two pillars of their business model: payment settlement (they receive a markup on every kwh of energy sold) and poles and wires (they receive a fixed return on capital for infrastructure investments deemed necessary). While they perform other important tasks, such as reliability and grid balancing, those are the two by which they are most financially incentivized.

The DSO led distribution grid — if it is anything like the ISO that people relate to it — must become comprised of three distinct categories: poles and wires (Distribution Service Providers), reliability (the DSO), and prosumers/consumers (buyers and sellers). But currently, the utility will be involved in all three of those categories, unlike what wholesale markets look like. In fact, this is stated explicitly:

“ The NYREV DSP and [the] DSO are very similar concepts, and in its recent Order the NYPSC states that the DSP core functions would be highly integrated with utility planning and system operations, and assigning them to an independent party would be redundant, inefficient and unnecessarily costly.”

I am going to argue over the course of four distinct articles, this being the first, that assigning it to a third party is absolutely necessary — at a minimum. Along with stating how the DSO could look, I will also discuss the role of infrastructure and how the modern utility will potentially form. For now, we will simply demonstrate why the DSO or DSP as stated by NYREV is a bad idea, from a market standpoint.

In order to build a market with proper incentives, the utility will have to decide if it wants to set bids or fulfill them, and we will have to decide if them owning the poles and wires, especially with franchise rights still in effect in many areas, becomes a conflict of interest. Otherwise we are building a market with some very poor incentive structures, one that does not at all resemble the functioning ISO model.

Thus, there are three overarching concerns in the creation of a DSO, or more specifically, the NYREV DSP. These concerns also coincide with the differences already showing between a DSO and an ISO. Inadequate market segmentation, franchise rights, and flawed incentive structures are three solvable problems that must be addressed by NYREV regulators.

Inadequate Market Segmentation

As previously stated, electricity markets, in order to be effective, need be segmented into three different categories: infrastructure, reliability, and payment settlement. Currently, the NYREV solution is to have the utility perform all three tasks. The only positive indication of the NYREV push is that utilities are not allowed to own behind-the-meter assets; the goal is to stimulate private and third-party development of distributed generating assets on distribution feeders. But not all markets are created equal, and we have not collectively thought out the implications of a distribution level market. When DG was not so prevalent, it made sense to a degree for utilities to handle both distribution and payment settlement from a reliability standpoint. But now that power producers are entering the distribution grid, it may not make so much sense.

Currently, the NYREV DSO will have the following structure: Utilities submit tariff proposals to the PUC for compensation of DG projects. The utilities decide what value distributed resources will have in their territory. Many utilities, like ConEdison in NYC, own the wires that these bids are being settled on. Lastly, they still procure power at the wholesale level and sell it to costumers. Thus the DG projects they’re enabling are direct retail competitors with one pillar of their business model.

Remember that utilities are investor owned, and are not benevolent watchdogs. How do we know, then, that the tariffs they are structuring are accurate, when they’re not responding to a dynamic market? What does a a so-called dynamic market look like once the tariffs get phased out? A utility could easily offer a lower-than-fair-market price for power, and then sell it to the producer’s neighbor at a premium. All they have to do is convince regulators what “fair” is. Even worse, because of franchise rights, the power producer has no other option than to sell to the utility.

Franchise Rights

A dynamic, transactive market cannot properly exist if regulators write off infrastructure as unimportant. Some utilities, like ConEdison, have exclusive rights to build infrastructure in a given area. The broader problem that this creates is addressed in full in another piece written by this author. It can be read here.

Essentially, in a distributed generation era, where local generation sources can power entire campuses, developments, and by extension city blocks or neighborhoods, there must be alternative means of building local infrastructure that are not owned by an entity by divine right. The ability to collocate energy to supply, with very short distances between generators and loads, means that this artificial constraint on building such local infrastructure will actually have an effect on market prices themselves. The removal of franchise rights will allow local systems to self-organize, while still interconnecting to the bulk grid.

If we want to call the DSO an ISO analog, then we must recognize that the TDSP’s of wholesale markets absolutely do not have franchise rights. In competitive retail areas, which do indeed exist in NY already, utilities compete for customers on top of infrastructure that they do not own. However, this must happen unilaterally. If public poles and wires are owned, no wholesale procurement can take place. And vice versa.

Flawed Incentive Structures

The reason that power procurement and power distribution should become segmented is because of incentives. By now, this should be clear: for a single financially incentivized entity to have the power to set bids and to fulfill them, all while having exclusive rights to the pathways upon which the good is being traded, is a marketplace with obscenely foolish incentive structures. Having a non-financially incentivized benevolent watchdog capable of taking care of reliability standards and market settlement is indeed a functioning market.

Behind-the-meter generation is a direct, existential threat to the utility payment settlement model. It makes them sell fewer kwh. Period. However, bidirectional, transactive distribution grids may actually improve the polls and wires model. If I am a toll man, I want as many people crossing my bridge in both directions as possible. But if I am a toll man, and I get paid more for certain people walking in one direction than others walking in another direction, how can I be trusted to be benevolent? Furthermore, if all other toll men are barred from building a new bridge next to me, then how would these prices reflect the fair price of building and operating a bridge?

This is what a utility-controlled DSO model while the utility is still allowed to both own polls and wires and procure wholesale power looks like. Because what this means is that if I wanted to build a multi-block microgrid in, say, Brooklyn, I would have to use the utility’s infrastructure, even if I could prove that I would provide better service to the customers by building an isolated grid. This is where the utility derives the majority of their power, and coupled with them running the DSO, leads to an illogical retail market structure. This is to say that, simply, the financial self-interest of the utility conflicts directly with an efficient and fair market structure.


The only argument for the NYREV DSP model is that, for reliability’s sake when dealing with a bidirectional grid, utilities must coordinate distribution level demand with wholesale market generation. Obviously, we will not be able to build enough generating assets in a place like NYC to fulfill all power requirements. So, how do we both compensate distributed assets at the retail level fairly, and appropriately leverage them into the bulk grid? Hint: the NYREV DSP is not the only way, or nearly the best way, that that can be accomplished.

Technologically, bidrectional grids are more than possible, and do not in their own right mandate naturally that the utility must handle retail to wholesale market “communication”. That can happen via aggregators, but this point will also be expanded on in a later piece. For now, notice that the NYPSC quote referenced earlier states that an independent DSO would be “redundant, inefficient, and expensive”, but does not say anything about being unreliable. This is because we know that independent ISO’s work very well from a reliability standpoint indeed, and bidirectionality should not change that.

We could let utilities keep poles and wires, payment settling, and play the role of DSO by regulating the heck out of them, forcing them to play ball in the way we want them to, but what would this accomplish? This will not lead to accurate prices and compensation of the true value of distributed resources in a dynamic market. Instead, if we’re going to build a “market”, we should build a market that’s properly designed. Like wholesale markets. One that is segmented and has competition both at the infrastructure and payment settlement levels.

Yet, there’s hope. The NYREV DSP, even if acted out as proposed, will still lead us to a segmented market structure in time. I say this because that’s how it happened with wholesale markets. PURPA in 1978 allowed generators to start selling power on the utility owned wholesale grid at a rate of “avoided costs” set by the utility, just how NYREV is allowing third parties to sell on the distribution grid, with utilities still in control. Over time, this tariff structure will break down, and a transactional market will not be able to evolve without major changes. By the new millennium, wholesale markets became liberalized completely, and the ISO emerged. Hopefully, by 2040, we’ll be saying the same thing about retail markets.

At the very least, those using the ISO model to justify a DSO must recognize first that they are currently not equivalent ideas. What’s worse, this problem is not limited to NYREV: it will be true anywhere the legacy utilities possess the same amount of power. Still, I remain unconcerned, due to the logic of the wheels we’ve set in motion.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store