Back to basics: Some Definitions

Today I am restating some definitions of terms, reflecting how my own understanding has changed over time.

Transactive Energy

Transactive Energy (TE) is the term for an energy balancing approach using economic techniques for dynamic balance of supply and demand within energy and power grids.

TE is a particular case of Transactive Resource Management which uses markets to allocate any commodity (resource) whose value is determined solely by its time of delivery. Transactive Resource Management was first described at the Xerox Palo Alto Research Center. The GridWise Olympic Peninsula Testbed Demonstration Project was the first transactive active energy field experiment in 2008.Transactive Energy was described by Edward Cazalet in a number of papers and talks culminating in the reference book “Transactive Energy: A Sustainable Business and Regulatory Model for Electricity” (Barrager & Cazalet, 2014).

Transactive techniques allow dynamic balance of supply and demand where energy is in surplus or shortage, in contrast to traditional techniques which address surplus less effectively. Transactive Energy (TE) is seen as a fundamental organizing principle of future smart grids.

Microgrids & Micromarkets

Microgrids are independent systems that manage the distribution of power. Microgrid components may supply, consume, or store electric power. TE can be paired with microgrids to make decisions about power management and distribution without constraining technology choice or technology evolution inside each microgrid component.Micro-markets require physical delivery of the product or service. Micro-grids allow (and in fact are defined by) the ability to shift energy and power within them. There is a useful symmetry of managing the balance of supply and demand in a micro-grid by means of a co-extensive micro-market.

Participants in more than one micro-grid must be able to deliver and receive the products bought and sold. North American markets typically distinguish between transmission (longer distances) and distribution (shorter distances, more end points) with different regulatory regimes. Micro-grids (composed or standalone) can be structured to allow avoidance of complex regulations designed for much larger scale enterprises.

Common Transactive Services

The interactions of Transactive Energy were defined in the Energy Interoperation Specification (OASIS, 2012). The Common Transactive Services (CTS) refers to a minimal set of standardized services originally developed as part of the NIST Transactive Energy Challenge.

The Market Code resource uses CTS-defined messages for all interactions between components. CTS defines a restricted profile of Energy Interoperation that can interoperate with each of the transactive systems within a microgrid. CTS includes minimal extensions and is at an architectural level appropriate to the semantics of all transactive systems minimized for local micromarkets, and able to enable decoupled evolution. A recent project of NIST and the Energy Mashup Lab made the first use of CTS. The Energy Mashup Lab (http://www.theenergymashuplab.org/) (EML). The Lab is a non-profit (501C3) whose purpose is the development and promulgation of microgrids through promoting open source software for TE based microgrids.

Cyber-Physical Systems

Cyber-physical systems (CPS) are systems built around co-engineered interacting networks of physical and computational (IT) components. The term CPS includes the overlapping technologies referred to as the Internet of Things (IoT), the Industrial Internet, and Operational Technology (OT). CPS applications in specific economic sectors are referred to with the terms smart manufacturing, smart transportation (including autonomous vehicles), smart healthcare, and smart energy. (NIST Cyber-Physical Systems Public Working Group , 2017).

Many CPS applications are inherently distributed and equipped with wire-bound or wireless communication facilities. When their components are largely autonomous, the organizing principle is coordination rather than control. A CPS providing critical services such as dynamic and prospective traffic safety, factory and process control, or healthcare needs to be highly dependable requiring the availability of reliability performance, availability, safety, and security.

Any consideration of CPS always includes the physical. Whatever software they run, their physical actions are constrained by physics: mass, momentum, chemistry, biology, and, for TE, electricity. When CPS are deployed in infrastructure, they will likely operate in place longer than typical IT systems. With long-life comes diversity; even if one built a CPS monoculture, with common ownership and a single vendor, the evolution of products and technology over time would lead to diversity of components.

This inevitable necessary internal diversity of components within a CPS rewards an abstract or service-oriented model for CPS integration. Service-Oriented Architectures (SOA) coordinate systems not by orchestrating processes, but by coordinating effects. For TE, one can understand this not as “turn off pump #3” but instead “use less power for 10 minutes”, not as “charge battery” but as “power is currently available”. This consideration points to the desirability of integrating CPS using abstract coordination such as CTS.

Many CPS are social systems. Autonomous cars drive to human chosen destinations. Autonomous domotic power systems must balance domestic services while they balance power and price. Even factory systems may respond to labor shifts and human-provided maintenance schedules. The social component means that few CPS can be “set-and-forget” after install, but instead must respond to and provide services to humans.

Spontaneous Order on a Continental Scale

A recent conversation about European power markets and some “glitches” in early June shown a light on profound issues in cybersecurity, in system architectures for big infrastructure, and to an extent the scalability problems with many of the hottest applications for the Internet of Things (IOT).

The specific observations was a plea for direct central control, even as it used an example that showed the shortcoming of infrastructure architecture based on assumptions of central control. It then learned the wrong lesson, that spontaneous order is too “risky” at large scale.

>>> Something went wrong on the 6., 12. and 25. June 2019.
>>> The belief in the Market to fix everything ... may end up in a big
>>> blackout.
>>>
>>> Add-On (2019-07-03):
>>> Today I found more details on the likely reason why we were so close
>>> to big trouble:
>>>
>>> "Due to a faulty data package, the European electricity
>>> exchange EPEX in Paris decoupled the European
>>> electricity market on June 7, 2019. This caused a great
>>> deal of excitement on the markets. Johannes Päffgen,
>>> Head of Energy Trading at Next Kraftwerke, explains the
>>> causes and consequences in an interview.
>>>
>>> Christian Sperling: Johannes - What happened? Why
>>> was there so much trouble at EPEX on the Friday before
>>> the Whitsun holidays?
>>>
>>> Johannes Päffgen: Well - in the end it's a computer error...
>>> but we should go into that later. At about 11:40 this Friday
>>> we noticed that something was wrong at EPEX.
>>> We couldn't place any more bids for the day-ahead electricity
>>> auction on Saturday. ..."
>>>
>>> I guess it was a human error ... somebody didn't take into account
>>> that corrupted data packages will be sent and received ... how could
>>> a faulty package have such a dangerous result?!?!
>>>

While Transactive Energy is superficially similar to the way the bulk power markets have long operated, the power of TE is in local markets. The first benefit of TE is to hide the control complexity/diversity of different technologies behind common signaling. The second benefit is to permit diversity of motivation of each participant in the TE market, as those are also hidden behind the common signals. The power of TE is to allow an emergent order to arise, with balancing of supply and demand occurring without respect to technology or control system or personal beliefs.

One can think of TE as embracing that the Knowledge Problem described by Economics applies to the world of things as well, and that we can use markets, i.e., small decisions made by the participants to participate or not at each moment, to solve power availability without central control. The evolution of life on Earth, of language, of the brain, and of a free market economy are considered systems which evolved through spontaneous order. Naturalists often point to the inherent "watch-like" precision of uncultivated ecosystems and to the universe itself as ultimate examples of this phenomenon.

TE implementations must be aligned with the newer methodology of Laminar Control. Mid-level lamina can coordinate lower level nodes, but do not reach in to provide direct controls. Lamina may however share situation awareness, local effects up, wider area conditions down, to improve the decision-making within each. No Lamina requires the situation awareness of the adjacent lamina.

This has important implications for security and for future technological evolution of power systems on the grid. Aside from the very top level, all lamina are discontinuous. The layer that controls one neighborhood is not actually connected to the controls of a nearby neighborhood except through a common higher level lamina.

The loose coupling of component systems based on abstract communications is characterized as an anti-fragile software pattern. Lightly managed systems coordinated by abstract communications create spontaneous order. Spontaneous orders are distinguished as being scale-free networks, as opposed to the hierarchical networks traditionally used in power distribution management. Spontaneous order is defined as the result of actions, not of design.

For anti-fragile patterns to create resilience and stability, their interactions must be properly scoped so at to not create additional dependencies that create fragility. For TE, this means that not only must the market be local, consistent with the grid lamina, but each market must not rely on additional fragile elements. Making local decisions directly dependent on the communications infrastructure and market infrastructure far away, say at EPEX in Paris, reduces grid resiliency and introduces new cybersecurity challenges.

Besides, the grid is not Magic, and one really cannot buy power from Castille in Antwerp absent the power transmission capability to support such local delivery.

The markets of Transactive Energy will work best when they are based on local markets, able to balance not only power but voltage and frequency within the local distribution loop. Another market may use TE in the district, managing flows between the local distribution systems, and, again, not requiring detailed knowledge of what is inside each. Ideally the market for each will be collocated with the nodes and the controls for each.

Loosely coupled systems in organized in an anti-fragile pattern are manage by objectives and for results. They have no need to expose their internal operations or controls. From a security perspective, this greatly reduces potential attack surfaces. From a policy perspective, this reduces barriers to rapid future introduction of new technologies into a system of systems.

ASHRAE finished defining the Facility/Smart Grid Information Model (FSGIM) some years ago to describe what a Facility should know about itself to participate in these distributed local markets (ASHRAE 201). The abstract information model is consistent with the information model of the Transactive Energy market operations. A Facility that knows its FSGIM, is ready to participate in the local market. Local distribution markets can then replace the wasteful statistical and historic models that manage local power delivery today.

From the SCADA Security perspective, this model moves intrinsically toward defense in depth. From a social and organizational level, each market is a move toward liquid democracy as neighborhoods with their own goals interact with the wider grid. From a technology market perspective, this enables more rapid introduction of new technologies, including those of distributed generation and storage.

ESIF and Security at the Edge of Smart Grids

I attended the NREL ESIF Cybersecurity Workshop last month. ESIF names the Energy Systems Integration Facility. The workshop demonstrated both what should be done to secure future energy systems, and how difficult, labor intensive, and non-scalable this is using standard practice.

The first morning showed off the ESIF’s model of how to secure the un-securable. Using a rat’s nest of proprietary products, all communications to and from every sensor were firewalled and only specific interactions enabled. No messages were encrypted so every message could be inspected for appropriateness. The security infrastructure was itself secured and logged.

The rest of the conference aimed at specific interoperable approaches to accomplish the goals of securing Operational Technology or OT.

Part of the problem with securing OT is a fundamentally outmoded approach to operation. At a time when computing was expensive, phone lines cheap, and data logging infrequent, a model developed of putting every sensor and every actuator directly connected to a single computer. This model has long been named SCADA (Supervisory Control and Data Acquisition).

Two things happened to break the SCADA model. Phone companies moved out of the business of providing actual wires to connect sites, and moved toward shared networks. SCADA systems have never been fully secure in shared networks. Systems became more complex, and required faster response. In power distribution, this is due to a combination smaller operating margins (excess power available at every moment), more systems to control, including smart meters, and the arrival of distributed energy resources (DER).

As we move further into DER, we will see more diversity in ownership and in technology.

An owner of an expensive power production or storage system in a microgrid will want to operate it for their own benefit. As sophisticated owners add their own local monitoring and control software, they will begin to see how often remote operators mis-operate the locally-owned equipment, increasing maintenance requirements while shortening its life.

Distributed ownership and operation will also move toward diverse technology. A local owner will make his own investment decisions, and a remote operator such as a distribution utility may not know how to operate it. From the earliest efforts by utilities to tell owners operate buildings, following the energy price shocks of 1973, we have seen smart people forget that the primary purpose of a building system is not to provide managed load. (Consider the role of energy “efficiency” recommendations that did not consider health implications of short cycling HVAC in a Philadelphia Hotel in 1976).

The future of smart grids is on the edge, in autonomous systems that are built around a deep understanding of each buildings role and services. Edge based-operation offers both challenges and benefits to security. Incorporating systems with different ownership, and operated for different purposes makes security more complex. For now, regulatory mandates require that utilities still maintain detailed situation awareness into edge-based microgrids. Abstract interactions, including those based on the common transactive services, simplify security while reducing the attack surface. We will be rebalancing this border continually over the next decade.

The solution is abstract interactions between autonomous systems that can be locally operated and maintained. In power markets, this means that systems can negotiate whether to provide power or not, or to purchase power or not, while the inner workings of each system remain private. The interaction between the grid and a wind farm that occasionally sells power to the grid and a district associate that never buys power but occasionally sells it should be identical. Large system integration relies on integration using abstract communications, that is, the exchange of information that does not change often. Fragile or concrete information, such as the specific internal operations that are directly affected by changes in technology or equipment, are kept internal to the systems. This approach to integration is characterized as an “anti-fragile pattern”.

Until we reduce the attack surface, how will we increase security while increasing interaction? The ESIF security model requires too much hand-work, and does not support multiple ownership.

The Security Fabric Alliance has spent four years defining a more forward looking approach within the Object Management Group (OMG). OMG specifications are cookbooks for interoperable implementations of complex combinations of specifications by multiple vendors. The OMG Security Fabric, due out in February in 2018, incorporates best practices in military telemetry with directory-enabled security. Any communications must mutually authenticate before exchanging information. Despite this requirement, the Security Fabric has already been demonstrated in synchrophasor telemetry, a high volume, high frequency application. I look to the Fabric appearing in microgrids at the edge soon after its initial release.

Other efforts incorporate technologies to reduce wide area communications requirements and the effort to require detailed point-to-point security. Blockchain-style distributed immutable databases will replaces some requirements for remote data harvesting, and perhaps move into directory services to support security and policy. Edge-based Artificial Intelligence (AI) will reduce the manual set-up required for point-to-point and message-content based rules. I hope to write about these approaches later.

Architectural Principals of Transactive Energy

This post is part of the continuing Paths to Transactive Energy series. You can find them all listed by clicking on the matching metatag at the bottom of each post.

Transactive energy describes a pattern of integration where parties exchange the value or a commodity resource [power] over time and make forward commitments to sell or purchase that commodity. The Common Transactive Services (CTS) can be used in central auction-type systems, where a single entity announces or broadcasts prices or in markets were two or more parties come to a mutual agreement on price and delivery.

All forward transactions are committed, that is one party commits to delivering the service or commodity, one commits to buying it. If a provider wishes not to deliver, or if a purchaser wishes not to take delivery, they can participate in a separate negotiation, with a separate price, that can be netted against the original committed transaction. Such a buy-back resembles today’s Demand Response.

If one purchaser wishes to acquire more power at the last minute, and one wishes to acquire less, they can negotiate an exchange on the spot market. Different market structures and market rules will change the format, but not the substance of this transaction.

The CTS are essentially identical for any commodity resource or service. CTS works for transmission rights and ancillary services, as well as for other resource markets such as transactive water or transactive thermal markets. In each case, the product is delivery of the commodity at the designated time at the designated rate.

The CTS can work in many market structures. CTS can be used with a single (for the microgrid / micromarket) brokered trading floor or with peer-to-peer transactions. Compound transactions can link multiple simple transactions, such as paired transmission and delivery. Different circumstances will work best with different market structures, but in all cases, the communications can use the CTS.

1. Each party represents a node that acts in its own interests to support its own purposes.
2. The internal mechanisms and systems of a node are not communicated as part of the CTS.
3. The system of systems that make up a node may choose to organize some or part of their internal operations using transactive energy / transactive agents.
4. Actors inside a node interact with the internal market, not the external; there is no direct market interaction with things / markets / prices external to the node.
5. The purpose of an transactive node is to support the purposes of its owners and occupants, and not to support the things outside the node.
6. Economic signals or availability from outside the node might influence the market, if any, inside the node, but only as the market interface on the node relays that information. This may include markups, smoothing, discounts or any other means or mechanism that the owner of the node chooses to use (or that the maker of the system that operates the node chooses to use so that the owner of the node box will choose that system).
7. Parties external to the node should not use the possible existence of an economic entity inside the box as an excuse to penetrate the veil of the black box.