Actors for Buildings & Energy

Fifty-five years ago, we chose the wrong path. Small code that could run on small CPUs was recognized as a better path than large, probably single-threaded code. These elements of small code could be proven to be correct, unlike the large code programs that characterized mainframe-era programs. The software industry took a fork in the road, and led by the siren song of Intel CPUs, took the fork in the road of keeping the mainframe model of programming.

Nowhere was software hurt more by this than in building systems and IoT. These systems have a natural cadence based around independent actors for each mechanical subsystem, communicating in a service mesh. The lure of the large program you wrote last year being twice as fast next year due to Moore’s law, even without re-writing, was too enticing.

Somewhere around a decade ago, Moore’s Law hit the wall. We all pretend that it did not, based on renaming the old CPUs as “cores”, and packing a number of them on a chip. A well-threaded program can take advantage of all these cores. A few well known utilities programs do. If you look in the per-core performance on a laptop, it can look like a well-balanced threaded system. Too many of those CPUs are consumer apps, wastes of processing power on a control system, giving us the illusion of computational density. There is a better way.

Software in clouds have been moving toward swarms of smaller bits of code for some time. These actors are independent and arranged as need to meet business purposes and adapt to changing requirements without re-write. This style of programming is known as Cloud-Native computing (https://www.cncf.io/). Properly done it increases computational density for any system you have. It need not be just in “the cloud”.

Industry thought leaders such as Alper Uzmeller have long advocated for a more object-oriented approach to building controls. Actors have the independence and interoperability that objects promise, while having better manageability.

A simulation actor can be subscribed as a digital twin to the same message channel as the live system. This enables one to continuously compare results of the actual to the twin, whether for predictive maintenance or to detect cyberphysical security breaches. Small AI or ML actors can watch both twins continuously to create new insights.

For now this cloud-native computing style is mostly confined to the big cloud. Those premises that have their own on-site cloud can use the same code there. The cloud is moving to smaller ands smaller virtual machines for the actors (https://dapr.io/). Soon, not yet, but soon, real-time actors will be assigned their own cores to inhabit in multi-core CPUs on-site. The processing density of these multi-core systems will skyrocket.

This will bring us back to the Actor pattern from 1967. It is a natural fit for building systems and the Internet of Things.

It’s all about the connections

Angered and motivated by my experience preparing a large state university for Y2K, I made my public entrance to the public building systems space in 2002. Y2K was a crisis when it was anticipated that any program that used a two-digit year in the date (as in 99, and it was all of them) would fail after the year 2000 (when the year might be 01). State universities build using low bidders in accord with state construction law, and the University of North Carolina had accumulated a hodge-podge of systems for building operations, steam distribution, chill water distribution, cogeneration, and electricity purchases that barely interoperated. Worse still, the interoperations were fragile, and upgrading any one system would break the connections with any number of other systems. I simply wanted stable inter-system connections that did not break with any minor change to either system.

We were using system interoperation to address problems of smart energy. Back then, an operator would log into a utility web portal in each afternoon and download a CSV file with 24 power prices for the next day. We would then adjust the interactions of all these incompatible systems to align with the day’s prices. When the process broke without warning, we found that the file now included 96 15-minute prices. The utility had given us no warning. When asked, the utility replied that we should not worry, that they had no plans for 15-minute prices; but had merely upgraded their software. Connections without some sort of machine-readable contract are not reliable.

In the early 2000s, system interoperation meant XML and messages. Most accounting and line of business applications were exchanging XML. I worked with many industry leaders to define OBIX—which then became the heart interactions of the Niagara system and others. The effort made it easier for one HVAC system ti integrate with another, but was rarely used to enable enterprise interaction The whole building industry knew we needed an easier and more stable way to make connections between systems.

A decade later, the smart grid recognized that smart energy must be a conversation between buildings and power grids. Standards for M2M schedule negotiation, for energy market information, and for service-oriented energy came out of that, with a central place held by OASIS Energy Interoperation. OpenADR 2.0 and TEMIX are the two largest and most successful message exchanges based on that work. These connections work because they are requesting a single service, not trying to replace local control. Standard purpose-built connections help us connect systems, but only if they work for that single purpose.

Connecting power grids to building systems became easier, but I was consumed with connections with a smaller scope. Green Registrar’s Offices rely on interactions between class scheduling and building operations. Buildings adjacent to a BMS with a weather station all want to use that weather data to improve their own operations. BAS systems can tell physical security and emergency management systems if a building is occupied. Door locks and foot traffic systems can tell a BAS when to turn on. For three years, I worked on BIFER, Building Information For Emergency Responders, with target users from fire control to hazmat response. Each connection between systems increases the value of each system.

We have just begun to discover the lightweight interactions that should be easy to create and use. COEL-based applications would like to interact with conference room environmental controls to evaluate how alert attendees are before critical votes. Smart streets want to know when a mass of people is leaving a building. Easy-to-create connections are the path to create tenant value and to build smart cities.

Three years ago, Anto Budiardjo asked me to work with him to define mechanisms for defining and publishing limited connection points between systems. Anto was the first person that I was told to meet when I began work on OBIX. Anto’s new company is Padi, the Indonesian word for rice. Anto’s vision was to easily connect all the grains of rice in a bowl. Too many sophisticated interactions today are lost when one system or another is upgraded, and the original integrator is no longer on site. The mechanisms we defined had to not only be easy to use, but be repeatable, cybersecure, and self-documenting. We met with anyone who would listen.

Anto and I worked with the Digital Twin Consortium to build their model of systems of systems, work that was mostly defining capabilities for connections. Digital twins use intersystem connections to enable AI (artificial intelligence) and ML (machine learning) to constantly monitor cyberphysical systems. These tools can detect changes in configuration or performance by comparing actual performance of a system with a simulation, or with an emulation from yesterday, in real time. Connections between systems are the foundation of digital twins.

Related work, with a longer-range focus, is defining the future of the Internet, sometimes called Web 3.0, The Spatial Web, Architecture and Governance Working Group looks to combining the Internet of Things and the Internet of Systems at the edge, without required reliance on central monitoring and control. IEEE P2874 has many parts, from decentralized identity and security, to edge-based decision-making, to support for virtual and augmented reality (VR and AR). The Spatial Web will encompass ever-growing diversity of systems through use of common connection definitions.

The result of this work is the Connection Naming System / Connection Profiles (CNS/CP), a simple specification to create a control plane for the Internet of Things. (You can see the current draft at https://github.com/CNSCP/specification/blob/main/cns-cp.md.)  We have shared this work with the T2T (thing to thing) committee of the Internet Research Task force. We plan to submit CNS/CP to be a standard internet specification (RFC). CNS/CP will connect buildings to enterprises, systems to their twins, and maintenance personnel to augmented reality. Connections will continue to grow more pervasive and are central to future systems of systems.

We invite you to review the specification and provide feedback, comments, and suggestions. Let us know what you think.

The Last Big Thing

Developers of the Internet of Things always seems to be moving into the last big thing—at least as far as communications expectations and protocols. Too often security is an afterthought, something that can be bolted on afterward.

I often have to design secure communications for new deployments on a University campus. Many new roll-pits are still using RESTfull JSON. Remote systems often transfer telemetry to the cloud using unencrypted FTP. OpenADR generally uses reverse polling because corporate security won’t let external systems interact with on-premises systems secured with last generation security.

BACnet is moving closer to modern expectations with BACnet/SC. Control nodes and sensors can communicate using TLS-secured messages. Devices within the internal internet can work with certificates issued by the BACnet hub. Legacy systems can hide behind a BACnet hub and act AS IF they were secured.

Even so, older protocols and expectations sink in. BACnet router to BACnet application is still limited to Web Socket. ASHRAE specifies TLS 1.2 when many enterprises have moved to TLS 1.3. It is difficult to match the nimbleness of modern IT systems when putting in place systems that will not be replaced or re-programmed for a couple decades.

(Let me be clear here—my biggest complaint about BACnet SC is that I cannot yet deploy it. It is far more secure, and far better architected than what came before.)

Newer IT expectations are expected to continuously tune themselves based upon actual observed performance within their own environment. Applications that cannot do this on their own will end up sharing their data to cloud AI, with resulting loss of performance and loss of privacy and security. We all should know by now that data that goes to the cloud tends to get free in the cloud, offering the hacker or commercial competitor information for a decade. Once released, privacy never comes back.

Some IoT platform models have moved toward Docker. Docker provides a minimal Linux-like operating system (OS) to deploy code anywhere. I’m afraid that mainline IoT will get to Dockers just as the cloud moves to the next thing. On the edge, with the devices themselves, developer may wish to have multiple operating systems: one for Control, one for User Interface, one for AI. A Docker supporting Python for AI may require a lot of resources. Docker is and will remain to fat resource-demanding to support such applications on the edge.

I recently have seen some movement past Docker to DAPR (the Distributed Application Runtime). One can consider DAPR as a much lighter weight Docker. Different DAPR nodes are optimized for different languages. For example, there is a DAPR node pre-adapted to run the GO language (GOLANG or simply GO). GO is ideally suited to develop tiny replacements for Python AI routines. A GOLANG DAPR node can be much smaller and more efficient than is a Python routine on a Docker. Three DAPR nodes, one for control, one for AI based on GO, and one for UI based on .NET core can fit on a thermostat or other small system.

Upgrading some part of such a system, say upgrading the AI, could be as simple as swapping out the single DAPR node without touching the rest.

Don’t be slow to the last big thing. I recommend that smart building developers and smart energy developers consider what they might do with DAPR today.

Adjusting the Primal Forces of Transactive Energy

In a famous scene written by Paddy Chayefsky for the movie Network, Howard Beale is summoned by Arthur Jenson who explains to the crazed newsman that when he clings to his outmoded notions of how the world works, he is messing with the primal forces of nature. In transactive energy, the transactions are the only primal force, and notions of specific controls, and detailed manipulations of devices are out of date. Systems communicate only by the transactions, by prices to devices. To be consistence, future guidance to autonomous energy systems must be based on financial communications about energy transfers and energy resilience.

Transactive microgrids can balance energy supply and use over time with transactive energy. Internal prices adjust over time in response to supply and demand. No system needs to know anything about any other system, only what the market tells them. A transactive microgrid acts as a sealed system.

A microgrid manages energy resilience over time. An entire microgrid can act as a single system participating in in a larger microgrid, or interacting with a peer microgrid. A single node in the smaller microgrid also interacts with the larger microgrid. In each microgrid it interacts with, such a node is merely another peer. It buys and sells power in the smaller grid as it does in the bigger grid. The smaller microgrid may appear to be a power source to the larger. The smaller microgrid may always buy from the larger. In either microgrid, it acts precisely as do all others. No node knows that another node has a microgrid “within it”.

A proper transactive energy microgrid is essentially autonomous, creating the best balance with the information it has, making the decisions on the edge. Transactive energy markets are a proxy for management and control decisions. Internal system needs are projected into the market through the activation of bidding strategies. Budgets for each system determine priority, but unless a microgrid is badly under-sourced, systems can find some time to run. Within a microgrid, one can increase how frequently, or for how long a system can run by boosting its budget.

In fractal microgrids, one can identify “bridge” nodes, i.e., a node with a connection into the micromarkets that run each microgrid. In each microgrid it participates in, this bridge node uses only the market communications as does any other node. A bridge node likely has some internal “special” logic, just as a storage node has internal knowledge of its own battery chemistry, but this logic is of no concern to the microgrid markets. If the bridge is bringing ISO power and prices to the microgrid, it may compute a markup based on locational pricing, or based on transmission loss. There is no expectation that a node that interacts with two microgrids pass information from one to the other, unchanged.

The transactive energy markets in the two microgrids do not even need to use the same currency. One could use a fiat national currency, such as a US dollar. The other could be using some sort of cryptocurrency, or even a nominal currency defined only within the microgrid.

Direct policy directives can change the behavior of individual systems. Before the hurricane, I may want the power storage system to be fully charged. Because a storage system both buys and sells, it can be reluctant to sell, and eager to buy, until it is fully charged. Because a microgrid may also be able to buy and to sell power, the bridge node may be able to respond to a directive similar to the one that the storage system receives. These need not be the same at all levels. I may want to “starve” a microgrid even as I tell the same microgrid’s storage systems to charge up. How the overall system of systems responds is an emergent behavior.

Social policy, or command intent, can likely be implemented by a small range of directives issued only to the bridge nodes. In an industrial park, I may issue a directive to make all bridge nodes other than, say, the node representing a hospital microgrid to prefer exporting, just a little. The hospital may then be able to buy all it needs—even while all other microgrids find their new equilibria with somewhat less power.

Because fractal microgrids are self-similar, a directive may have effects several microgrids away. A “downstream” microgrid which is represented by a node in a hungry microgrid, may find itself on a diet as well, as the node is less able to acquire power to sell to that microgrid.

The basic market interactions needed by microgrids are well understood and few. They are being standardized this year in the OASIS Energy Interoperation TC. The directives to alter power flows between and across microgrids are not as well defined in standards. I think I have convinced myself, while writing this, that they can be accomplished solely with directives on preparing for resilience through financial information, just as they are for directives to storage systems.