The Last Big Thing

Developers of the Internet of Things always seems to be moving into the last big thing—at least as far as communications expectations and protocols. Too often security is an afterthought, something that can be bolted on afterward.

I often have to design secure communications for new deployments on a University campus. Many new roll-pits are still using RESTfull JSON. Remote systems often transfer telemetry to the cloud using unencrypted FTP. OpenADR generally uses reverse polling because corporate security won’t let external systems interact with on-premises systems secured with last generation security.

BACnet is moving closer to modern expectations with BACnet/SC. Control nodes and sensors can communicate using TLS-secured messages. Devices within the internal internet can work with certificates issued by the BACnet hub. Legacy systems can hide behind a BACnet hub and act AS IF they were secured.

Even so, older protocols and expectations sink in. BACnet router to BACnet application is still limited to Web Socket. ASHRAE specifies TLS 1.2 when many enterprises have moved to TLS 1.3. It is difficult to match the nimbleness of modern IT systems when putting in place systems that will not be replaced or re-programmed for a couple decades.

(Let me be clear here—my biggest complaint about BACnet SC is that I cannot yet deploy it. It is far more secure, and far better architected than what came before.)

Newer IT expectations are expected to continuously tune themselves based upon actual observed performance within their own environment. Applications that cannot do this on their own will end up sharing their data to cloud AI, with resulting loss of performance and loss of privacy and security. We all should know by now that data that goes to the cloud tends to get free in the cloud, offering the hacker or commercial competitor information for a decade. Once released, privacy never comes back.

Some IoT platform models have moved toward Docker. Docker provides a minimal Linux-like operating system (OS) to deploy code anywhere. I’m afraid that mainline IoT will get to Dockers just as the cloud moves to the next thing. On the edge, with the devices themselves, developer may wish to have multiple operating systems: one for Control, one for User Interface, one for AI. A Docker supporting Python for AI may require a lot of resources. Docker is and will remain to fat resource-demanding to support such applications on the edge.

I recently have seen some movement past Docker to DAPR (the Distributed Application Runtime). One can consider DAPR as a much lighter weight Docker. Different DAPR nodes are optimized for different languages. For example, there is a DAPR node pre-adapted to run the GO language (GOLANG or simply GO). GO is ideally suited to develop tiny replacements for Python AI routines. A GOLANG DAPR node can be much smaller and more efficient than is a Python routine on a Docker. Three DAPR nodes, one for control, one for AI based on GO, and one for UI based on .NET core can fit on a thermostat or other small system.

Upgrading some part of such a system, say upgrading the AI, could be as simple as swapping out the single DAPR node without touching the rest.

Don’t be slow to the last big thing. I recommend that smart building developers and smart energy developers consider what they might do with DAPR today.

Spontaneous Order on a Continental Scale

A recent conversation about European power markets and some “glitches” in early June shown a light on profound issues in cybersecurity, in system architectures for big infrastructure, and to an extent the scalability problems with many of the hottest applications for the Internet of Things (IOT).

The specific observations was a plea for direct central control, even as it used an example that showed the shortcoming of infrastructure architecture based on assumptions of central control. It then learned the wrong lesson, that spontaneous order is too “risky” at large scale.

>>> Something went wrong on the 6., 12. and 25. June 2019.
>>> The belief in the Market to fix everything ... may end up in a big
>>> blackout.
>>>
>>> Add-On (2019-07-03):
>>> Today I found more details on the likely reason why we were so close
>>> to big trouble:
>>>
>>> "Due to a faulty data package, the European electricity
>>> exchange EPEX in Paris decoupled the European
>>> electricity market on June 7, 2019. This caused a great
>>> deal of excitement on the markets. Johannes Päffgen,
>>> Head of Energy Trading at Next Kraftwerke, explains the
>>> causes and consequences in an interview.
>>>
>>> Christian Sperling: Johannes - What happened? Why
>>> was there so much trouble at EPEX on the Friday before
>>> the Whitsun holidays?
>>>
>>> Johannes Päffgen: Well - in the end it's a computer error...
>>> but we should go into that later. At about 11:40 this Friday
>>> we noticed that something was wrong at EPEX.
>>> We couldn't place any more bids for the day-ahead electricity
>>> auction on Saturday. ..."
>>>
>>> I guess it was a human error ... somebody didn't take into account
>>> that corrupted data packages will be sent and received ... how could
>>> a faulty package have such a dangerous result?!?!
>>>

While Transactive Energy is superficially similar to the way the bulk power markets have long operated, the power of TE is in local markets. The first benefit of TE is to hide the control complexity/diversity of different technologies behind common signaling. The second benefit is to permit diversity of motivation of each participant in the TE market, as those are also hidden behind the common signals. The power of TE is to allow an emergent order to arise, with balancing of supply and demand occurring without respect to technology or control system or personal beliefs.

One can think of TE as embracing that the Knowledge Problem described by Economics applies to the world of things as well, and that we can use markets, i.e., small decisions made by the participants to participate or not at each moment, to solve power availability without central control. The evolution of life on Earth, of language, of the brain, and of a free market economy are considered systems which evolved through spontaneous order. Naturalists often point to the inherent "watch-like" precision of uncultivated ecosystems and to the universe itself as ultimate examples of this phenomenon.

TE implementations must be aligned with the newer methodology of Laminar Control. Mid-level lamina can coordinate lower level nodes, but do not reach in to provide direct controls. Lamina may however share situation awareness, local effects up, wider area conditions down, to improve the decision-making within each. No Lamina requires the situation awareness of the adjacent lamina.

This has important implications for security and for future technological evolution of power systems on the grid. Aside from the very top level, all lamina are discontinuous. The layer that controls one neighborhood is not actually connected to the controls of a nearby neighborhood except through a common higher level lamina.

The loose coupling of component systems based on abstract communications is characterized as an anti-fragile software pattern. Lightly managed systems coordinated by abstract communications create spontaneous order. Spontaneous orders are distinguished as being scale-free networks, as opposed to the hierarchical networks traditionally used in power distribution management. Spontaneous order is defined as the result of actions, not of design.

For anti-fragile patterns to create resilience and stability, their interactions must be properly scoped so at to not create additional dependencies that create fragility. For TE, this means that not only must the market be local, consistent with the grid lamina, but each market must not rely on additional fragile elements. Making local decisions directly dependent on the communications infrastructure and market infrastructure far away, say at EPEX in Paris, reduces grid resiliency and introduces new cybersecurity challenges.

Besides, the grid is not Magic, and one really cannot buy power from Castille in Antwerp absent the power transmission capability to support such local delivery.

The markets of Transactive Energy will work best when they are based on local markets, able to balance not only power but voltage and frequency within the local distribution loop. Another market may use TE in the district, managing flows between the local distribution systems, and, again, not requiring detailed knowledge of what is inside each. Ideally the market for each will be collocated with the nodes and the controls for each.

Loosely coupled systems in organized in an anti-fragile pattern are manage by objectives and for results. They have no need to expose their internal operations or controls. From a security perspective, this greatly reduces potential attack surfaces. From a policy perspective, this reduces barriers to rapid future introduction of new technologies into a system of systems.

ASHRAE finished defining the Facility/Smart Grid Information Model (FSGIM) some years ago to describe what a Facility should know about itself to participate in these distributed local markets (ASHRAE 201). The abstract information model is consistent with the information model of the Transactive Energy market operations. A Facility that knows its FSGIM, is ready to participate in the local market. Local distribution markets can then replace the wasteful statistical and historic models that manage local power delivery today.

From the SCADA Security perspective, this model moves intrinsically toward defense in depth. From a social and organizational level, each market is a move toward liquid democracy as neighborhoods with their own goals interact with the wider grid. From a technology market perspective, this enables more rapid introduction of new technologies, including those of distributed generation and storage.