Pervasive Security and Control Systems

With cybersecurity so much in the news, I found myself in a heated discussion the other day about whether IT should take over SCADA, and in particular SCADA security, or whether it should not. SCADA (System Control And Data Acquisition) refers to the technologies that run large processes. In common use, it refers primarily to the large distribution systems, such as those for electricity, water, and gas. SCADA systems were usually designed to operate with the extreme resource constraints of last generation technology. SCADA systems have traditionally been secured primarily through isolation. Any signal that breached the outer shell was considered trusted.

It is an interesting characteristic of technology that when it is everywhere, it is no longer anywhere. Take timekeeping, one of the oldest automated technologies. Modern time technology sprang from the monastic orders of the middle ages, wherein it was important to track the time for prayer and the ordered life. As time tracking technology improved, it was moved into the clock tower or cathedral in the center of town, and used to order the economic life of the townsfolk.

Time was later brought into the homes of the wealthy, and then adopted, in the form of mantle clocks by the middle classes as a  luxury. This was followed by personal time, as pocket watches which were a sign of wealth or awarded, at retirement, as thanks for long service. Time kept growing cheaper until digital time arrived in accurate wrist watches at disposable prices. Today, time is everywhere, in ovens and in coffee-makers. Time is more important than ever, as very precise time-keeping is at the heart of telecommunications and the internet. Precise centrally managed is in every cell phone—yet time is nowhere, and watches and mantle clocks are becoming scarce.

There is a common meme in management circles that IT is becoming pervasive, and therefore beginning to fade as a separate department within companies. We have central management of network communications as a critical facility. There may even be central operating system and hardware management within a data center; that data center may instead be outsourced and no longer part of the corporate skill-set. In the service oriented world, there is central technology governance to describe how technology from each division fits together. Subject to that guidance, the divisions and department are free to manage their own development, and their own decisions.

At the beginning of the 20^th century, it was not uncommon for manufacturing corporations to have people with titles like Vice President of Electricity. The person who held this title had all sorts of strategic responsibilities. As electricity became pervasive, this role became less important. As everyone grew to understand, more or less, how to use electricity (Use the plug. Don’t drop a paperclip on the leads), the need for specialists at every step of the process became less. I have seen hotel wiring for lights installed by Edison own hand; none of us can imagine the CEO of a large research and engineering doing that contract today.

Today, electricity is everywhere and it is nowhere. Outside of those businesses that are directly involved with the production and distribution, the strategic use of electricity has vanished. Oh, you still need an electrician or two on the maintenance staff; he may also be a plumber. Electrical engineers are needed to design systems for factories or buildings. Electricity as a profession in each organization is gone. Plug in your own lamp and computer!

In a similar way, IT is becoming everywhere and nowhere. I have a computer far more powerful than any available in 1970, and with more networking bandwidth than any in 1990 sitting in pocket. It is also able to create and process video and has a display capability greater than any but the highest end computers of two decades ago. I carry it everywhere, it may be company issued, but it is never touched by company IT. Sometimes I make phone calls with it.

A decade ago, every resume claimed some experience as a webmaster. Now very few do, although they have Facebook pages and a facile familiarity with HTML. Every salesman and every factory quality team performs computerized statistical analysis as part of their work, although none of them claim to work in IT. The specialized staff who install the physical infrastructure of networking have fused with those doing analog telephony.

Much of IT is gone. Security policy staff are rising in visibility, but growing fewer in number as they use policy based tools. Software installation staff, necessary as policy locks out most users from modifying system configurations, grow closer to electricians in education and in perspective. The CIO becomes a specialized sort of efficiency expert. From this perspective, either control systems staff and the accounting staff are both IT, or are both “not IT”.

IT security offers a set of disciplines and mind-sets useful to those building their current systems with today’s tools. Knowing IT Security assists the control system engineer in the same way that knowing accounting is the path to advancement for the accounting clerk. Knowledge of auditing principles makes a better manager just as other IT-security skills make a better SCADA system architect.

I think most organizations will not have IT functions per se in the future, unless they are designing electronics, or creating new graphics systems. I think SCADA and control systems will not be run by IT, but will be perfused by the pervasive IT all around. System design, and system architecture will still matter. IT Security, with the newly popular moniker cybersecurity, will be everywhere. But IT will be gone.

Cyborg Beetles, Cyber-security, Smart Buildings, and the Smart Grid

Cyber beetles provide an interesting glimpse into agent based interactions. Smart grids and smart buildings are integrated today using deep, integration, and complete control of the underlying processes. As more and more nodes are added to any system, the overhead of maintaining all interactions at a central point becomes more significant. In grid-scale systems, system designers have managed complexity by limiting diversity; a system may be managing ten thousand substations, but at least they are identical systems. A current DARPA project dramatically demonstrates a better approach.

At a recent IEEE meeting in Italy, Michel Maharbiz of the University of California demonstrated his Cyborg Beetle. His team has implanted electrodes in a giant flower beetle and mounted a wireless receiver on its back. The team is able to cause the beetle to take off, to hover, to turn left and right, and to land. Someday, a system like this may be used for surveillance or to guide rescue operations.

The beauty of the system is its simplicity. It uses an off-the-shelf wireless receiver. The signals sent to the beetle are very simple. The beetle performs all complex acts without requiring direct control. The biggest challenge is placing the electrodes. The interface consists of six electrodes implanted in the basal nodes of the flight muscles and in its optic lobes.

The beetle is arrives able to maintain its equilibrium. It comes able to synchronize its muscles to maintain efficient flight. The system uses the minimum signals needed to make the beetle do so. Like scratching a dog on the side to get that hind leg going, the cyborg beetle gets an itch to fly and takes off. Because the messages are so insignificant, this approach saves battery life as well programming complexity.

The beetle has evolved for efficient flight and balance. A core principle of ecology is that the most intense competition is always intra-niche competition. Beetles compete with other beetles, and compete most intensely with beetles that seek the same food, and live in the same place. This is a good model for the smart grid and for smart building interactions.

We want the most rapid development we can get for each of the nodes of the smart grid, and for each of the technologies of smart energy. To get this rapid development, we must put these technologies in direct intra-niche competition, and not allow competition to be lessened by large product lines or entrenched systems.

We can do this by limiting the control and integration we use between each node on the grid. We must eschew deep integration and direct control of the processes of each substation. Just as the Cyborg Beetle operators leave flying to the evolved processes, we should leave substation operation to the substation, and home device operation to the home devices. We want a rich, diverse ecosystem of energy strategies, an ecosystem with intense competition.

Control of the Beetle is limited to deciding whether to hover or to land, to turn left or to turn right. The beetle knows how to fly, and how to land. A beetle that will not fly can be replaced. That’s how it should be in the smart grid and in the smart building and home. Let the node take care of security. Let the node take care of operations.

You can watch the flight of the Beetle at MIT Technology Review Multimedia (http://www.technologyreview.com/video/?vid=217)

Smartgrid Basics: The Demand Side Problem

Last week the Smartgrid-discuss group opened up within OASIS, introducing power grid technologies to the architects of e-commerce and internet security standards. Some of the latter are trying to understand the problem, and learn the jargon. I wrote this as the second of a series of posts introduce the issues in a simplified, almost cartoon form.

Building systems have traditionally been invisible and uncontrollable. They have been managed to reduce costs with no real focus on the service they are providing. They have grown up in sandboxes, using their own peculiar protocols. These protocols are deep and technology specific, and often without effective interface. These systems are operated, when they are operated by process specialists.

Building occupants rarely have a precise understanding of how these systems affect their business. They may know exactly what a too-hot or too-cold call costs. They know that tenant dissatisfaction may lead to un-renewed leases. They may suspect that under ventilation may lead to sleepy occupants, but can rarely put any exact price tag on that. This makes them conservative about making changes in building operations.

Demand Response (DR) is emerging a critical tool for dealing with peak load management. Peak loads are by far the most expensive and dirtiest electricity we have; their costs, on both bottom lines, swamping others. Demand response is moving from direct control to economic incentives, but underneath, today’s integrations are process centric rather than service oriented. Energy providers order or pay energy customers to turn off things on just a few days a year, to manage the peak. We encourage only the crudest, least effective energy savings, while denying the market the energy signals that would cause better.

At the commodity system level, DR is already moving to services and agents. Agents defend their own mission while responding to the outside world. Washing machines know not to respond to grid signals until they determine that the current laundry is not soaking in bleach. Refrigerators know not to respond if they have just finished a defrost cycle. These systems know and understand what services they provide and so are ready to be responsive. Building systems are not.

We will get larger DR when we talk to the building occupant. We will get better participation when the occupant remains in control. The occupant will not allow DR when the in-laws are coming for the weekend. The occupant knows the family overspent at Christmas and is willing to respond to any and all incentives. The access control system may know that only three people on the fourth floor came to work today. Human resources knows that the sales force is on a retreat. Together, they can choreograph far greater response from the building systems then ever will be permitted as an automatic response from control communications.

Demand Response must be about economic signals to a business entity. When thought of in this way, there is no need for different signals to Industry and to Business (and to home and to vehicle). The business may choose to automate this. The business may benefit from templates for response, whether developed by EPRI or by ASHRAE, which reduce the risk of considering participation. These choices and these templates are not part of the interface.

The interface should not does not concern itself with the underlying technology and control protocols. It should not be based upon BACnet, or OPC, or LON any number of other low level control system protocols. The interface must be one that enables business decisions. Control systems should offer up service interfaces for choreographed response. Whatever offer and counter offer DR requires, whether amount of load shed or maximum load used or time to respond must be in the interface, but no deep process.

The smartgrid to building/industry/home interface is about how the Service Oriented Building can respond to the Service Oriented Grid. Just as in other services, the underlying processes should be hidden.

If you want to join the public discussion at OASIS, send a message to smartgrid-discuss-subscribe@lists.oasis-open.org.

Smartgrid Basics: The Supply Side Problem

Last week the Smartgrid-discuss group opened up within OASIS, introducing power grid technologies to the architects of e-commerce and internet security standards. Some of the latter are trying to understand the problem, and learn the jargon. I wrote this as one of a series of posts introduce the issues in a simplified, almost cartoon form.

The North American power grid is the world’s largest robot. It was imagined in the 30’s, designed in the 50’s and has been built out and patched ever since. Some very bright people have done extraordinary things to retrofit the system with digital descendants of the original analog controls. It is very much less stable than folks let on. It suffers from an instability condition that occurs periodically and has for years. This condition was occurring when a tree branch took a transmission line and thereby a third of North America on August 14, 2003. That underlying instability occurs an order of magnitude more frequently today than it did then. Something has to change.

The archetype for modern power markets was established 100 years ago in Chicago on April 1908. At that time, power demands were low, and electric metering consisted of pens on mechanical turntables that spun as power was used. These paper sheets were collected and read periodically. Modern power marketing was established a natural monopoly with regulated cost recovery, much as telecommunications used to be. The regulated cost recovery market is only slowing to take advantage of digital metering using two way communications. Many new installations are still being designed as asymmetric interfaces, with the demand side, i.e., the building inhabitant, excluded from direct communication. New business models must support transparency and symmetry.

The Carterphone law suit established that third party equipment could be attached directly to the phone system, and Judge Green tore down the natural monopolies. The model of 25 year depreciation of black handsets owned by the phone company began to erode. New business models, beginning with fax, continuing to modem-based communications began to arise. Today deep process interactions running through slow moving standards bodies prevent the attachment of new types of systems. Innovations must be approved as expenditures by 50 public utilities commissions. Today’s need for rapid innovation in energy generation, storage, and conversions demand more agile business models.

In 1908, there was no exchange of power between local markets. There was no dynamic pricing. Consumers still use power as if it were a static resource; wholesale prices oscillate though each day. In many parts of the country, power prices are actually negative at regular times each week. Most goods can stay in the warehouse overnight; electricity cannot. We can win great savings by smoothing power demand. Without price signals, end users in buildings and homes have no incentive to help.

The grid is built for peak capacity. 17% of the grid’s generating capacity is used for less than 110 hours a year. This capacity is the dirtiest and by far the most expensive generation. These plants may even be spun up but idle, ready to be called into use if needed. The system as a whole bears the cost of this very expensive peak load. If consumers in buildings, homes and industry could respond rapidly to signals that the grid was nearing the need to use these resources, it would greatly reduce costs, both monetary and environmental. The power industry calls this Demand-Response, and as of yet there are no standards. OpenADR is a good start.

Power Grid operation is like Windows 95. I say that as someone who considers Windows 95 one of the supreme engineering achievement in software. Windows 95 had to support every bit of software that had ever been written, including some horrible mistakes. Windows 95 had to create an environment that made it possible for new markets using 32 bit software to develop, while running all the old software. Windows 95 had to support old drivers and memory management based on the old 840K and 32K memory thunking, while switching to virtual memory management in mid-boot if no such drivers were found. Windows 95 was a shaky bridge built over a chasm, made entirely of bent toothpicks and wet tissue paper. It would be easier with structural steel and suspension materials, but that easier job was not the task. It was a wonder that Windows 95 could work at all. Today’s power grid, and SCADA (Supervisory Control and Data Acquisition) strategies, and system operations are like Windows 95, tied down to backward compatibility and hampered by the reasonable decisions of long ago. Perfecting Windows 95 led to the increasingly unwieldy Windows 98 and Windows ME. Sometimes it is better to do things that aren’t so hard.

In summary, inquiries about how it is done today are not always useful. Paving the cow paths to handle heavy traffic is not the best way forward. The GridWise effort is to find something new, and that something will support new markets that we do not today know or understand. It must do so while stabilizing the grid even as we add de-stabilizing new energy sources. It must promote better control even as we accept new players and more point sources of generation.

What is the model? If we do this right, that question will be like asking what the new economy would look like before the DotCom boom...

If you want to join the public discussion at OASIS, send a message to

smartgrid-discuss-subscribe@lists.oasis-open.org.