Virginia Tech, Emergency Communications, and Academic Sheep

When the Virginia Tech shootings hit the news two years ago, I was sitting in on a meeting of the committee developing standards for communication in emergencies. Interoperability is critical to innovation, and emergency scenarios make the innovation scenarios crystal clear. Unfortunately, emergencies also cause the timid to stampede, and there is no more timid class in America than the academic leadership at our colleges and universities. I began thinking of this entry during the anniversary recognitions on the UNC campus.

Every school in the country rushed to do something, anything in the aftermath. Campus police chiefs were instructed to make a decision now, without waiting to consider the future. Campus decision makers rushed to spend money as fast as they could. On campuses, outcomes are measured on care demonstrated rather than on effective results. Care is demonstrated by new initiatives and by money spent; no campus leader wished to be left behind in number of initiatives. Millions were wasted creating systems that do not work well, and provide no foundation for future growth.

At Carolina, the two initiatives were public address systems and automated phone trees. Neither has proved particularly useful, or effective. Standards-based systems based upon EDXL and its peers were explicitly rejected.

The public address system lends an odd cold war edge to the campus. I grew up in San Diego, a town target rich with Navy base home to multiple fleets. Emergency response and emergency communications were part of the town culture. Warning sirens were mounted on tall white towers throughout the neighborhood I grew up in. They were tested every month, at noon on Friday if I recall correctly. Today, towers like this dot the campus.

This system might have been useful on the campus of the fifties. In today’s world, in which every student walks in a personal shell created by a booming IPod, it is unclear how well they work or even can work. Campus initiatives may not be effective, but they are thorough. The outlying fringe area of office space have the same sirens as on campus. The towers are ostentatious, installed without consideration of cost or effectiveness, and demonstrate caring.

The other initiative is what I call an automated phone tree. The campus signed up with some third party provider to automate calls to campus denizens. The campus asked students, faculty, and staff to enter their numbers into the database. It should come as no surprise that many never learned of this option, and that many more declined to be listed. As time goes on, the list can only be maintained by inculcating fear in each entering freshman class, and in all new employees.

A deeper problem is that this solution does not scale. Thousands of numbers must be dialed. Different cell phones have their own unique ways to go to voicemail. If the phone is busy, should the system try again?. In test emergencies, people routinely never receive the messages or receive them several hours later.

The correct target for today’s emergency messages is the cell phone and, to a lesser extent, the pager. EDXL alerts are all tagged with a geographic polygon of the affected area. EDXL alerts routinely include a narrative message, usually in a CAP Alert. Every cell provider knows the location of its cell towers. Every phone is talking to a particular tower at a particular time. There is no technical reason each of those phones could not receive a simple text message at the same time using existing infrastructure.

Do we really need "IP Everywhere" in the smart grid?

If you want to start a fight in a crowd of smart grid participants, you can begin one by announcing unambiguously how you feel about IP (Internet Protocol) everywhere. Vendors fight to gain advantage for or to forefend elimination of their product lines. Utilities become passionate to defend their AMI projects and their rate bases.

Many of these conversations are premised on (to my mind) flawed thinking. Others need to define what they really want rather than relying on a simple slogan. I am a passionate believer in both open access to information and to open interfaces. I am also against IP everywhere.

One frequent claim is that I may need to talk to any device from anywhere in the future. I need no communication protocol for the car next to me on the free way to access my carburetion strategy. It is a security feature that the pierced guy next to me at the coffee shop does not have an IP address in the credit card in my wallet. Remote access reduces accountability. Remote access creates security requirements. Security requirements create expense and complexity.

We understand this everywhere but the grid and other aspects of the Internet of Things (IOT). When integrating engineered systems, there is a pervasive urge that everything must be able to address everything else at all times. Direct control of remote systems usually reduces quality of both experience and performance. As Gail Horst has explained succinctly, a clothes washing machine already is able to operate its internal controls; it knows that it can’t respond unless it is not full of bleach. It needs to expose only enough to indicate how and when it can respond, and to receive plaints of urgency and notifications of price.

For example, the Energy Management Service (EMS) manages the internal energy use in the home or commercial building. Ideally, an EMS needs communications of price, and of how much to shed, and to make a commitment. Period.

If the occupant chooses to outsource the operation of its EMS to an external third party, then the EMS needs additional capabilities to pass messages about internal devices and capabilities to that third party and to relay commands from that third party to the systems and agents within the building. If the third party happens to be a utility, and the utility business and regulatory model includes direct control by the utility, all messages should still be through the EMS. Today, third party management by the Utility just happens to be the default set of decisions in many parts of the country.

Nothing about this model mandates any shared IP space, or any direct addressability. I would argue that this model accurately describes the *business* model. So what are the IP wars about?

IP interfaces support easy interoperability within a domain—but interoperability between what. I do not need an IP address on my disk drive, although there are business cases when I may want it. The interoperability between things is needed for those loosely coupled situations that I may want to reconfigure/reassemble easily.

Building operators and building integrators are often frustrated by their inability to directly read meter data. The utility may have carefully engineered a solution to collect meter data at fifteen minute intervals to support billing. That solution may use non-standard protocols to wring every bit of performance through a limited communication channel. The billing system may use a batch process to post this collected data against each customer hours later. That information may only be available in a web page after carefully logging in.

The building system integrator would like to access live data for shorter intervals when tuning systems. The building operator would like to access this information in real time to support demand response. These functions require reading the meter on demand. The barrier is that meter data is collected only to support the billing system, and only to meet the needs of the billing system. The problem is sharing information only after processing. If IP were used to support the existing process, none of that would change.

In between domains, there is always a gateway. That gateway may be translating from CDMA to 1000BASEFL, it may be merely performing Network Address Translation (NAT), it may be doing semantic and ontological translation. It is still a gateway from one world to another. As such, either side should barely trust it. As such, it can have different protocols on either side.

The smart grid needs information sharing and informational interfaces. It needs discoverable interfaces at the domain transition, because I don’t care how hard the CPUs are processing, I’m concerned about the 3 days of head scratching, cursing human time needed to integrate each interface (which means every home, building, and factory) when someone switches to a new version of something somewhere.

The smart grid should leverage web developed and web-derived technologies, protocols, and interactions wherever in the smart grid they can speed development, increase transparency, and ease interoperability with adjacent domains to meet business goals. It does not need IP everywhere.

Schedules for Things and Markets

Yesterday, I sat in on the final day of the Calendaring and Scheduling Consortium's semiannual conference (www.CalConnect.org). CalConnect is a consortium that promotes interoperability between dissimilar calendaring and scheduling systems. I was there to scout out their just unveiled xml serialization of ICalendar. I think we will use it a lot in buildings and on the smart grid.

I have written before that we need a simple way to exchange information about events. This is a quite different task than time synchronization. Events have duration and may evolve multiple participants. In the old Mission Impossible shows I watched while growing up, they would plan a series of events, often events that require close cooperation between several actors, whether they were good guys or bad guys. Then they would synchronize their watches and the drama would begin. CalConnect does not worry about synchronizing the watches, but about all the planning for activities that need to happen together.

In the enterprise-responsive building, rooms and access control should respond to normal business events in the building. Schedule a meeting for 9 people in Conference Room 2 on the third floor? Catering a wedding for 200 in ballrooms A and B? The building system should prepare that room to be comfortable by then, and plan for adequate ventilation to keep that many people alert. Enterprise responsive buildings can move beyond efficiency to doing the right thing at the right time.

On the smart grid, we will be constantly comparing events. Next Tuesday at 10:00 power will be expensive. The factory is able to sell excess cogeneration back to the grid during the lunch break, 12:15 to 1:00 five days a week. The office building can shut down early in response to the grid because almost everyone is at the sales meeting. The primary benefits of the smart grid are from aligning supply and demand, and thereby being able to rely on intermittent energy while avoiding expensive or "dirty” energy".

As people, we share schedules and meetings every day using ICalendar. Surprisingly, there has been no accepted standard for writing ICalendar in XML.

On Thursday, Steve Lees (Microsoft), Cyrus Daboo (Apple), and Mike Douglas (RPI) posted a draft of an XML serialization of ICalendar to the IETF (Links below). It will be assigned an RFC "real soon now"™. Steve also created a web based converter into which you can paste your own ICalendar object and get a translation.

I have some quibbles with the draft:

I would like to see the location (latitude and longitude) component of the specification use standards from the Open Geospatial Consortium. For now, they should use KML; you used KML the last time you pinned something to Google Earth. (Imagine all the concert listings in your town automatically pinning themselves to a map). I would like them to consider allowing other OGC defined objects—such as a polygon, tracing out the area affected by the event. I would even like to make this field multi-valued, allowing multiple points to be affected by the event.

The standards also includes the Uniform Resource Indicator (URI) as does the ICalendar object. You may be familiar with the more specific form of the URI, the Uniform Resource Locator (URL)—you typed the URL into the browser to get to this page. The word URI is the more general case and includes more options, including all URLs. I would like to be able to add a list or URIs to an event.

I would like to see the description be typed and multi-valued as well. When you are using a calendar, the description is the part of the message that may show up as the conference call #, the agenda, etc. In the standard this shows up as:

      <description>
         <text>
            Looking forward to a good discussion, 1-800-888-1234
         </text>
      </description>

I would like this the be multi-valued, and able to support more object types than text.

Of course, each of these multi-value issues (location, reference, description) would break backward compatibility. As they are not defined in iCalendar, they could not be converted back into iCalendar. These are not complaints; as I said above, that’s why people put drafts up in public.

I am very glad to see this one up in public...and I hope to incorporate their work into oBIX soon.

The Impulse to Run Around Naked

We were discussing the proposed Energy Market Information Exchange (EMIE) Technical Committee last week when a participant asked "What’s wrong with having devices communicate in their own native languages and over their most optimal media?"

At its heart, this query is a request to let first costs equipment trump all other concerns. It ignores cost of ownership. It ignores the costs of security. It even ignores initial integration costs. It is a naïve plea for a simpler world.

When they were young, I remember my children regularly escaping after the evening bath and scampering through the house. As we’d capture them to stuff them into their warm winter pajamas, we’d hear the joyous plea "Want to run around naked!" It was a happy request, one that always made me smile.In my, uhmm, mature and fully deployed state, few would be as charmed if I made the same request.

Look, I don’t care if you and your family walk around naked in your house. When you go on the street, and expect to interact with others, then societal expectations for behavior and dress kick in. I don’t care if you create small naturist clubs where you can walk around naked with a larger group. Every naturist camp always has a sign by the door "Did you remember to put on clothes?" Streaking, though, is always disruptive. As someone who has managed any number of "native protocols" interacting on a campus backbone, I know that those are far more disruptive then the kids who streak the library each semester before exams.

Tunneling protocols over IP is the like late night explicit romantic phone call. It may be an expedient solution to a short term problem in a niche situation, but it is no architecture. Such phone calls have their own protocol, and their own semantic choices. If that same communication style extends to other phone calls, you get social problems, and potential law suits. Tunneling protocols, xxxx over IP, are just as problematic. They are barriers to interoperability. They don’t recognize external costs. I have seen dozens of high-dollar man hours expended to avoid a second $500 gateway. I’ve seen larger numbers of man hours expended again and again by network operations staff to sustain the protections that these tunneled protocols need.

Based on experience and battle scars, here are a few principles that *I* hold dear:

• Anything that can be attached to the internet, will be. This means that it will be exposed to unanticipated protocols, hostile interactions, and even accidental DOS attacks. We should define interfaces to systems accordingly.
• Systems should be small and coherent, and should not have the internet in the middle.
• If the internet is in the middle, or perhaps even if IP is in the middle, what you have is two systems, and you should treat it as such.
• Internal "native" protocols should not be used to communicate between systems.
• The communication stack at the edge of a system should be well tested and well debugged, and have been used in as many open scenarios as possible so that all exceptionalism will have been eliminated. I never want to discover a new "unanticipated interaction"
• When any combination of systems gets to a sufficient size, interoperability (or the lack thereof) becomes the most significant determinant of expense.
• In any significant system integration, you will be unable to prevent diversity. (Every now and then, someone asks me "Wouldn’t it be easier if we just picked one vendor, one brand, and..." I point out that if we did that, it would take us 20 years to get the "one true protocol" installed, and by that time, we would be unable to buy the legacy systems any more.)

At the edge of each system, we should have well defined discoverable interfaces. There will be circumstances, few and rare, in which we legitimately need to split a system in half<—>but not many. There will always be a need for tunneled protocols, just as there will always be those late night phone calls. We rely on them when we must, but are fooling ourselves if we rely on either one by design.

System providers should always ask these questions.

• What is the interoperability requirement?
• Do you want the integration to scale?
• Will one integrator be responsible for all systems, and all systems that interact with them?
• Over time, will the system ever interact with additional systems?
• Will there ever be any new security requirements.

Answer all five questions. Ask yourself how you define system. Consider whether you are able accurately to predict the changes that will occur in the internet over the life of the system, which may be 20 years? Then, and only then, is it time to consider the justification of the native protocol outside the core system. Then consider if you would be willing to put *that* full explanation into your sales literature...

As your systems mature, as they begin interacting with others, don't let them run around naked.