Thinking about Snowden and Smart Grids

Privacy activists have long warned about the massive data collection enabled by smart grids. Utility representatives have long defended the smart grid by asserting that they have no interest in analyzing the lives of their customers. The recent revelations of government activity in the US make that defense irrelevant, as company after company confesses to have shared operational data with the government agencies. The lesson of current headlines is that it does not matter who collects big data, or what their motives are. Big data is a honeypot that will attract surveillance by someone.

One of the oldest stories of smart grids is of early researchers attempting to analyze the activity in a house from the meter. Early going was slow, as this was before modern tools were available. Still, after a couple weeks, analysts had figured out most of the electrical activity in the house: the big load of the air conditioning, the periodic spikes of the refrigerator, etc. Still, one activity seemed to fit no pattern.

Each week day, between fifteen and forty-five minutes after everyone left the house, there was a change in electrical activity. The researchers were stumped for some time. Eventually, they realized that when the family dog decided that “they” were finally gone for the day and not coming back, it would get up onto the nice warm waterbed. This changed the heating pattern for the bed.

After the researchers tracked this pattern for longer, they realized they could tell whether the dog was sleeping peacefully on the water bed and when the dog was restless.

In recent news, we have seen the US government asserting that there is no privacy right to transactional metadata. Times, durations, and phone numbers of all calls are shared freely. Businesses are subject to prosecution if they do not cooperate freely, or even if they reveal that they have been asked for information. Sniffers in data centers capture even secure information after decryption, so that even the internet service provider cannot see what is being tracked and recorded.

Some communications providers, such as Verizon, have bad records of privacy protection; they respond to all requests without push-back. Others occasionally push back on over-reaching calls. Based on their stated goals and communications documents, the utilities plan to share freely.

In the US, the utilities develop communication standards within the UCAIUG association. The UCAIUG develops communication processes and business process common to all the US Utilities. The standard for communication of meter data to third parties developed in the UCAIUG is OpenADE (Automated Data Exchange). It is notable that in their own OpenADE development documents, exchange of information with law enforcement is given a higher priority than exchange of information with customers.

The stated priorities of OpenADE have always been troubling. In the last few weeks, even the skeptical have come to see that big data is irresistible to government agencies. Protests by power utilities that they do not want to use the data are meaningless. FISA court data requests typically are known only to small numbers of a company’s employees. Discussing the requests openly, either within or beyond the company can violate federal law. A couple years ago, if worried publicly about this, one could be accused of being a conspiracy theorist. Today, doing so is evidence merely that one reads the paper.

We have it in each of our hands to preserve our own privacy. Consumer technologies exist to smooth power curves and permanently shift load. Energy storage technologies able to accept and provide trickle charges within a business or home can be used to hide the details of our activities and our lives. These technologies don’t let energy data out while they accomplish the goals of smart energy. You can adopt them, or you can allow further monitoring of every activity in your life.

Whatever one feels about Snowden’s revelations about current behavior of the NSA, they are part of making public how government agencies will make use of any sufficiently large trove of data gathered by others. Other news demonstrates a willingness to use information gleaned by one agency to coordinate public and political pressure by other agencies against those who dissent, and to do so without regard for regulation or fourth amendment. This should give anyone pause before contributing reating with the government.

We have it in each of our hands to preserve our own privacy. Consumer technologies exist to smooth power curves and permanently shift load. Energy storage technologies able to accept and provide trickle charges within a business or home can be used to hide the details of our activities and our lives. We can use minimalist economic signals to accomplish everything hoped for of the smart grid. These technologies don’t let energy data out while they accomplish the goals of smart energy. You can adopt them, or you can allow further monitoring of every activity in your life.

Energy and the Microsoft ROC

Yesterday I had the pleasure of a tour by Darrell Smith, Director of Microsoft Facilities & Energy, of the Redmond Operations Center (ROC). Facilities & Energy provides internal support; it is not a product line. The ROC applies Big Data to the operations of buildings on Microsoft’s home campus. In concept, the ROC is much like the Enterprise Building Management System (EBMS) at the University of North Carolina that I have written of. The results at Microsoft, though, are much more successful.

Darrell avoided the trap that we fell into at UNC, the trap that says that until the in-house enterprise system can control all the systems, it is a failure. In the Redmond Operations Center, Microsoft concentrated first on data mining and operational improvement. By Darrell’s definition, his system will be mature when it does control as well, and may never reach that level of maturity. In the meantime, he is finding wins every day by applying analytics to improve operations.

The center produces two kinds of analytics, alerts and reports. Each of them is focused on finding ways that Microsoft is wasting money every day, and fixing them.

Everybody who works with building systems is aware of the alarm spam that traditional systems send out. Turning off is an Alarm. Turing on is an Alarm, Catching on Fire is an alarm. Build alarms are events with almost no meaning. The Redmond system reads the low level protocols and harvests state information and alarms. The system also gathers other factors from a number of sources, including essential weather information.

To generate Alerts at the ROC, building engineers instead describe patterns. The system gathers a mass of information from each system and air handler tracked. The in-house engineers create queries that identify such issues as short cycling, or incomplete closing of dampers, which they name Faults. The engineer carefully examines the operation a single air handler, or a single building to find a problem. This problem is then described by means of a query that brings back this problem and similar faults in other buildings. For each fault, a cost is assigned by the system based on the size of the system, how much energy it uses, the system’s operating schedule, etc.

In the ROC, all active faults can be seen by building or by system, and can be sorted by cost or criticality. Generally, expensive ones are fixed first. Each work order is tagged with the cost that is being avoided by a timely fix.

Because Microsoft has a learning culture, this process reduces energy use even in the buildings not yet accessible to the ROC. Building mechanics fix problems early in monitored buildings and can see results. They share what they have learned with others in buildings not yet monitored, and the underlying faults are fixed in them, too.

Reports, as I understand them, look for more inchoate patterns. A report can describe many attributes of system or group of similar systems. Reports reveal operational outliers, i.e., the air handler that runs all night, or the system whose damper opens unexpectedly. A newly trained mechanical engineer may take a day or two to define a report. Once defined, it can be run again and again, including for other systems and for other buildings.

Think for a moment of the most powerful thing you have done with a spreadsheet pivot table. Now consider applying that task and that visualization to raw operating data from your buildings every day, and using the results to generate work orders. This is what the Facilities and Energy group has done in the Redmond Operations Center. They find real problems in operation and configuration of building systems every day, price them, and fix them. That is enough, even without control.

Microsoft is doing other things with their own facilities operations which I may write about later. But today, I am marveling at the data-based operation of facilities.

Big Data, Buildings, and the Internet of Things

Big Data is the hot new buzz-phrase for something that buildings system integrators have long struggled with. Last Thursday (3/29), the White House Office of Science and Technology Policy (OSTP) launched its public initiative on big data for government, the Big Data Research and Development Initiative.

The purpose of big data is to support analytics, that is the massive crunching and correlating of data to find patterns. Early targets of the initiative include:

• putting the government’s own data sets into open formats
• pushing states to include a data or statistical literacy component in their education plans
• establishing ways to continuously collect data on prescribed topics as opposed to relying on temporary snapshots

The real time use of big data that is most commonly in the news click-stream and advertising analytics. This back-room technology only makes the news when there are privacy violations. Big data analytics are why Google is now in a death-match with Facebook, and why the European Union is in a privacy face-down with Google.

In government, the best known big data analytics are in security and crime prevention. Einstein systems gate all information in or out of each cabinet-level department, searching for patterns that indicate intrusion. The NSA and FBI are doing something with big data; the NSA may or may not be consolidating information on all internet communications at its Utah Data Center.

Buildings have long struggled with big data. They are not designed for storing or to processing too much. System instructions regularly warn to minimize trend reports. Product from a number of leading makers of environmental controls struggle with monitoring just a small portion of the buildings on the UNC campus. Building systems houses all aim at cloud-based analytics in their next release, but each that I have seen struggles with pushing information to the cloud. I have watched very fast networks struggling to handle data collection from a 100 buildings, and watched data edifices crack under the hundreds of gigabytes they produce each week.

We are just now entering the period in which the internet of things (IOT) becomes real, and the IOT stores its data in the cloud. Last month, Ninja Blocks (http://ninjablocks.com/) got its initial funding. Ninja blocks are consumer sensors that are as cheap as X10, and send their data to the cloud. Ninja blocks use open source hardware (download schematics from the site) to sense their environment: acceleration, temperature, current, humidity, motion, distance, sound, light and even capture video. You can create and sell your own Ninja Blocks to connect to the Ninja Cloud.

The Ninja Cloud connects this sensor information to social and cloud services. Sensor events can send tweets, SMS, or email. Ninja photos and video can move automatically into Facebook or Dropbox. The user plugs in a Ninja Block and then uses the web to develop scripts in the Ninja Cloud using point and click.

This may not be the same as energy management, but one of the more successful campus energy projects of recent years set up Facebook pages for buildings on the University of Mississippi campus. Students were encouraged to friend the buildings; systems in the buildings tweeted their energy use. The project raised Student awareness.

Ninja Blocks is a new company. They can probably do most of what they claim. Their team of entrepreneurial young engineers seems smart, quick, and committed. Their business plan is inspired using open source hardware to let others create new value sources for the Ninja Cloud. Still, I wonder whether their approach will scale well. They may hit the same wall that I have seen, when too many sensors are continuously logging too many points to the cloud.

Whether or not Ninja Blocks makes it, they are the future. Other start-ups, such as the Bluetooth-based, open source i-voltmeter will change the way we think sensors work. The data gathered by the internet of things will make its way to the cloud, where it will be Big Data. Building systems that do not participate will find themselves pushed aside.

The value of Big Data is in re-purposing and in re-use. The cost of gathering big data is going down, and will continue to go down. The Big Data from buildings will accumulate at an astounding rate. The value of Big Data will be in continuous re-harvesting for more information, the way click-streams and advertising are harvested again and again. Building operations and failure predictions are only the start.

Big Data from Building systems must learn to share well with others. This industry must consider its own version of the federal goals: open formats for data, better statistical literacy in systems, and the methods to collect and store very large volumes of data loom large. We may need to use the common semantics from Project Haystack as a common ontology for our big data. It will be mandatory to share with the Big Data from the IOT, both to accept IOT data into Building Clouds, and to send Building data into the IOT clouds, including the Ninja Cloud.

It will be a fast ride. Into the Clouds!