Choosing a Light or a Dark Mirror

“If it has to do with heating, lighting, or mobility for human beings on this planet, we’re interested in it”
- Darryl Willis, Google

Last month, in the July issue of Automated Buildings, Ken Sinclair called for smart buildings to spearhead an improved relationship between the physical, the virtual and the emotional world. Relationships go two ways. When we consider how buildings can manipulate our emotions, we also are considering how our emotions can manipulate buildings,

The sci-fi anthology series Black Mirror explores a near-future where humanity's greatest innovations and dark side collide. Last week, Daikin and NEC announced that they have developed a system that monitors the movement of the employee's eyelids and hits dozing workeers with a blast of cold air. More are growing aware that traditional cloud practices have a dark side in the erosion of privacy and often misuses of personal data. Will Ken’s call lead to better buildings or to their dark mirror?

Crude interactions will predominate at first because buildings have no way to empathize with their occupants. The early phases of emotional relationships with buildings will be based on specific purpose driven metrics developed by building engineers responding to occupant middle management—two groups that may not be the most empathic themselves.

More subtly, Daikin / NEC collaboration begins lowering the ambient temperature when it detects people getting sleepier.

Today, IT throws up artificial intelligence (AI) as the answer to every new problem. In the Internet of Things (IoT), this usually means combining several variants of regression analysis based on concrete models of mechanical systems. This model will not take us far down the road to Artificial Emotional Intelligence (AEI)

Humans can respond to mutual emotions because they are able to share them. In some theories, this is based on our mirror neurons. A mirror neuron fires both when we act and when we see someone else performing the same action. As we subtly shift our posture and our face to match that of those around us, we learn how they feel by feeling how we feel. Buildings can’t do this. Yet.

AEI will rely on highly abstract models of human actions and interactions. Humans are too complex to collect and transmit all data to a remote cloud, or even to the building-based cloud if there are more than a few people being tracked. Simple systems will transform data into these abstract models at the edge, and only the abstractions will be sent to the cloud. Where desirable, this enables anonymous and privatized data to be processed alongside personalized data.

These abstract models will become the “mirror neurons” of the building-based systems. Building-based systems will respond not by trying to mimic humans, but by comparing edge-based abstraction of human behavior to the abstract human models they have internally. Potential responses will then be filtered to the IoT by a repeated de-abstraction (“make alert” to “more cooling” or “more ventilation” or “more light”) to potential specific concrete choices. The final choices will be made by traditional engineered systems, based on economic outcomes (such as energy use) and engineered choices such as ASHRAE considerations (air turns, humidity control, etc.). Edge processing will the send the abstracted effects of these choices back into the regression models.

The Classification of Everyday Life (COEL) is a recently completed specification. COEL is an OASIS specification, just as are OBIX, SAML, and the specifications for Transactive Energy. COEL is already an international ecosystem with multiple implementations based around Coelition. COEL was designed from the ground up to support modern privacy law, necessary for products to reach to international markets. COEL defines creating, transmitting, and storing the behavioral abstractions needed to create the “mirror neurons” for AEI.

This can be hard to map one’s head around. I’ll start with my own child-like understanding and description of some early COEL apps.

The hottest topics in health care are Evidence Based Medicine and Standards of Care. Evidence Based Medicine aims to optimize decision-making by emphasizing well-designed and well-conducted research to build strong recommendations meta-analyses, systematic reviews, and randomized controlled trials. Standards of Care refers to detailed sequences of medicine that may continue over years, and may include, in the most difficult processes, hundreds of clinical events. A Standard of Care for orthopedic surgery may start with pre-surgery “pre-hab” (getting strong enough to benefit from the surgery), to a couple weeks of pre-surgery preparation, to the all the events the day of and the day after the surgery, to programs for rehabilitation after the surgery.

Zooming in, without evidence of pre-hab fitness, it may be worthless or even dangerous to proceed to surgery. The best post-surgery outcomes involve both sending the patient home quickly and making sure the patient is returning to level of exercise and activity. For the single patient, this requires tracking and analysis of what the patient is doing outside the hospital. For Evidence-based Medicine, this requires factoring the patient response and activity back into to the meta-analyses and systematic reviews.

But what is the patient doing at home? Medical decisions during pre-hab and re-hab may be based on levels of physical activity. Counting trips to the gym or physical therapist is at best inadequate and at worst misleading. One patient may go to the gym and stand around watching CNN. Another patient may not go to the gym often, but might use the stairs at home and at work. Coelition member Activinsights already makes Android apps that can analyze the individual from a wearable device, abstract the data into COEL-based information, and present privacy-protecting, and pseudo-anonymized COEL Atoms to support clinical and research decisions.

While developed to support clinical work, these Apps can make personally useful predictions. Active Insights apps can predict when each person is most likely to be alert, and able to make good decisions. Simple environmental monitoring can bring a feedback loop into building operations. It is easy to imagine apps that also COEL abstractions for physical activity into personal recommendations for alertness and for re-setting the body after jet-lag.

But this is a building-based audience. It is not hard to imagine a critical meeting with attendees from many geographic locations. Personal but fully anonymized COEL biorhythm data is submitted to the building for each participant. The building then solves final schedule, ventilation, temperature, lighting level, and perhaps even lighting color to create the best chance for the best work from each participant. A conference center that can reliably do this makes a good case for a premium price. When many knowledge worker work from home or coffee shop, COEL-submissions to the scheduling server might determine the time and location of even in-town events.

It has been said that the essence of marketing is to build a relationship and engagement. Engagement can be measured as demonstrating to an individual that you know them. In smart health care, patient engagement is best when the patient can recognize themselves in the data. Building-based AEI enables a building to show its occupants a mirror to show them that it knows them. That mirror will be a Black Mirror unless it knows this in a way that  protects privacy and anonymity.

RoSy outlook for distributed autonomy within systems

I feel I must be one of the last people to discover the open source Robotic Operating Systems (RoS). RoS is more of a framework than an operating system. The framework could be atop any operating system. In practice, for now, it is on Linux. (There are some interesting DotNet / Mono extensions, but those appear incomplete). ROS is providing the base for open source robotics, and the effect of robotics on all our lives will expand because of it.

RoS engages my imagination because it is inherently distributed. “Service Oriented Robotics” as a phrase that is used. Replacing the step-by-step commands that have ruled robotic manufacturing, ROS developers aim at tasks such as “Go upstairs, go to my room, find my stapler on my desk, and bring it back”. This must be decoupled into applications for climbing stairs, navigating a floor plan, identifying a stapler, and picking that stapler up.

Just as smart energy looks to fractal dis-assembling of power grids, RoS looks to fractal dis-assembly of robotic tasks. There are multiple ROS services for a robotic hand, decoupling the technology and the mechanics from the request. As ROS-capable systems get smaller and cheaper, there will likely be RoS applications for each knuckle on a hand. A RoS-enabled knuckle can more easily incorporate advanced features such as haptic feedback leading to a “gentle touch”. Gentle touch and heavy lifting can be different limbs responding to the same command.

Robotics is outside of my wheel-house. Service enabling of the internet of things is in. Service oriented energy is in. Fractal microgrids as described by the Galvin Initiative seem natural, and they will have their decision-making local, where they can respond to the needs of site, and the owner, and the situation.

Robotics started out with fixed activities under direct control. In the larger systems, one can still see the single control even as they grow more autonomous. The future is distributed service oriented robotics. In the same way smart grids started with planned sequence to control transmission. It evolved into fixed sequences to control energy consumption, centrally operated, by OpenADR and by EnerNOC and by Constellation. It is slowly evolving into centrally orchestrated DR services.

Even Microgrids are often simply the old architecture, and the old protocols, but just a little bit of isolation. Duke is pushing microgrids barely distinguishable from their distribution networks. Oncor salutes service orientation while extending the old technologies. The real advances are among those building those “smart hands”, autonomous microgrids that make their own decisions and technology choices. >Eventually, just as in the smart knuckles, the same service orientations will arrive in the end appliances and systems of the end nodes.

When I was young, in my Dinosaur age, I was fascinated by the Stegosaurus, and its hind-brain bigger than its fore-brain. That was settled science then, although controversial now. I enjoyed imagining a slow placid creature able to defend itself with some nimble, precise tail-bludgeoning.

The microgrids of the future will leverage distributed energy and local storage to for some precise tail-bludgeoning in the smart building—the far away head will not even be sure what is going on.

Finding a Needle in the Internet of Things (part 2)—Buildings and Building Systems

In a previous post, I described how vCards are used throughout standards-based scheduling and calendaring systems. Many different vCard standards coexist in today’s organizations. I also described how directory services, especially LDAP (the Lightweight Directory Access Protocol), are the well-established means to enable wide secure access to the information in vCards. In this post I discuss current efforts that will expand these existing standards to support buildings and their systems.

The most frequently scheduled building-based resources are public rooms and building systems. Public rooms are invited to meetings as are other attendees. Smart buildings can optimize energy use while preserving amenity if they know when and by whom the building will be used.

An enterprise scheduling may include hundreds of schedulable conference rooms. These resources are generic to some extent, but the potential scheduler would like to filter the list. Show me the conference rooms that are near me, and that will seat at least 8 people, have an internet connection, and have a projection screen. If there is a cost, show me what each costs per hour.

Two things stand in the way of adding this as a standard function. Today, there is no standard for what the names of each of these features is. In other words, there is no Resource vCard standards for rooms. The second is that there is no source for this information. Few want to take on an additional data maintenance task to enter this information or to keep it up to date. Fortunately, there is a solution to both of these problems, and that solution is BIM. More particularly, the solution lies in COBie Lite.

COBie Lite describes a strongly typed and validateable data model. COBie Lite has been stripped of all process, it does not matter what the source of the information is. The information in COBie Lite can be exported from the BIM used to design and build a building. COBie Lite provides a formal definition of the information that should be collected during commissioning. COBie can be imported into all of the major Computerized Maintenance Management Systems (CMMS). Today these systems are roach motels holes of COBie—data checks in, it doesn’t check out. That can and will change.

There are many sources of COBie lite. In each of them, the information is created or maintained to support an existing business process. A standard transform of COBie Lite can produce all the information needed for a standard Resource vCard for rooms. I call this standard the BIMcard.

Building-based systems also face problems of dynamic integration. Traditional building management systems are highly proprietary. Even when fronted by standards-based middleware, say a Tridium JACE exposing oBIX, it is still hard to integrate with business functions in any scaleable way. Let me be clear what I mean by scaleable. A BAS might take one engineer one week to link up BAS and some fixed enterprise functions. To link up 5 buildings might take a single engineers 5 weeks, or a 5 engineers one week. If it was a scalable process, we might expect the 5 engineers could integrate 100 buildings in two weeks. If the buildings can integrate themselves, that number goes way up.

A common BIM-based model provides a path forward. A commissioning report can produce an equipment-only COBie-based BIM. If there is a building model from construction, no matter how incomplete, it can provide a framework to host that COBie-based BIM. A profile for Building System Resource vCards can be defined based again upon COBie Lite. BIMcards, then become the searchable entrée to the systems in buildings. It is not hard to imagine BIMcards for temporary equipment, wherein they can register themselves in the building.

Slim BIM: The Middle Ground between Document and Service Part 1

Engineering information is document oriented. Large documents, even sheaths of documents, are exchanged, specifying in great detail exactly what to do, and how to do it. Modern IT (Information Technology) is based on Services. Service exchanges are minimal, as small as can specify results, and do not specify the means of execution at all. For the last 50 years, IT has moved far faster than have engineered system, the things we can touch, inhabit, or ride around in. For the next 50 years, when engineered systems will need to evolve as fast as IT has for the last 50, we will need a middle ground, between document and service call. This is the challenge of configuration, shared configuration that will enable big systems to interact as nimbly as does IT does today.

Buildings are big systems, composed of big systems, that must interact with the IT-based systems of their occupants. The systems of the occupants will change many times during the life of a building. If we are to meet national and international energy goals, the collection of systems in each building will change frequently as well. These systems will interact with services, simple calls conveying only requests and results. Before they can communicate with each other as services, each must learn about the other. Each system must be configured with the information it needs to request services. This information must be non-specific, to avoid the complexity of details. This information must be specific, cataloguing service entry points and potential performance.

For buildings, designed by architects and engineers, the design and specification uses BIM (Building Information Model). These are traditionally very large and cumbersome files. The National BIM Specification (NBIMS) describes documents based on the International Foundation Classes (IFCs). The IFCs are two cumbersome for exchange, so NBIMS specifies Information Delivery Models (IDMs) for each structured hand-off of information, and a model view for each IDM. These information exchanges are detailed and overly specific. They rely on document-centric notions of XML from long ago, seen as a “replacement” for large the documents in SGML. The IDM for each stage of a project is different, even if the information is essentially the same.

The problem is, no one outside of architecture and construction uses these approaches, and few seem willing to adopt them.

Recently, members of the National Institute of Building Science (NIBS) have worked on the hand-off of information at the end of a construction project to the maintenance management system (CMMS). They have developed the Construction Operations Building Information Exchange (COBIE). COBIE lists the spaces and their fittings, the systems and the spaces they support, and the equipment in each system with its maintenance requirements and spare parts. The market leaders in CMMS each support COBIE import. Maintenance staffs have reported replacing weeks of error-prone hand entry with 15 minutes of COBIE import, and had their Preventive Maintenance (PM) and spare parts management ready to go.

Other systems could benefit by importing COBIE as well. Building owners often run many Line-Of-Business (LOB) systems, often selected by different parts of the company, from different vendors. Asset Management, Capital Renewal, and the Registrar’s Classroom Scheduling, each has its view of the core facility information in COBIE. An owner may outsource maintenance to several different businesses that need to share information. The enterprise scheduling software, used to schedule staff and meetings, has its own view of the same data. If each system is initially configured through the import of the same COBIE data set, if each system uses the same identities for spaces and systems, then these systems will be ready to exchange Service calls sharing expectations and requests.