Houses, Papers, and Effects: Updated

This blog writes about control systems, sensors, and the internet of things (IoT). Long-time readers know that I am concerned about privacy, and how the techniques of big data can erode privacy through accumulation of the most insignificant facts. Occasionally, I even slide over into my perception that the Supreme Court Justices all too frequently err because they do not understand technology, and how easy it is to erode the protections our Constitution grants us.

In pre-revolutionary times, officers of the crown used writs of assistance to justify widespread searches of people and their possessions. The business documents and correspondence would frequently be confiscated and or destroyed. This violated the rights of Englishmen, long established, to be secure in their homes from even the King’s men, unless there was a warrant served, based upon testimony made under oath. This violation of long-standing law was a significant factor in convincing many that a break with England was necessary.

In the 18th century, people kept their most significant documents in their desk at home. If they wanted to transport their papers or to conduct business, they would bring their papers, often in a locked box. There was no easy way to make copies of these documents; if they were lost they were irreplaceable. There was no general right recognized which permitted government agents to check the contents of these papers. In the Declaration of Independence, the then colonists chastised their government for “for abolishing the free System of English Laws in a neighbouring Province, establishing therein an Arbitrary government, and enlarging its Boundaries so as to render it at once an example and fit instrument for introducing the same absolute rule into these Colonies”

The Bill of Rights codifies things that governments must not do to its citizens, no matter their motive. The 4^th amendment codifies the handling of personal and business documents.

“The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.”

Even mail, sorted, handled, and delivered by a government agency, the post office, could not be searched while in transit without specific warrant.

Today, we store our documents in the cloud. If we don’t store them there, we back them up to the cloud. Our correspondence goes through Gmail. We share documents with partners using DropBox. Every persona and professional document we have is stored on a computer somewhere, and usually off-site.

Because the Justices have not understood technology, they have treated this information as if it were transactional documents belonging to the company that holds it. Electronic documents held in the clouds by a third party today receive almost no protection. These documents are indistinguishable from the private correspondence and papers of colonial times. They are no more novel than papers typed out with a typewriter were different from those hand-copied in prior times.

There are several proposed laws in congress that would assign to these electronic documents the same protections as are held by paper documents. In essence, these laws would declare that our electronic backups, our email, and our cloud shared storage are exactly the type of papers and effects described in the fourth amendment.

A petition as whitehouse.gov asks the executive branch to support these laws.

https://petitions.whitehouse.gov/petition/reform-ecpa-tell-government-get-warrant/nq258dxk

Please, go to whitehouse.gov and sign the petition.

Finding a Needle in the Internet of Things (part 1)

Things cost what they cost to install. Ongoing charges are, in the short term, fixed. Value may be the only thing you can control. In the Internet of Things, value will be determined by how many ways you can use that Thing. Value will be determined by how many different uses can use that thing. Some of those users will be other things.

Things (as in the Internet Of…) tend to be commodities. One thing is inherently like another. Once I have more than a small number of things, I need a way to distinguish between sensors, between pieces of building equipment, and even to distinguish between tangible services such as catering and transport services. One of the first places that all of us have seen things on the internet is in conference rooms, vehicles, and tools as they are scheduled within enterprise scheduling systems.

Discovery is essential to agile integration. If systems can discover each other, we do not need to map them. If systems can understand what they discover, they can integrate themselves. One of the requirements is common semantics, that is agreements as to how something is described (or describes itself). This post is about directory directory services for the internet of things.

In Calendars, people are identified by vCards. VCards are deeply embedded in CalDav, stil the most common protocol for writing new Calendar Apps and user interfaces. Many email systems support attaching your vCard to each outgoing message. VCards are electronic business cards, and just as in traditional business cards, different people choose to include different information on them. Also like paper business cards, various companies and organizations have developed their information and format standards for vCards.

After email, the most common place to find a vCard is in a directory. There is a deep historic link between the Lightweight Directory Access Protocol (LDAP) and the vCard. LDAP is used to manage security as well as directory services in the biggest organizations. The link between directories and security is as natural as is role-based security. LDAP records include have the same variability as does the vCard standard. At some level, though, LDAP is the thing you use to get a vCard.

LDAP supports diversity of information returned, including security on particular aspects of the information. Over time, a varied set of what I will call here, for brevity, vCard profiles have been developed. These often have object-type characteristics. For example, the inetOrgPerson (RFC2798) is defined as a standard set of extensions to the organizationalPerson as defined in the ITU standard X.521. Many colleges and universities have collaborated to define the eduPerson to handle students, grad students, and faculty.

In LDAP, there is a long history returning different information about the same object in different security contexts. At universities, an LDAP will return different information when asked about a student or about staff, even when the target is the same person. Just as in medical information, the portions of student information that can be exposed to query vary widely by querier and context, and are controlled by federal privacy law. There are many security contexts, including when domestic relations go wrong, under which access to some attributes of a particular directory entry is limited. LDAP also supports multi-valued attributes easily. This is in sharp contrast with the normal expectations from, say, a SQL query.

In calendars, things that are not people are designated as Resources. Improvements to Resource vCards is a long-standing project of CalConnect. Some Resources may have a schedule, but not be schedulable. This usually means that it is scheduled by someone else using means you do not have access to, but to the user, it is not schedulable even so. Schedulable Resources may use vAvailability to indicate when and how they can be scheduled. It is of course possible, even probable, that different entities will receive a different vAvailability from the same Resource.

It will come as no surprise to my readers that I now come around to building-based resources. These are most frequently public rooms and building systems. Public rooms are invited to meetings as are other attendees. Smart buildings can optimize energy use while preserving amenity if they know when and by whom the building will be used. In a related post, I will describe how vCards for building-based resources will be standardized.

A key consumer of secure directory services for systems will be other systems. With discovery and directory services, they can find each other to interact. Through developing calendar standards, including vAvailability, they will learn when they can interact.

Secure discovery, self-integration, and autonomous assembly will be part of how we maximize the value of the internet of things.

Service Oriented Scheduling (Part 3): Examples

In parts one and two, I described a model whereby long running processes, including physical processes, can be advertised and invoked within service architectures. A system can advertise when it is willing to offer a service, set prices for different schedules, indicate limitations in its ability to respond, and otherwise describe what it is bringing to market. A system seeking a service can efficiently compare performance characteristics and prices for acquiring / invoking these services. Client and server can negotiate when and how a service is provided. These information exchanges are at the heart of smart grid communications. In this post, I examine some other examples.

Long time readers know that I have long been concerned with getting buildings to respond to their occupants, or, in particular, to the schedules of their occupants. Many of us use our mail or calendar systems to coordinate meetings. It is common practice to invite the room as well, so the others can review the availability of the room for other meetings. Using WS-Calendar, one can post from the enterprise calendar of the resource [room] to the building automation system. This interaction does not require any service advertisement.

Sometimes the meeting organizer wants to compare several rooms. Some of the potential meeting rooms may charge rent or booking fees. There may be a lag the first time a room is scheduled on a given day, as the room is heated up or cooled down before use. Some rooms may use more or less energy, and the building automation through metering and introspection and weather predictions, may be able to calculate this energy requirement. If the building receives price signals from its energy supplier, it may be able to provide comparative prices for different rooms. These prices can be advertised in the service that books each meeting room in a building. A room can be scheduled with less advanced notice than it needs; when this happens the organizer can be notified that the conditions offered by the room will be sub-optimal for some or all of the meeting.

But this information model is useful beyond energy, buildings, and rooms. The patterns of information and communication described above are common to the processes of business and people. Their use opens the processes of business and people to automated negotiation.

Maybe the local coffee shops offer meeting support services. One is less expensive, but needs one day’s notice. The other is more expensive, but can respond with a two-hour’s notice. A scheduling system can search out the options, and offer the organizer choices and prices. Just as the BAS was notified of a meeting for 20, so, too, is the chosen caterer. The service provided by the coffee shop may be a process that arrives with coffee half an hour in advance. The service request can be the same as received by the BAS. The facility’s housekeeping service, whether in-house or outsourced, can receive the same notification, even though the service it offers is to arrange the furniture in advance and to clean up afterwards.

This pattern is a general pattern. Consider scheduling medical facilities. An expensive resource, such as an MRI system, is often scheduled day and night. A one-hour apointment using the MRI may require considerable set-up time. A single patient visit may involve multiple groups with multiple lead times and costs that vary by time of day. Medical groups may provide disabled veterans with transportation to and from the hospital. This transport service may be provided between 7:00 AM and 3:00 AM by in-house staff, requiring three-hour’s notice. After hours, the group may arrange for taxi or medical transport service. Depending upon schedule, patient medical state, origination and destination, and costs, the scheduler may be offered a number of decisions. The software presenting the choices needs only a limited amount of information to filter the services and to present them to the scheduler.

For each of the examples described above, a system is able to select from among services advertised, by schedule, by price, and by requirements, and narrow down to services that fit the need. Such a system may make a final decision of which service to use, or can present the final candidates to an end user with properly distinguishing characteristics. The distinguishing information can easily be exchanged using WS-Calendar and EMIX Terms.

In today’s new economy, people are inventing their own jobs based on radical outsourcing. New service markets are being built around services such as TaskRabbit. Local cooperatives such as LoadedBikes in Chicago offer just-in-time delivery to local and national services through aggregation of peers. Within a class of service, the WS-Calendar and EMIX Terms provide the information needed to select services and to assemble seamless aggregates.

The common information models of schedule and of EMIX Terms are the foundation for even more interesting applications that can be built using peer-to-peer interactions. Peers can compete for constrained resources over time, trading slight changes in schedules using market-like interactions. Because of the simplicity of the interactions, systems assembled in this way can be self-integrating and self-optimizing. This offers interesting possibilities for microgrids and other systems of systems. That conversation is out of scope in this series, but it can begin through thinking about these common information exchanges.

Service Oriented Scheduling - Background Q&A

I received some very good feedback and questions on the first two parts of the Service Oriented Scheduling (SOS) series. In particular, there were questions the relationship between EMIX and WS-Calendar, about the difficulty of creating Calendar artifacts, and about some elements that have been missing from traditional Calendar communications. In this post, I will try to address these.

How do EMIX and iCal relate. Is EMIX an extension of iCal that could be processed by generic iCal tools? Or does it just use the format?

iCalendar is a specification for communications, currently defined by the Internet Engineering Task Force (IETF) in the specification RFC 5545. iCal is particular implementation on Apple systems that exchanges iCalendar information. RFC5545 defines a general pattern for creating schedule components, as well as specific semantics for information within those components. RFC5545 further defines specific components including the VTODO, VJOURNAL, VFREEBUSY, VALARM, and, most familiar to most of us, VEVENT. Each component is defined as a bundle of properties and parameters; each property can have parameters, and each parameter can have properties.

Other RFCs (specifications) published by the IETF define additional components that follow the general patterns and semantic rules established by RFC 5545. VAVAILABILITY and VPOLL are two examples of new components that follow the pattern and semantics of RFC 5545 and so conform to that specification but are part of that specification.

These specifications are so widely used that a number of specialized interactions have been codified. RFC 5546 defines the iCalendar Transport-Independent Interoperability Protocol (iTIP) as well as a mail-based version (IMIP). Many calendar servers support using WebDAV (Distributed Authoring and Versioning) to support scheduling and updating, a specification known as CalDAV (RFC 4791 and RFC 6638). There are many more.

All of these specifications are described, loosely, as iCalendar or iCal.

Each of these communications relied on strings of text that many find quirky to create, quirky to read. Almost every popular calendar system implemented only part of the specifications. To address this confusion, the Calendar and Scheduling Consortium (CalConnect) formed, with wide industry participation, to address issues of interoperability and to further advance these specifications.

In parallel with the formation of WS-Calendar, CalConnect began defining rules for XML serialization of iCalendar objects. The OASIS WS-Calendar Technical Committee prepared normative XML schema (XSD) for iCalendar. XML Schemas can be consumed by many programming tools, removing the drudgery and human errors from creating valid XML artifacts. The two committees worked together closely on this project.

There has been some interoperability testing between these XML artifacts and several “main-line” systems. The open source enterprise calendar “Bedework” is the best source for XML-based exchanges between calendar systems.

Do service interactions require a specific protocol? Must they use XML or can they use JSON or other technologies?

Service Oriented Architecture (SOA) is a style of integration. SOA does not specify any particular protocol or binding. XML over HTTP is the most common, REST and SOAP are the most commonly used implementations of SOA ,but there are others. The CalConnect group is currently defining a standard for JSON serialization of iCalendar objects.

We tried using iCalendar before. We ran up against a wall with granularity. It was too hard to model complex behaviors. Does WS-Calendar address this?

The biggest differences between XML iCalendar and WS-Calendar is that WS-Calendar added some elements for finer grained control and project management type interactions.

Tolerance, precision, and granularity are new semantic elements in WS-Calendar. One can specify whether a response must be at exactly 8:15 am, or can be five minutes early, or up to 10 seconds late, etc. A requester can indicate what precision is required in tracking and reporting time. Granularity combined with vAvailability adds some interesting service advertisement. Consider a service that is available between 9:00 and 10:00 with a granularity of 15 minutes. Such a service can be scheduled only at 4 times: 9:00, 9:15, 9:30, and 9:45.

WS-Calendar also defined temporal relationships. Temporal relations allows the programmer to define schedule sub-routines, known as Sequences. iCalendar already defined relationships as a means to express the two events were related to each other. Temporal relationships describe how to fit events together. B must start right after A. C must start 10 minutes after B ends. D and C must finish at the same time. Events A, B, C, and D together make a Sequence. Each of these events is a valid iCalendar component.

Through a simple algebra of time, if you provide a start time for any of these events, you define the respective start-times for all. WS-Calendar also added a component to advertise and to schedule sequences. A Sequence can be advertised with a single service entry point. Invoking that service includes providing that single start time. The sequence, once defined, can be invoked again and again.

What is the relation between EMIX and WS-Calendar. Does every calendar server understand EMIX?

EMIX is not part of WS-Calendar. iCalendar has included the capability of including a MIME component inside, say, a vEvent. In WS-Calendar, we extended this to include options for an XML payload inside a vComponent.

In energy markets the time and schedule of delivery is very important. EMIX incorporates semantic elements from the WS-Calendar. Some EMIX components are valid iCalendar artifacts. Others are not, but can be transformed into WS-Calendar components or sequences.

When an EMIX element describes time and duration, it does so by reference to and incorporation of the WS-Calendar schema. A parsing routine that understands a duration as expressed in WS-Calendar, will correctly parse a duration as expressed in an EMIX Term.

What industries are likely to adopt EMIX and WS-Calendar first. Is there a codebase out there that includes early adopters? Are any large companies developing products that uses these standards?

CalConnect members tested interactions between existing enterprise calendar systems as these specifications were being developed. As noted above, BedeWork is a good source for implementation. The Paris office of the ARC Informatique has developed a WS-Calendar interface to the building management systems.

WS-Calendar and EMIX are each incorporated into the OASIS Energy Interoperation specification. The OpenADR Alliance is defining interoperable profiles of Energy Interoperation just as the WiFi Alliance defines interoperable profiles of 802.11. OpenADR Alliance members include are an international list of the largest and best known engineering and technology companies as well as some of the largest electric utilities in North America.

OpenADR is being used to implement wide-area distributed scheduling of energy consumption. OpenADR has a limited view of scheduling, based on the times when there is not enough electric power available. Other groups, such as the Transactive Energy Association are exploring more general use of Energy Interoperation.

This series of posts is about using WS-Calendar and a small portion of EMIX for more general purposes.