Service Oriented Scheduling (Part 3): Examples

In parts one and two, I described a model whereby long running processes, including physical processes, can be advertised and invoked within service architectures. A system can advertise when it is willing to offer a service, set prices for different schedules, indicate limitations in its ability to respond, and otherwise describe what it is bringing to market. A system seeking a service can efficiently compare performance characteristics and prices for acquiring / invoking these services. Client and server can negotiate when and how a service is provided. These information exchanges are at the heart of smart grid communications. In this post, I examine some other examples.

Long time readers know that I have long been concerned with getting buildings to respond to their occupants, or, in particular, to the schedules of their occupants. Many of us use our mail or calendar systems to coordinate meetings. It is common practice to invite the room as well, so the others can review the availability of the room for other meetings. Using WS-Calendar, one can post from the enterprise calendar of the resource [room] to the building automation system. This interaction does not require any service advertisement.

Sometimes the meeting organizer wants to compare several rooms. Some of the potential meeting rooms may charge rent or booking fees. There may be a lag the first time a room is scheduled on a given day, as the room is heated up or cooled down before use. Some rooms may use more or less energy, and the building automation through metering and introspection and weather predictions, may be able to calculate this energy requirement. If the building receives price signals from its energy supplier, it may be able to provide comparative prices for different rooms. These prices can be advertised in the service that books each meeting room in a building. A room can be scheduled with less advanced notice than it needs; when this happens the organizer can be notified that the conditions offered by the room will be sub-optimal for some or all of the meeting.

But this information model is useful beyond energy, buildings, and rooms. The patterns of information and communication described above are common to the processes of business and people. Their use opens the processes of business and people to automated negotiation.

Maybe the local coffee shops offer meeting support services. One is less expensive, but needs one day’s notice. The other is more expensive, but can respond with a two-hour’s notice. A scheduling system can search out the options, and offer the organizer choices and prices. Just as the BAS was notified of a meeting for 20, so, too, is the chosen caterer. The service provided by the coffee shop may be a process that arrives with coffee half an hour in advance. The service request can be the same as received by the BAS. The facility’s housekeeping service, whether in-house or outsourced, can receive the same notification, even though the service it offers is to arrange the furniture in advance and to clean up afterwards.

This pattern is a general pattern. Consider scheduling medical facilities. An expensive resource, such as an MRI system, is often scheduled day and night. A one-hour apointment using the MRI may require considerable set-up time. A single patient visit may involve multiple groups with multiple lead times and costs that vary by time of day. Medical groups may provide disabled veterans with transportation to and from the hospital. This transport service may be provided between 7:00 AM and 3:00 AM by in-house staff, requiring three-hour’s notice. After hours, the group may arrange for taxi or medical transport service. Depending upon schedule, patient medical state, origination and destination, and costs, the scheduler may be offered a number of decisions. The software presenting the choices needs only a limited amount of information to filter the services and to present them to the scheduler.

For each of the examples described above, a system is able to select from among services advertised, by schedule, by price, and by requirements, and narrow down to services that fit the need. Such a system may make a final decision of which service to use, or can present the final candidates to an end user with properly distinguishing characteristics. The distinguishing information can easily be exchanged using WS-Calendar and EMIX Terms.

In today’s new economy, people are inventing their own jobs based on radical outsourcing. New service markets are being built around services such as TaskRabbit. Local cooperatives such as LoadedBikes in Chicago offer just-in-time delivery to local and national services through aggregation of peers. Within a class of service, the WS-Calendar and EMIX Terms provide the information needed to select services and to assemble seamless aggregates.

The common information models of schedule and of EMIX Terms are the foundation for even more interesting applications that can be built using peer-to-peer interactions. Peers can compete for constrained resources over time, trading slight changes in schedules using market-like interactions. Because of the simplicity of the interactions, systems assembled in this way can be self-integrating and self-optimizing. This offers interesting possibilities for microgrids and other systems of systems. That conversation is out of scope in this series, but it can begin through thinking about these common information exchanges.

Service Oriented Scheduling - Background Q&A

I received some very good feedback and questions on the first two parts of the Service Oriented Scheduling (SOS) series. In particular, there were questions the relationship between EMIX and WS-Calendar, about the difficulty of creating Calendar artifacts, and about some elements that have been missing from traditional Calendar communications. In this post, I will try to address these.

How do EMIX and iCal relate. Is EMIX an extension of iCal that could be processed by generic iCal tools? Or does it just use the format?

iCalendar is a specification for communications, currently defined by the Internet Engineering Task Force (IETF) in the specification RFC 5545. iCal is particular implementation on Apple systems that exchanges iCalendar information. RFC5545 defines a general pattern for creating schedule components, as well as specific semantics for information within those components. RFC5545 further defines specific components including the VTODO, VJOURNAL, VFREEBUSY, VALARM, and, most familiar to most of us, VEVENT. Each component is defined as a bundle of properties and parameters; each property can have parameters, and each parameter can have properties.

Other RFCs (specifications) published by the IETF define additional components that follow the general patterns and semantic rules established by RFC 5545. VAVAILABILITY and VPOLL are two examples of new components that follow the pattern and semantics of RFC 5545 and so conform to that specification but are part of that specification.

These specifications are so widely used that a number of specialized interactions have been codified. RFC 5546 defines the iCalendar Transport-Independent Interoperability Protocol (iTIP) as well as a mail-based version (IMIP). Many calendar servers support using WebDAV (Distributed Authoring and Versioning) to support scheduling and updating, a specification known as CalDAV (RFC 4791 and RFC 6638). There are many more.

All of these specifications are described, loosely, as iCalendar or iCal.

Each of these communications relied on strings of text that many find quirky to create, quirky to read. Almost every popular calendar system implemented only part of the specifications. To address this confusion, the Calendar and Scheduling Consortium (CalConnect) formed, with wide industry participation, to address issues of interoperability and to further advance these specifications.

In parallel with the formation of WS-Calendar, CalConnect began defining rules for XML serialization of iCalendar objects. The OASIS WS-Calendar Technical Committee prepared normative XML schema (XSD) for iCalendar. XML Schemas can be consumed by many programming tools, removing the drudgery and human errors from creating valid XML artifacts. The two committees worked together closely on this project.

There has been some interoperability testing between these XML artifacts and several “main-line” systems. The open source enterprise calendar “Bedework” is the best source for XML-based exchanges between calendar systems.

Do service interactions require a specific protocol? Must they use XML or can they use JSON or other technologies?

Service Oriented Architecture (SOA) is a style of integration. SOA does not specify any particular protocol or binding. XML over HTTP is the most common, REST and SOAP are the most commonly used implementations of SOA ,but there are others. The CalConnect group is currently defining a standard for JSON serialization of iCalendar objects.

We tried using iCalendar before. We ran up against a wall with granularity. It was too hard to model complex behaviors. Does WS-Calendar address this?

The biggest differences between XML iCalendar and WS-Calendar is that WS-Calendar added some elements for finer grained control and project management type interactions.

Tolerance, precision, and granularity are new semantic elements in WS-Calendar. One can specify whether a response must be at exactly 8:15 am, or can be five minutes early, or up to 10 seconds late, etc. A requester can indicate what precision is required in tracking and reporting time. Granularity combined with vAvailability adds some interesting service advertisement. Consider a service that is available between 9:00 and 10:00 with a granularity of 15 minutes. Such a service can be scheduled only at 4 times: 9:00, 9:15, 9:30, and 9:45.

WS-Calendar also defined temporal relationships. Temporal relations allows the programmer to define schedule sub-routines, known as Sequences. iCalendar already defined relationships as a means to express the two events were related to each other. Temporal relationships describe how to fit events together. B must start right after A. C must start 10 minutes after B ends. D and C must finish at the same time. Events A, B, C, and D together make a Sequence. Each of these events is a valid iCalendar component.

Through a simple algebra of time, if you provide a start time for any of these events, you define the respective start-times for all. WS-Calendar also added a component to advertise and to schedule sequences. A Sequence can be advertised with a single service entry point. Invoking that service includes providing that single start time. The sequence, once defined, can be invoked again and again.

What is the relation between EMIX and WS-Calendar. Does every calendar server understand EMIX?

EMIX is not part of WS-Calendar. iCalendar has included the capability of including a MIME component inside, say, a vEvent. In WS-Calendar, we extended this to include options for an XML payload inside a vComponent.

In energy markets the time and schedule of delivery is very important. EMIX incorporates semantic elements from the WS-Calendar. Some EMIX components are valid iCalendar artifacts. Others are not, but can be transformed into WS-Calendar components or sequences.

When an EMIX element describes time and duration, it does so by reference to and incorporation of the WS-Calendar schema. A parsing routine that understands a duration as expressed in WS-Calendar, will correctly parse a duration as expressed in an EMIX Term.

What industries are likely to adopt EMIX and WS-Calendar first. Is there a codebase out there that includes early adopters? Are any large companies developing products that uses these standards?

CalConnect members tested interactions between existing enterprise calendar systems as these specifications were being developed. As noted above, BedeWork is a good source for implementation. The Paris office of the ARC Informatique has developed a WS-Calendar interface to the building management systems.

WS-Calendar and EMIX are each incorporated into the OASIS Energy Interoperation specification. The OpenADR Alliance is defining interoperable profiles of Energy Interoperation just as the WiFi Alliance defines interoperable profiles of 802.11. OpenADR Alliance members include are an international list of the largest and best known engineering and technology companies as well as some of the largest electric utilities in North America.

OpenADR is being used to implement wide-area distributed scheduling of energy consumption. OpenADR has a limited view of scheduling, based on the times when there is not enough electric power available. Other groups, such as the Transactive Energy Association are exploring more general use of Energy Interoperation.

This series of posts is about using WS-Calendar and a small portion of EMIX for more general purposes.

Service Oriented Scheduling (Part 2): EMIX Terms

In a previous post, I described how WS-Calendar introduces schedules to service interactions. It is now straight-forward for two systems to negotiate operating schedules, and to contract for long running processes. If there are many choices for service provider, choosing the best service may take many negotiations. In this post, I describe how using EMIX terms can allow the service requestor rapidly to narrow the choice of service provider.

EMIX, or Energy Market Information Exchange, is a specification developed to meet the information needs of distributed energy markets. Production, distribution, and storage in power markets, in particular, are based upon physical processes and the value at the point of sale is dependent on the time of delivery. EMIX incorporated many semantic elements from WS-Calendar to describe market products over time.

Power generation, in particular, involves large machines, each with particular operating characteristics. Starting a generator may be more expensive than running a generator. Wear and tear may be driven by operating cycles. In EMIX, we took the relied on the characteristics developed for use in wholesale power markets and simplified them. By simplify, I mean we removed the process knowledge. For service interactions, we do not want to know the processes, only the affects.

From a very large number of characteristics that affect how generation can be marketed, we came up with a short list of Market Terms. Using Market Terms, a Service can indicate whether it would be willing to accept an offer.

Operating Terms

Operating terms describe limitations in how the mechanism behind the service operates. It may be unwieldy to provide the service for less than 10 minutes, or the engine may overheat after 4 hours. Operating Terms include:

• Maximum Run Duration
• Minimum Run Duration
• Minimum Recovery Duration
• Minimum Duration Between Invocations
• Response Time

Schedule Terms

Schedule terms use vAvailability to describe when the service is available. A single system might offer several services that are identical except for the terms, and terms may vary by schedule. Again let’s go back to the plumber who may have different response times and rates based on time of day.

Summary of SO Scheduling

These are not only the terms of generation market, they are the terms for negotiations between any two systems, or even for the plumber I described in part 1. For example, a plumber could publish a schedule (availability) with a labor rate for business hours, another schedule with rate for early evening and Saturday service, and still a third for overnight service. Availability can be stacked; that plumber can lay a short-term unavailability atop the other schedules, interrupting the standing availabilities with a vacation.

Together, WS-Calendar and EMIX Terms provide the semantics needed to advertise, negotiate, and transact for long running services and physical interactions.

Service Oriented Scheduling (Part 1)

Some interesting new interaction patterns, and new business models, can be found by combining WS-Calendar and EMIX Terms. WS-Calendar is a specification for constructing web-services that incorporate iCalendar, the long-established basis for personal scheduling. EMIX is an information model built to support the exchange of market related information between suppliers and buyers of energy.

Service orientation names a pattern for systems interaction in which system exchange minimal information about each other. Service interactions do not specify underlying mechanisms and processes. A system that offers a service does not care which system invokes it; a service can be used by many systems. Service integration pulls system together in a manner analogous to how we build the web; we link pages and applications together without worrying what software operates each server. Service integration maximizes code re-use while enabling rapid evolution of systems.

WS-Calendar addresses the implicit assumption that all services are “instant”. Everyone knows they are not. A merchant might select a credit card processor because of a faster approval service. Still, the request is always “Approve this now!” WS-Calendar defines the messages to request “Do it tomorrow, at 9:00, and keep on doing it for one hour.” As we begin interacting with the internet of things, this capability will grow in importance.

The iCalendar family of standards is broader than the simple meeting request most of us are familiar with. iCalendar describes a family of message types: events, tasks, to-dos, et al. in the core specification, recently updated in RFC 5545. iCalendar also defines a pattern of building messages so that new types can be defined. Two new message types that are drawing interest are Availability and Polling.

vAvailability (all iCalendar message types begin with a “v”) describes how to indicate recurring patterns of time during which one is available (or unavailable). Depending upon application, other information could be included. For example, a plumber could publish a schedule (availability) with a labor rate for business hours, another schedule with rate for early evening and Saturday service, and still a third for overnight service. Availability can be stacked; that plumber can lay a short-term unavailability atop the other schedules, interrupting the standing availabilities with a vacation. vAvailability optionally includes an indication of granularity of schedule perhaps the plumber indicates a one hour minimum. Using WS-Calendar, we have a machine-readable way to advertise when a service is available for invocation.

vPoll addresses the process of “voting” for a schedule. An event organizer can send out a range of times (indicated with vAvailability) for a meeting. Recipients can rank the options, including pricing the various options. After polling, the decision of which time to select is still left to the organizer.

WS-Calendar gives us the semantic tools for machine-to-machines scheduling and optimization of resources.

In a later note, I will describe how EMIX Terms add critical additional information for service oriented interactions in the Internet of Things.