Odds and Ends: Looking to 2015

I have been quiet here for too long, and have made a New Year’s resolution to get back to writing. Many of my recent projects I cannot write about, for competitive or contractual reasons. Still, there are some big themes coming to light, ones that I have been writing about for years, and that are now hitting the market.

Microgrids, broadly defined, have been a place with a lot of demonstrated movement in this last year. The most expensive thing about the obsolete grid is the assumption that everything happens centrally, and that the local node does not have any responsibility. This might be true if our world was run on incandescent light bulbs and ceramic space heaters. In a digital world, aggregate load and rhomboidal curves are growing problems, ones that cost a lot of power and shorten the lives of a lot of equipment.

Storage remains the most important enabling technology for alternate and distributed energy. The storage symposium at the California Energy Commission on December 1 brought some powerful choices into the open. Grid-scale storage is important, and will grow more important. I think that neighborhood scale, and even commercial building scale storage will have more effect in the long term. Look to announcements in the mid-year.

Smart water and smart energy continue to entangle themselves. Pumped water is pre-consumed energy, stored for future use. Reliable distributed energy fits naturally with reliable distributed water pumping, which is the key to avoiding sewage spills. This challenge has been met with portable generators and other technologies that require nimble deployments of work forces. Batteries with up-front capital costs and life spans of only four or five years, don’t make sense here. I look to experiments with 25 and even 45 year storage systems in 2015.

Golf courses have a reputation as despoilers of the environment, with over fertilization and chemical pest control leading to run-off and despoliation of habitat. For years the best practices in turf management have made that reputation un-true for the best run golf courses. Look to a combination of distributed energy, energy storage, water pumping, and the DC club house to appear at selected locations this year. Golf courses may be just the right size to lead the way in new microgrid approaches.

New players keep cropping up applying digital signal processing to power distribution. Early players, some of which I have written about before, have struggled to connect work in their labs to customer service oriented organizations. Early adopters are scared off by costs that have not dropped yet, and not quite understanding the offerings. New players like 3DFS are preparing production offerings. One of these guys is going to make it big, particularly in light industrial or commercial settings which rely on motors.

The high cost of per-site integration remains a brake on microgrid deployment. Semantic integration is going to be critical to reducing this integration cost. Maybe this is the year…

I hope to be more diligent in writing this year. Keep those notes coming.

tc

Bidding for Schedules—VPOLL and VAVAILABILITY

Last week I watched live multi-vendor demonstrations using the new specifications vPoll and vAvailability. These extend calendar interactions to support live negotiations about schedule and performance. These negotiations can be machine-to-machine (M2M) or augmented by human input. These were not applications, these were live interactions between mainstream calendar servers. The testing used simple user interfaces, just enough to operate the tests. These simple information exchanges extend existing systems for schedule negotiations into automated polling and bidding.

As I have written before, I spent three days with the Calendaring and Scheduling Consortium (CalConnect) at AOL headquarters in the suburbs of Washington DC. Although the meeting was technical, it was against a backdrop of a scandal in the VA that is summed up, by the press, in the single word “scheduling”. Members of CalConnect went quietly about demonstrating their new work, based on iCalendar, that points to a new way forward, one that extends schedules negotiations into automated polling and bidding. CalConnect rules forbid any direct discussion of who was at the meeting, and what their precise interactions are.

CalConnect works to extend the reach of existing Calendar and Schedule systems (new specification development) while testing whether they work in exchanges between vendors with different code bases. Sometimes the code is changed, and sometimes the specifications.

ICalendar (RFC5545) defines a common grammar for creating information exchanges information related to schedules and defines a several such exchanges. The iCalendar event (vEvent) is familiar to anyone who has added a meeting to their calendar by clicking on a web site or opening an email. ICalendar defines other common exchanges such as tasks (vToDo) and alarms (vAlarm). OASIS WS-Calendar describes how to use iCalendar inside service oriented exchanges, with a focus on describing sets of schedule information that hang together in a series. There is a growing use of WS-Calendar world-wide in energy markets and smart grids.

vPoll is a new iCalendar exchange that uses iCalendar grammar to present a choice. Assume it is a simple meeting (vEvent). A vPoll could present a number of meeting times and ask the potential participants to vote. This is different than asking for a summary of when a recipient id currently scheduled (vFreeBusy). A recipient may choose to vote for a time knowing he can cancel a meeting. A recipient may be keeping that afternoon free for a visitor from out of town—even though nothing is scheduled. A recipient, for security reasons, may choose not to share any free-busy information.

vPoll further defines how responses go back to the originator. A poll typically includes an ending time for polling. The originator may choose to schedule the meeting in accord with the most votes, or when a few key personnel can make it or for any other reason. vPoll does not define the business rules or the application, just the messages and message pattern.

vPolls may be tied to prices rather than to votes. Maybe the single best price, or the three best prices, get invited to the meeting. Maybe prices determine the venue. The vPoll specification defines the BASIC Poll Type. Others will be defined later for specific use cases.

A recipient may choose to respond to a vPoll with vAvailability. Availability uses recurring patterns to indicate when something might happen, and what it might cost. Consider a meeting room in a commercial building. It can be scheduled during business house, using the calendar, for free. After hours, because of additional security and custodial needs, the same room can be scheduled for $200 / hour with a minimum charge of $300. These schedules can be expressed with vAvailability.

So far, I have described vPoll using only meetings. A Poll can also include tasks (vToDo). A task is often distinguished by its required completion time. Recipients can use their calendars to bid on various completion times. Presumably, bid for completion tomorrow will be more costly than a bid for completion next month. VPoll can gather the bids and present them to the originator, to choose one.

Most of us think of calendar communications (ITIP) conveyed inside email (IMIP). We open email and we accept the iCalendar request. At the CalConnect meeting this year, multiple participants demonstrated interoperable iSchedule, used for direct server-to-server communications. This promotes the polling described above from the personal to the enterprise. I will write more about iSchedule later.

Service Oriented Scheduling - Background Q&A

I received some very good feedback and questions on the first two parts of the Service Oriented Scheduling (SOS) series. In particular, there were questions the relationship between EMIX and WS-Calendar, about the difficulty of creating Calendar artifacts, and about some elements that have been missing from traditional Calendar communications. In this post, I will try to address these.

How do EMIX and iCal relate. Is EMIX an extension of iCal that could be processed by generic iCal tools? Or does it just use the format?

iCalendar is a specification for communications, currently defined by the Internet Engineering Task Force (IETF) in the specification RFC 5545. iCal is particular implementation on Apple systems that exchanges iCalendar information. RFC5545 defines a general pattern for creating schedule components, as well as specific semantics for information within those components. RFC5545 further defines specific components including the VTODO, VJOURNAL, VFREEBUSY, VALARM, and, most familiar to most of us, VEVENT. Each component is defined as a bundle of properties and parameters; each property can have parameters, and each parameter can have properties.

Other RFCs (specifications) published by the IETF define additional components that follow the general patterns and semantic rules established by RFC 5545. VAVAILABILITY and VPOLL are two examples of new components that follow the pattern and semantics of RFC 5545 and so conform to that specification but are part of that specification.

These specifications are so widely used that a number of specialized interactions have been codified. RFC 5546 defines the iCalendar Transport-Independent Interoperability Protocol (iTIP) as well as a mail-based version (IMIP). Many calendar servers support using WebDAV (Distributed Authoring and Versioning) to support scheduling and updating, a specification known as CalDAV (RFC 4791 and RFC 6638). There are many more.

All of these specifications are described, loosely, as iCalendar or iCal.

Each of these communications relied on strings of text that many find quirky to create, quirky to read. Almost every popular calendar system implemented only part of the specifications. To address this confusion, the Calendar and Scheduling Consortium (CalConnect) formed, with wide industry participation, to address issues of interoperability and to further advance these specifications.

In parallel with the formation of WS-Calendar, CalConnect began defining rules for XML serialization of iCalendar objects. The OASIS WS-Calendar Technical Committee prepared normative XML schema (XSD) for iCalendar. XML Schemas can be consumed by many programming tools, removing the drudgery and human errors from creating valid XML artifacts. The two committees worked together closely on this project.

There has been some interoperability testing between these XML artifacts and several “main-line” systems. The open source enterprise calendar “Bedework” is the best source for XML-based exchanges between calendar systems.

Do service interactions require a specific protocol? Must they use XML or can they use JSON or other technologies?

Service Oriented Architecture (SOA) is a style of integration. SOA does not specify any particular protocol or binding. XML over HTTP is the most common, REST and SOAP are the most commonly used implementations of SOA ,but there are others. The CalConnect group is currently defining a standard for JSON serialization of iCalendar objects.

We tried using iCalendar before. We ran up against a wall with granularity. It was too hard to model complex behaviors. Does WS-Calendar address this?

The biggest differences between XML iCalendar and WS-Calendar is that WS-Calendar added some elements for finer grained control and project management type interactions.

Tolerance, precision, and granularity are new semantic elements in WS-Calendar. One can specify whether a response must be at exactly 8:15 am, or can be five minutes early, or up to 10 seconds late, etc. A requester can indicate what precision is required in tracking and reporting time. Granularity combined with vAvailability adds some interesting service advertisement. Consider a service that is available between 9:00 and 10:00 with a granularity of 15 minutes. Such a service can be scheduled only at 4 times: 9:00, 9:15, 9:30, and 9:45.

WS-Calendar also defined temporal relationships. Temporal relations allows the programmer to define schedule sub-routines, known as Sequences. iCalendar already defined relationships as a means to express the two events were related to each other. Temporal relationships describe how to fit events together. B must start right after A. C must start 10 minutes after B ends. D and C must finish at the same time. Events A, B, C, and D together make a Sequence. Each of these events is a valid iCalendar component.

Through a simple algebra of time, if you provide a start time for any of these events, you define the respective start-times for all. WS-Calendar also added a component to advertise and to schedule sequences. A Sequence can be advertised with a single service entry point. Invoking that service includes providing that single start time. The sequence, once defined, can be invoked again and again.

What is the relation between EMIX and WS-Calendar. Does every calendar server understand EMIX?

EMIX is not part of WS-Calendar. iCalendar has included the capability of including a MIME component inside, say, a vEvent. In WS-Calendar, we extended this to include options for an XML payload inside a vComponent.

In energy markets the time and schedule of delivery is very important. EMIX incorporates semantic elements from the WS-Calendar. Some EMIX components are valid iCalendar artifacts. Others are not, but can be transformed into WS-Calendar components or sequences.

When an EMIX element describes time and duration, it does so by reference to and incorporation of the WS-Calendar schema. A parsing routine that understands a duration as expressed in WS-Calendar, will correctly parse a duration as expressed in an EMIX Term.

What industries are likely to adopt EMIX and WS-Calendar first. Is there a codebase out there that includes early adopters? Are any large companies developing products that uses these standards?

CalConnect members tested interactions between existing enterprise calendar systems as these specifications were being developed. As noted above, BedeWork is a good source for implementation. The Paris office of the ARC Informatique has developed a WS-Calendar interface to the building management systems.

WS-Calendar and EMIX are each incorporated into the OASIS Energy Interoperation specification. The OpenADR Alliance is defining interoperable profiles of Energy Interoperation just as the WiFi Alliance defines interoperable profiles of 802.11. OpenADR Alliance members include are an international list of the largest and best known engineering and technology companies as well as some of the largest electric utilities in North America.

OpenADR is being used to implement wide-area distributed scheduling of energy consumption. OpenADR has a limited view of scheduling, based on the times when there is not enough electric power available. Other groups, such as the Transactive Energy Association are exploring more general use of Energy Interoperation.

This series of posts is about using WS-Calendar and a small portion of EMIX for more general purposes.

Service Oriented Scheduling (Part 2): EMIX Terms

In a previous post, I described how WS-Calendar introduces schedules to service interactions. It is now straight-forward for two systems to negotiate operating schedules, and to contract for long running processes. If there are many choices for service provider, choosing the best service may take many negotiations. In this post, I describe how using EMIX terms can allow the service requestor rapidly to narrow the choice of service provider.

EMIX, or Energy Market Information Exchange, is a specification developed to meet the information needs of distributed energy markets. Production, distribution, and storage in power markets, in particular, are based upon physical processes and the value at the point of sale is dependent on the time of delivery. EMIX incorporated many semantic elements from WS-Calendar to describe market products over time.

Power generation, in particular, involves large machines, each with particular operating characteristics. Starting a generator may be more expensive than running a generator. Wear and tear may be driven by operating cycles. In EMIX, we took the relied on the characteristics developed for use in wholesale power markets and simplified them. By simplify, I mean we removed the process knowledge. For service interactions, we do not want to know the processes, only the affects.

From a very large number of characteristics that affect how generation can be marketed, we came up with a short list of Market Terms. Using Market Terms, a Service can indicate whether it would be willing to accept an offer.

Operating Terms

Operating terms describe limitations in how the mechanism behind the service operates. It may be unwieldy to provide the service for less than 10 minutes, or the engine may overheat after 4 hours. Operating Terms include:

• Maximum Run Duration
• Minimum Run Duration
• Minimum Recovery Duration
• Minimum Duration Between Invocations
• Response Time

Schedule Terms

Schedule terms use vAvailability to describe when the service is available. A single system might offer several services that are identical except for the terms, and terms may vary by schedule. Again let’s go back to the plumber who may have different response times and rates based on time of day.

Summary of SO Scheduling

These are not only the terms of generation market, they are the terms for negotiations between any two systems, or even for the plumber I described in part 1. For example, a plumber could publish a schedule (availability) with a labor rate for business hours, another schedule with rate for early evening and Saturday service, and still a third for overnight service. Availability can be stacked; that plumber can lay a short-term unavailability atop the other schedules, interrupting the standing availabilities with a vacation.

Together, WS-Calendar and EMIX Terms provide the semantics needed to advertise, negotiate, and transact for long running services and physical interactions.