Blockchain and the Rise of the Machine Economy

On Christmas Eve, I received some correspondence on using blockchain in the Internet of Things. I have long been convinced that blockchain would be important in smart energy. Dr. Lynne Kiesling has been a leader in calling for the use of blockchain in power markets to create neighborhood energy resilience. This new letter has gotten me wondering how it might be used much more widely.

Blockchain names a suite of technologies that replace central authority with shared moments of reality. Essentially, blockchain is just a record, or ledger, of digital events — one that’s “distributed,” or shared among many different parties. It can only be updated by consensus of a majority of the participants in the system. And, once entered, information can never be erased.

Imagine being in the check-out line at Target and credit card approval system goes off line. So you and the cashier whip out your smart phones and take a picture of the credit card by the receipt. The guy behind you in line, and the bagger take a picture as well. Nobody knows where your money is coming from, or why you are buying what you are, but there are now four identical records of the transaction. If any record is changed, it no longer lines up with the others. Together we create a journal entry in a ledger that cannot be faked or changed, and we did it without central authority or even mutual trust.

Blockchain works without compromising privacy. Participants can record the fact that the event happened, and even that it happened correctly, without knowing or being able to expose confidential details about the subject matter or the parties involved. This explains how bitcoin, the best known of the blockchain technologies, enables black-market transactions; despite the public nature of the ledger, the users themselves can remain completely anonymous.

One key emerging use of the blockchain involves “smart contracts.” Smart contracts rely on the decentralized network to confirm that a contract of any kind was executed properly (or even to execute it automatically), without revealing any confidential information about the parties or the transaction.

While the idea of blockchain is simple enough, it can be difficult to implement. The publicity around bitcoin, including its well-publicized failures, may make it harder for blockchain to gain wide acceptance. Bitcoin does not use a particularly good approach to blockchain. There are better.

The roots of the technology which makes blockchain possible go back to the 1970s. At that time they were already working on elements as fault-tolerant systems, consensus principles and distributed systems. By some definitions, blockchain is in constant use within each of the major cloud infrastructures. Microsoft, Bosch, Samsung, and IBM already have notable efforts on applying blockchain within the internet of things.

The IBM open source work on blockchain has attracted the participation big banks.

http://www.coindesk.com/ibm-launches-open-source-blockchain-project-backed-by-linux-and-big-banks/

The purpose of the Energy Mashup Lab is to develop open source software for systems that can self-assemble into microgrids. This self-assembly is based upon the minimal integration required for Transactive Operation, that is, using a micromarket to operate a microgrid. Nodes in a microgrid buy or sell power over time, and power use is aligned with power supply by the market. This smooths the power consumption within the micromarket, even as it prepares the micromarket as a whole to engage with larger markets. Transactive operation is well understood and long tested in power markets.

A node is either a buyer, a seller (generator), or a trader (storage) (The Lab has identified 8 types of Agents as both necessary and sufficient) than any system can participate in the micromarket, so long as it has a budget. The aggregate of a market is a single market position for the microgrid as a whole, enabling any microgrid to participate in a containing microgrid, also operated as a micromarket. Fractal micromarkets are the *only* mechanism for smart energy that both protects privacy and provides defense in depth cybersecurity.

A key issue in Transactive Operation, especially within a small microgrid, is where is the market? Who keeps track of the transactions? In a larger market, such as that for bulk power generation, there are extensive means to approve participation and to assess penalties for non-performance. In the smallest micromarkets, all parties share common ownership, and so some security and authority issues are minimized.

In local microgrids, say for the neighborhood or the industrial park, it becomes necessary to track transactions accurately. If each house in a neighborhood is a microgrid, then, in contrast to the inside-the-house market with a single owner, the neighborhood market will have multiple owners with multiple interests. All houses could participate in a cloud-based micromarket, accepting a third-party referee, but that means that the neighborhood fails if the communications connection. A neighborhood blockchain market can run even if the cloud communications are interrupted

This spring, Alex Puig is putting together a conference in London to explore the implication of the use of blockchain in the Internet of Things (IoT). Blockchain is seen as the basis of a machine economy supporting three fundamental roles in this economy. (1) Blockchain can track identities in the IoT, including descriptions, locations, and services proffered or needed by each system. The relations between systems, and how they work with each other can be tracked through registration of smart contracts, eliminating the need for intermediaries when a device buys information or hires services from any other device. And of course, blockchain can support payments, even nano-payments, upon successful negotiation of contracts between systems.

In the machine economy, smart devices become independent agents. The services exchanged potentially reach far beyond services that humans buy now. Automobiles could negotiate for higher-speed highway passage. A washing machine could negotiate for the best price on a replacement part, and then order it. Systems could negotiate for temporary use of Wi-Fi. A vending machine not only monitor its own stock, but could barter for bandwidth with passing phones to report and to solicit bids from distributors, and then pay upon delivery of new items.

Blockchain is a natural addendum to micromarket for Transactive operation of microgrids. Such markets already anticipate additional services in congestion, and distribution management. While this use is new, it has been long anticipated. General application of these concepts may take us far deeper into the machine economy.

IOT Apps and Competition for Resources in Seattle

Tomorrow, I am talking about a Resource Framework for the Internet of Things (IoT) at the summit of the AllSeen Alliance.

Traditional consumer programming has concerned itself with only a few resources, i.e., RAM (memory), storage (disk space), and communication (network speed). These programs live atop operating systems and device drivers that engage directly with physical things.

Third-wave Apps in the IoT, though, deal directly with resources. The second wave of the IoT, what I call the Internet of Sensors, may measure resources, but Apps are not competing for resources except, perhaps, bandwidth to report them. Two measurements of air temperature do not compete. And one does not “use up” the temperature that the other one wants.

Third-wave IoT Apps do things, and can only do things to the extent that have access to resources. Resources may be electrical power or heat or water or water pressure, or anything that the systems controlled by an App need to support their purposes.

Some resources exist as a fixed pool that is then drained over time. Other resources may have a steady supply over time. As other IoT Apps require the same resources, the size of the pool varies not by the schedule of its own ebb and flow (think power provided by Solar PV), but the supply changes as other Apps consume the same resources, or perhaps can even be induced to supply more of that resource. Resource availability, the net of supply and demand, is always changing over time.

With a predictable budget for a given resource at any moment in time, Apps must avoid interfering with each other. Sometime this is a competition, but often it may be as simple as avoiding the time that other Apps are using the same resource. Two Apps that use the same resource at the same time may both fail if there is a shortage of resources adequate for simultaneous operation. This is a problem of a moment in time. If one can delay its operation, or the other can accelerate its operation, they may be able to perform all functions, to get access to all of the resource each needs, by simply avoiding each other.

Traditional solutions to this problem posit a master controller, a single controlling program that understands each application and its needs. This works best when all systems and apps are provided by the same manufacturer, and the systems work together as slaves do: on command, as directed, and interchangeably.

With a resource framework, we hope to define a framework within which Apps in the same space can negotiate for resources over time. We can use the specifications built for Smart Energy, to negotiate power use and supply, for other commodities as well.

Resource Frameworks to Integrate the IoT

Last month in Monterrey I gave several talks about the how to make diverse apps in the Internet of Things (IoT) work together. It was an interesting crowd, with real problems in distributed telemetry (wastewater monitoring), big data (floor mat traffic analysis for retail) and missionary work. The problems ranged from few resource constraints to very tight constraints.

One of my favorite IoT App there was driven by missionary work, combining clean water and Gospel messages in Africa. Both were powered by small solar PV installations, working within a tight budget of electrical power to provide fresh water and direct broadcast to cell phones. I am always amazed at how deep into the undeveloped world quite advanced phones have penetrated.

As the conference continued, I saw the case for my themes be made by the conference participants. By the end, I was seeing the resource framework developed for the US National Smart Grid effort everywhere. The challenge in that national effort was to improve the ability of the power grid and its successors to solve problems of supply and demand locally, and to do so in a manner that increases consumer choice, and accepts rapid technology innovation.

The solution then and now was micromarkets for power. Micromarkets have performed better than control systems in numerous published reports. Micromarkets support simple addition and removal of participation systems; they support very light integration. Micromarkets are insensitive to the processes embedded in technology diversity, and so can accept innovation.

In the national efforts, we went beyond power markets. Our communications specifications supported capacity markets, congestion markets, transmission markets, and even ancillary services markets. The complex Power Reserves market was modelled as a simpler options market. In each case, the market was a resource whose value was determined by time of delivery.

The components were time, product, and market services. For time, we developed communications for machine negotiation of human-centric schedules (WS-Calendar). For product, we developed abstract (an important point) models for describing products over time (EMIX), incorporating WS-Calendar. For market services, we developed the communication patterns necessary to negotiate for EMIX products.

What came to the fore was the importance of abstraction. EMIX was created without defining any particular product. We defined concrete types, meaning real products, for power and capacity and congestion, and so on. In Monterrey, we discussed the same communications for network bandwidth in data centers, for apps on limited cellular data plans, as well as for water-power combinations in sub-Saharan Africa. This month in Seattle, we will discuss using similar products to control storm water surges in wastewater systems.

As the Internet of Things becomes the Internet of Doing Things, Apps will need to work within the boundaries of locally available resources. With more and more Apps, they will evolve to share these resources with each other. Integrating IoT applications through resource frameworks enables different Apps to share resources without knowing anything about each other. This “ignorance” is essential to rapid evolution of Apps in the IoT.

Resource markets are the simplest way to smooth interactions in a resource constrained world. In the IoT, there may be far more things than in traditional systems integration. Allocating budgets to participants in resource micromarkets may be the next thorny problem in the IoT. That leads to discussions of overlapping taxa and policy-based budgeting, subjects for another day.

I will be talking about resource frameworks this October at the AllSeen Alliance meeting in Seattle. I hope to see some of you there.

AllJoyn and the Azure Cloud

This was a fascinating week at the TechIntersection conference. TechIntersection is a new conference with three intertwining tracks, Architecture, Security, and IoT (Internet of Things}. The need for such cross pollination is obvious. Apps built for the Internet of Things rarely take account of the issues they will require to scale to millions of installations, each potentially interacting with other Apps from other developers. Security doesn’t really understand the special needs of things, just as IoT Apps often violate enterprise expectations for security and privacy. Enterprise architects have some nifty new tools that will provide great value to IoT developers, but they have no idea what an avalanche of data and connections that is headed toward their data center.

On the IoT side, we had developers of domotic systems, and mobile vehicle systems, intelligent floor pad market awareness systems and even a church missionary group working at the intersection of water and power and gospel in integrated delivery systems.

The architecture sessions covered approaches to modularization, project management, performance engineering and security from the always solid iDesign team. It included surveys of internet-centric design {notice I did not say web-centric), platform optimization, and microservices. There were also in-depth drill downs into the latest Azure technologies and service fabrics from Microsoft. My favorite demonstration requested a

The security was driven by a developer-centered approach to security with an emphasis on not doing stupid things. It included always enjoyable live penetrations of live sites using google tools and a little creativity. One government site still had an exposure of its own security and of the database behind it that had persisted for years.  It also included a live hijacking of all the phones and PCs in the room.

It was truly a high-powered set of speakers.

In my presentations, I shared some scars from my own decades of development, and cautioned about optimistic notions about how many IoT points could simultaneously deliver high speed telemetry to the cloud. People always seem to think that getting one point and then another one is the same as getting two point in one request. Emotionally (by which I mean not analytically), they seem to leave out the traffic to create a session, establish security, and then send a packet. They don’t think of the inefficiencies of sending half empty packets. Caching and batching of telemetry is going to matter much more than the glib presentations.

I mentioned microservices before. Microservices are isolated bits of code that provide some service. The Azure service fabric is able to support very large numbers of simultaneous microservices, both within a system, and across systems. Barry Briggs demonstrated the amusing and powerful open source Cloud Sheet. Each microservice was a small bit of code that was able to interpret an equation, including an equation the referenced another service by name. The services were all assigned names similar to A1, A2, and C3. There was also a web-based interface in which each microservice was referenced by a single on-screen cell. Cloud Sheet is a cloud-based scalable spreadsheet, able to spread across multiple cores, multiple systems, and even multiple data centers.

Well that was fun, but it almost does not seem worthwhile. That judgment, though is before considering additional code in each service. Barry went on to load million-row data sets into some of the “supercells”, some pulled from web sites such as NOAA, and some from “local” text files. The supercells would parse the data and support statistical queries (average of all humidity readings in 1984) from other cells. Suddenly a cute demonstration was a tool of surprising power.

My talks focused on a time and resource-based model for interactions between apps. If Apps, while performing their functions, influence the use of a constraining resource, i.e., power, then they can communicate between themselves to smooth and optimize that resource use, using lightweight services based on the three market oriented communication specifications (ws-calendar, EMIX, Energy Interoperation) of smart energy. WS-Calendar can be used even in non-resource constrained decision-making to negotiate optimum run times.

After the conference I drove up California 1 along the coast to Half Moon Bay to visit my nephew. As always, my world tour of coffee shops, writing in each one, continues.