CoreOS Fest 2016

etcd is a powerful and robust core component of modern infrastructures. Weave Mesh is the gossip protocol at the core of Weave Net, the battle-tested Docker SDN. This talk chronicles the implementation of etcd-on-Mesh, hosting the strongly-consistent etcd API on an eventually-consistent gossip protocol. The project enables developers to use etcd in a much wider variety of contexts than is currently assumed. For example, embedding an etcd cluster in their application and get an ephemeral lock store, or deploying a permanent etcd cluster in a heterogeneous, partially-connected network environment.

In this talk, I describe the core semantics of the Raft protocol, the requirements it has for peer-to-peer communication, and how it might be usable over a packet-oriented, rather than stream-oriented, transport. (Think: UDP rather than TCP.) Then, I describe the basics of gossip protocols, how they model state transfer, and Weave Mesh's particular approach. Finally, I describe how these two models of distribution can be married together, and how each one accommodates and compensates for deficiencies in the other.