etcd and ZooKeeper

In short: etcd and ZooKeeper are small, strongly consistent data stores that distributed programs use to coordinate configuration, locks and leader election.

In more detail: They store little but important data on several nodes and agree on it via a consensus protocol. etcd uses Raft, ZooKeeper the ZAB protocol.

In Depth

• Data model: etcd: flat key-value store. ZooKeeper: a tree of “znodes” like a file system.
• Watches: Clients get notified of changes instead of polling.
• Leases/ephemeral nodes: An entry disappears when its owner fails. Service registration and leader election build on this.
• Quorum: A majority of nodes is needed (3 or 5). If it is lost, writes stop.
• Use: etcd is the state store of Kubernetes. ZooKeeper long coordinated Kafka and Hadoop.
• Not meant for: large data volumes or heavy write load.

See also: Kubernetes