Distributed Computing

A model in which independent computers connect over a network and coordinate their work so that, to the user, they behave like a single system.

Parallel computing splits one task across processors that usually share memory or a fast interconnect. Distributed computing spans separate machines that communicate by messages and must tolerate partial failures.

Nodes can fail or become unreachable while others continue, so they need protocols such as Raft or Paxos to agree on a single value or order of events.

Begin with an introduction from the first group, then read the MapReduce and Spanner papers and try the Raft visualisation to see how consensus works.

Partial failure, unreliable networks, clock differences and the need for consensus. The fallacies of distributed computing list common wrong assumptions.

The Raft site, the Paxos Made Simple paper, MIT's distributed systems course and Jepsen's analyses.

The section 'Classic Papers & Courses' lists Paxos Made Simple, Raft Consensus, Jepsen, MIT 6.5840 Distributed Systems, Designing Data-Intensive Applications.

Links were checked when this page was updated. The ecomputernotes.com article timed out, so it was removed. Britannica and the Springer journal page block scripts but loaded in a browser.