it-ebooks - Distributed Systems Engineering Lecture notes (MIT 6.824)

Note: These lecture notes were slightly modified from the ones posted on the 6.824 course website from Spring 2015.

• multiple networked cooperating computers
• Example: Internet E-Mail, Athena file server, Google MapReduce, Dropbox, etc.

• to connect physically separate entities
• to achieve security via physical isolation
• to tolerate faults via replication at separate sites
• to increase performance via parallel CPUs/mem/disk/net

...but:

• complex, hard to debug
• new classes of problems, e.g. partial failure (did he accept my e-mail?)
• Leslie Lamport: "A distributed system is one in which the failure of acomputer you didn't even know existed can render your own computerunusable."
• Advice: don't distribute if a central system will work

• interesting -- hard problems, non-obvious solutions
• active research area -- lots of progress + big unsolved problems
• used by real systems -- unlike 10 years ago -- driven by the rise of big Web sites
• hands-on -- you'll build a real system in the labs

See the course website.

• Lectures about big ideas, papers, labs
• Readings: research papers as case studies
  • please read papers before class
  • paper for today:
  • each paper has a question for you to answer and one for you to ask (see web site)
  • submit question & answer before class, one or two paragraphs
• Mid-term quiz in class, and final exam
• Labs: build increasingly sophisticated fault-tolerant services
  • First lab is due on Monday
• Project: design and build a distributed system of your choice or the system we posein the last month of the course
  • teams of two or three
  • project meetings with course staff
  • demo in last class meeting

Example:

• a shared file system, so users can cooperate, like Dropbox
  • but this lecture isn't about dropbox specifically
  • just an example goal to get feel for distributed system problems
• lots of client computers

• Choice of interfaces
  • Monolithic file server?
  • Block server(s) -> FS logic in clients?
  • Separate naming + file servers?
  • Separate FS + block servers?
• Single machine room or unified wide area system?
  • Wide-area dramatically more difficult.
• Client/server or peer-to-peer?
  • Interact w/ performance, security, fault behavior.

• How do clients/servers communicate?
  • Direct network communication is pretty painful
  • Want to hide network stuff from application logic
• Most systems organize distribution with some structuring framework(s)
  • RPC, RMI, DSM, MapReduce, etc.

• Distribution can hurt: network b/w and latency bottlenecks
  • Lots of tricks, e.g. caching, threaded servers
• Distribution can help: parallelism, pick server near client
  • Idea: scalable design
    • We would like performance to scale linearly with the addition of machines
    • N x servers -> N x total performance
• Need a way to divide the load by N
  • divide the state by N
    • split by user
    • split by file name
    • "sharding" or "partitioning"
• Rarely perfect -> only scales so far
  • Global operations, e.g. search
  • Load imbalance
    • One very active user
    • One very popular file
      • -> one server 100%, added servers mostly idle
      • -> N x servers ->1 x performance

• Dropbox: ~10,000 servers; some fail
• Can I use my files if there's a failure?
  • Some part of network, some set of servers