Welcome to the graduate course on Topics in Modern Distributed Systems

Distributed Systems enable large-scale modern applications ranging from internet-scale applications like Facebook, and Google to web-based services like AWS, Azure, Amazon Lambda to distributed machine learning like Federated Learning. This course will cover various topics of a distributed system, storage, and networking; including but not limited to cloud object stores, serverless frameworks, AI/deep learning, scheduling internet-scale applications, containers, resource management, key value stores, and so on. We will discuss research papers from top conferences to fuel brainstorming sessions to generate cool and novel ideas. Students will also get the opportunity to build real-world large-scale distributed systems to realize the new ideas discussed in class.


Lecture Info


Have taken undergraduate-level Operating System or a similar course. This course assumes familiarity with systems, including how operating systems work and how networks work. If you don’t have this background, you should be in communication with me (the instructor) at the beginning of the semester.


Who is this course for?

This course is primarily intended for graduate students and motivated seniors who want to learn the latest research advances in distributed systems and cloud computing areas, and are interested in building distributed systems used/demanded by existing/emerging data-intensive applications.


Grading policy

Your grade will be calculated as follows:


Grading rules

The final grade is computed according to the following rules:


Late Days

Each student will be granted 5 late days to use for the homework. After all free late days are used up, the penalty is 10% for each additional late day. Late days can not be used for the mid-way and final research presentation.


Student Integrity

Cheating in this course will result in a grade of F for the course and the University policies will be followed.


Students with Disabilities

If you need any accommodation, you are highly encouraged to contact both your instructor and Disability Resources Center (DRC).


Office hours

The course schedule is tentative and subject to change.


Date Topic Reading Assignment
Week 1: Sep 6 Lec 1: Introduction and Logistics

Optional Reading:

Assignment 1: student info & signing up for presentations 
Week 1: Sep 8 Lec 2: Distributed Machine Learning

Optional Reading:

Please sign up for the paper presentations. 
Week 2: Sep 13 Lec 3: Federated Learning Systems

Optional Reading:

Week 2: Sep 15 Lec 4: Distributed Storage Systems

Optional Reading:

Homework 1 is published and available here. Due date is October 10.
Week 3: Sep 20 Lec 5: Cloud Computing Proposal and literature review for the final project: Due date is Sept 27 now.
Week 3: Sep 22 Lec 6: Distributed Machine Learning Frameworks

Optional Reading:

Week 4: Sep 27 Lec 7: Accelerating Distributed Machine Learning

Optional Reading:


Project Proposal Due Today.

Week 4: Sep 29 Lec 8: Internet-Scale Storage Systems
Week 5: Oct 4 Lec 9: ML Storage Systems
Week 5: Oct 6 Lec 10: Containers Vs Virtual Machines Homework 1 due on Oct 10.
Week 6: Oct 11 Lec 11: Serverless Functions Workload
Week 6: Oct 13 Lec 12: Serving Serverless Computing
Week 7: Oct 18 Lec 13: Serverless Computing Frameworks
Week 7: Oct 20 Mid-way Project Presentation 1) Data Heterogeneity in Federated Learning [Slides]
2) Incentivized Federated  Learning [Slides]
3) Resource Aware Federated Learning for Heterogeneous Edge Computing [Slides]
Week 8: Oct 25 Mid-way Project Presentation
Week 8: Oct 27 Mid-way Project Presentation
Week 9: Nov 1 Lec 14: Caching Systems Homework 2 is published. Due date Nov 15.
Week 9: Nov 3 Lec 15: Large Scale Caching
Week 10: Nov 8  Lec 16: Accelerating Federated Learning
Week 10: Nov 10 Lec 17: Asynchronous and Privacy-preserving Federated Learning
Week 11: Nov 15 Lec 18: Incentivization in Federated Learning Homework 2 Due Today.
Week 11: Nov 17 Lec 19: Benchmarking Distributed Systems Homework 2 due on Nov 20. 5 days extension.
Week 12: Nov 22 Lec 20: Guest Lecture (Zoom) + Hack Day for Final Project


Week 12: Nov 24 No Class
Week 13: Nov 29 Lec 21: Online Gaming
Week 13: Dec 1 Lec 22: Cluster Management
Week 14: Dec  6 Final Project Presentation
Week 14: Dec 8 Final Project Presentation
Week 15: Dec 13 Final Project Report Discussion


The research project can be done in a group of 2 students. The project will be judged on the following criteria:

Project Proposal Report

Recommended structure for the Proposal Report is:

Mid-way Project Presentation

You should present the current progress of your final project including design, related work review, any results if you have some (to show motivation or evaluation).


Final Project Presentation

You should present the final project including design, related work, and final results to showcase your project.


Project Final Report

The final deliverable of this course is a conference-quality paper. Focus on the following Questions 0 through 5 (an extended version of Question 0 through 3 in the Checkpoint Report requirement):

As well as pay attention to:

It is common that the implementation or experimental methodology gets adapted in the course of your investigation. And therefore, given that this is your final write-up, please make sure to refine/change/expand based on your mid-way presentation feedback to reflect your latest implementation & evaluation activities.