This course covers the architecture of modern data storage and processing systems, including relational databases, cluster computing systems, streaming and machine learning systems. Topics include database system architecture, storage, query optimization, transaction management, fault recovery, and parallel processing, with a focus on the key design ideas shared across many types of data-intensive systems.
Location: Tuesday/Thursday 3:15 PM in Skillaud or on Zoom. We're on Zoom only for the first two weeks; join our meeting from the Zoom tab in the Canvas sidebar.
Announcements and Questions
We're using Ed Discussion for all course communications. You can access it from the sidebar in Canvas.
Instructor: Matei Zaharia (Office hours: by appointment, please email firstname.lastname@example.org).
Course Assistants and Office Hours:
Currently, all office hours are online over zoom; as the quarter progresses, we may change some office hours to be in person, depending on the situation.
To join these zoom links, you must be logged into zoom with your Stanford email. We will be using QueueStatus at the link here: https://queuestatus.com/queues/1786. When entering the queue, please write a bit about the problem you're having and what you've done so far to address it.
The schedule is as follows.:
Office Hours: Monday 4-5:30
Office Hours: Tuesday 5-6:30
Office Hours: Friday 4:30-6,
Office Hours: Saturday 1-2:30
Silvia Gong (Office Hours: Monday 4-5:30, email: email@example.com)
Deepti Raghavan (Office Hours: Tuesday 5-6:30, email: firstname.lastname@example.org)
Tina Li (Office Hours: Friday 4:30-6, email: email@example.com)
Sanjari Srivastava (Office Hours: Saturday 1-2:30, email: firstname.lastname@example.org)
Reading: A History and Evaluation of System R (You can skip the sections on pages 643-644 called "The Recovery Subsystem", "The Locking Subsystem", "The Convoy Phenomeon" and "Additional Observations". Question: Why did the storage system change from inversions to B-trees?)
Database System Architecture 2 & Storage
|1/13||Storage Formats and Indexing|
Storage Formats and Indexing 2
Reading: Integrating Compression and Execution in Column-Oriented Database Systems (Question: How might the conclusions of the paper change if it ran on an NVMe SSD?)
Reading: Spark SQL: Relational Data Processing in Spark (Question: Can you think of any limitations to the way Catalyst supports external extensions?)
Guest Talk: Tianqi Chen on Abstractions for Machine Learning Compilation
Optional Reading: TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
|2/8||Failure Recovery 2|
Take-Home Midterm Posted (solutions)
You can do the midterm in any 2-hour window until midnight on 2/15.
Past midterms for reference: Winter 2021 (solutions), Winter 2020 (solutions), Spring 2019 (solutions)
No class, Take-Home Midterm Due (at midnight)
Reading: Granularity of Locks and Degrees of Consistency in a Shared Data Base (Read up to page 372. Question: Draw the data structures in a DBMS that holds a table sorted by primary key, with multiple pages. What locks have to be acquired to change one record's primary key?)
Concurrency 3 and Distributed Databases
Optional Reading: Time, Clocks and the Ordering of Events in a Distributed System
Optional Reading: Lessons from Internet Services: ACID vs. BASE
Optional Readings: Amazon Aurora, Dynamo, Delta Lake
Guest Talk: Rebecca Taft on CockroachDB: The Resilient Geo-Distributed SQL Database
Optional Reading: CockroachDB: The Resilient Geo-Distributed SQL Database
Optional Reading: The CQL Continous Query Language, The Dataflow Model
Reading: Privacy Integrated Queries (Question: Describe a computation on data for which it's hard to provide any differential privacy.)
Assignment 3 Due (at 11:59 pm)
Past finals for reference: Winter 2021 (solutions), Winter 2020 (solutions), Spring 2019 (solutions)
Students should ideally have taken CS 145 and CS 161. If you haven't taken CS 145, the main thing we assume from it is knowledge of basic SQL syntax; you can also read a SQL tutorial to learn about this.
Lectures and Video Recordings
Lectures for the class will be given live on Zoom and recorded. You can find our Zoom link and the lecture recordings on Canvas. Please note that these recordings might be reused in other Stanford courses, viewed by other Stanford students, faculty, or staff, or used for other education and research purposes. If you have questions about video recording, please contact a member of the teaching team.
Assignments and Exams
We will have three programming assignments and two exams. The programming assignments are designed to be runnable on your personal machine and should be submitted through Gradescope.
The exams are open-book, meaning that you can use your course notes, slides, books, or online resources, except that communication is not be allowed during them (e.g., you can't ask a question on Stack Overflow or contact another student). Tests will cover material in the lectures, readings and assignments.
We have occasional readings for the lectures. We expect students to complete these and think about the questions we list for each paper on their own (you do not need to turn in answers). Our exams will cover content in the readings.
Database Systems: The Complete Book (2nd Edition), by Garcia-Molina, Ullman and Widom, covers a lot of the technical material in the course and may be helpful as a study guide. We focus on chapters 13-20. We will also cover the material in lectures, but this book is a good source of additional information.
- Assignments: 16% each (total: 48%)
- Midterm: 22%
- Final: 30%
Students each have up to 2 late days that they may use for assignments. Assignments submitted after these late days have been used up will incur a penalty of 10% per extra day late. In addition, we will not accept submissions after March 19th at 11:59 PM Pacific to give the staff enough time for grading.
The syllabus page shows a table-oriented view of the course schedule, and the basics of course grading. You can add any other comments, notes, or thoughts you have about the course structure, course policies or anything else.
To add some comments, click the "Edit" link at the top.