Design Google Drive
Cloud storage services like Google Drive, Dropbox, Microsoft OneDrive and Apple iCloud let you store documents, photos, videos and other files in the cloud, access them from any device, and share them. This chapter designs one.
- 15.1Requirements and estimationAdd files (drag and drop), download files, sync files across devices, see file revisions, share files, and send notifications when a file…
- 15.2Start simple: one server and three APIsStart with a single server: a web server (Apache) to upload and download files, a database (MySQL) for metadata (user data, login info,…
- 15.3Moving away from one server, and sync conflictsOne day you get an alert: only 10 MB of storage left. First idea: shard the data by user_id across several storage servers. But a storage…
- 15.4High-level designUser: uses the app through a browser or mobile app. Block servers: upload blocks to cloud storage. A file is split into blocks (Dropbox…
- 15.5Block servers and delta syncUploading the whole file on every small change would waste bandwidth. Two optimizations: Delta sync: when a file is modified, only the…
- 15.6High consistency and the metadata databaseThe system needs strong consistency by default: it's unacceptable for one file to look different to different clients at the same time.…
- 15.7Upload flow, download flow and notificationsUpload flow (two requests in parallel) Add file metadata: client 1 sends the new file's metadata; it's stored with status "pending", and…
- 15.8Saving storage space, and handling failuresSave storage space File version history and multiple data center copies make storage explode. Three techniques: De-duplicate data blocks:…
- 15.9Wrap upGoogle Drive combines strong consistency, low network bandwidth and fast sync. The design has two flows: managing file metadata and syncing…