Object storage (S3-style): how to store files at scale
Images, videos and backups belong in object storage, not in a database. Learn how it works and the patterns that make uploads safe and cheap.
Object storage (Amazon S3, Google Cloud Storage, Cloudflare R2, MinIO) stores files as objects in buckets. Each object has a unique key (like photos/2026/10/a.jpg), the data itself and some metadata. There is no real folder tree. The slashes in a key are just part of the name.
Why not a database
Large files make a database slow and expensive: bigger backups, slower queries, memory pressure. Object storage is built for large, rarely changed blobs and is far cheaper per gigabyte. Keep the file in object storage and store only its key or URL in the database.
Main properties
- Very high durability. S3 Standard is designed for 99.999999999% (eleven nines) durability by storing data redundantly across locations.
- Virtually unlimited size.
- Objects are immutable. You replace a whole object rather than editing a part of it.
- Access over HTTP with an API, so it works well with a CDN in front.
- Storage classes trade cost for access speed (hot, infrequent, archive).
Upload patterns
Presigned URL. Your server creates a short-lived signed URL, and the client uploads directly to the storage. Your servers never carry the file, which saves bandwidth and CPU. Limit the size, content type and expiry.
Multipart upload. For big files, split into parts and upload in parallel. A failed part can be retried without starting over.
Serving files
Put a CDN in front for public content. For private files, use signed URLs with an expiry. Never make a bucket public by accident, because open buckets are a classic source of data leaks.
Other practices
- Lifecycle rules move or delete old objects automatically.
- Versioning protects against accidental overwrite or delete.
- Process uploads (resize images, scan for malware) asynchronously with a queue.
- Never trust the file name or type sent by the user.
Takeaway: store files in object storage, keep only keys in your database, upload with presigned URLs, and lock down bucket access.
System Design series, Part 2 (building blocks): topic 5 of 14. Previous: Elasticsearch and full-text search: how the inverted index works.
