Design Pastebin: storing and sharing text snippets
Pastebin looks like a URL shortener with bigger values. The difference is where you store the content, and how you handle expiry and abuse.
Goal: users paste text, get a link, and anyone with the link can read it. Pastes may expire.
Requirements
- Create a paste (text up to a size limit, such as 1 MB), with optional expiry and visibility.
- Read a paste by link, fast.
- Read-heavy, but with larger payloads than a URL shortener.
Estimates
Assume 1 million new pastes a day (about 12 writes/s) and 5 million reads a day (60 reads/s), average 10 KB. Storage is about 10 GB a day, or 3.6 TB a year. That is more than you want in a primary database.
Design
Split metadata and content:
- Metadata in a database:
id, content_key, created_at, expires_at, visibility, owner_id. - Content in object storage (S3-style), under
content_key.
This keeps the database small and cheap, and object storage handles the bulk.
Generating the ID
As in a URL shortener: random base62 or counter blocks. For private pastes use a long, unguessable ID (for example 12+ random characters) because the link is the only protection.
Read path
- Look up the ID in the cache (Redis). Hit: return.
- Miss: read metadata, check expiry and visibility, fetch the content from object storage, cache it and return it.
- Serve public pastes through a CDN.
Expiry
Check expires_at on every read so expired pastes are never served. Delete the data with a background cleanup job (or object-storage lifecycle rules) so storage does not grow forever.
Security and abuse
- Escape output. Show pastes as plain text or sanitised HTML to prevent XSS. Serve them with a safe content type.
- Limit size and rate per IP or account.
- Scan for secrets and malware. Pastebin-style sites are often used to leak credentials, so offer a takedown path.
- Do not let search engines index private pastes (
noindex).
Possible extras
Syntax highlighting on the client, "burn after reading", password-protected pastes, and edit history.
Takeaway: separate small metadata from large content, keep content in object storage, enforce expiry on read plus cleanup, and treat all pasted text as hostile input.
System Design series, Part 3 (case studies): 2 of 16. Previous: Design a URL shortener: from requirements to scale.
