Pastebin / Text Sharing Service
Problem statement
Design a service where users paste text (up to a few MB), get a short URL, and others can read it; pastes may expire, be private, or be syntax-highlighted.
Requirements
Functional
Create a paste with optional title, expiry, visibility (public/unlisted/private) and syntax.
Read a paste by its short id; raw and rendered views.
Delete by owner; list own pastes.
Optional custom id.
Non-functional
Highly available reads; p99 read < 100 ms.
Durable: a paste is never lost or corrupted before expiry.
Read-heavy (10:1); 1M new pastes/day.
Cost-efficient storage for large text.
Back-of-the-envelope estimates
- 1M pastes/day × 10 KB avg = 10 GB/day; 5 years ≈ 18 TB → content in object storage, metadata in a DB.
- Reads 10M/day ≈ 115/s; trivial; cache hot pastes.
- Ids: base62, 8 chars → 218 trillion.
- Metadata row ~200 B → 200 MB/day → 365 GB over 5 years; fine for a sharded relational DB or KV.
API
POST /v1/pastes { content, title?, syntax?, expiresIn?, visibility, customId? } → { id, url }
GET /v1/pastes/{id} → metadata + content (or 404/410 if expired)
GET /{id}/raw
DELETE /v1/pastes/{id} (owner)
GET /v1/users/me/pastes
Data model
pastes (paste_id PK, user_id, title, syntax, visibility, content_key, size, created_at, expires_at, views)
Object storage: pastes/{paste_id}.txt (or inline in the DB for < 4 KB)
users (user_id, ...)
High-level design
- API servers behind a load balancer.
- Create: get an id from a key-generation service (pre-generated unique keys, as in the URL shortener); write content to object storage; write metadata to the DB; return the URL.
- Read: check cache (Redis, id → content + metadata) → DB for metadata → object storage for content; cache the result. Enforce visibility and expiry.
- Expiry:
expires_atchecked on read (return 410) plus a cleanup job that deletes expired content from object storage and rows from the DB. - Small pastes can be stored inline in the DB row to save an object-storage round-trip; large ones in object storage; a CDN can serve public raw pastes.
- Abuse: size limits, rate limits, malware/spam scanning for public pastes.
Architecture diagram
Deep dives
Separate metadata from content: metadata is small and queried; content is large and only fetched by id. Object storage gives cheap durability; the DB stays small.
Private pastes: require auth; never CDN-cache; unlisted = accessible by id but not listed.
Consistency: write content before metadata; a metadata row without content is impossible, and orphan content is cleaned up by the job.
Bottlenecks & mitigations
Viral paste → cache + CDN; object storage handles the rest.
Cleanup job scanning → index on
expires_at; delete in batches.Large pastes (MBs) → streaming upload/download; size caps.
Failure modes
- Object storage write succeeds, DB write fails: client retries; orphan object reclaimed by cleanup.
- Cache down: DB + object storage serve directly.
- KGS unavailable: servers hold a local batch of keys; fall back to random ids with a uniqueness check.
Follow-up questions interviewers ask
How would you add versioning/edits to pastes?
How would you support burn-after-read pastes safely?
How would you implement full-text search over public pastes?