Video Streaming Platform (YouTube / Netflix)
Problem statement
Design a platform where users upload videos that are processed into multiple resolutions and streamed to viewers worldwide with adaptive bitrate, plus view counts and recommendations at a basic level.
Requirements
Functional
Upload videos up to several GB with resumable uploads.
Transcode into multiple resolutions/codecs; generate thumbnails.
Stream with adaptive bitrate (HLS/DASH) on any device.
Metadata: title, description, tags; search; view counts, likes, comments.
Non-functional
Playback start < 2 s; no rebuffering on stable connections.
Uploads durable — never lose a user's video.
Global scale: 1B+ hours watched/day, 500 h uploaded/minute.
Cost-efficient storage and bandwidth (CDN offload > 95%).
Back-of-the-envelope estimates
- Uploads: 500 h/min = 30k h/day. Raw ~1 GB/h → 30 TB/day raw; transcoded ladder (5 renditions) ≈ 3× → ~100 TB/day of stored video → ~36 PB/year.
- Views: 1B h/day at ~3 Mbps avg ≈ 1B × 3600 × 3 Mb ≈ 1.35 EB/day egress; that is why CDN and codec efficiency (VP9/AV1) matter more than any single server.
- Transcoding: ~1 CPU-hour per hour of video per rendition → 30k × 5 = 150k CPU-hours/day → ~6k always-on cores or GPU encoders.
API
POST /v1/videos/upload/init { filename, size, mime } → { uploadId, presignedParts[] }
PUT <presigned part url> (client uploads chunks directly to object storage)
POST /v1/videos/upload/complete { uploadId, etags[] } → { videoId, status:"processing" }
GET /v1/videos/{id} → metadata + { manifestUrl: "https://cdn/.../master.m3u8" }
GET /v1/videos/{id}/status
POST /v1/videos/{id}/view (beacon; batched client-side)
Data model
videos (video_id, owner_id, title, description, status, duration, created_at)
renditions (video_id, resolution, codec, bitrate, manifest_path, size)
thumbnails (video_id, path)
view_events → Kafka → aggregated daily/hourly counts in an OLAP store
comments (video_id, comment_id, user_id, text, ts) -- separate service
Object storage: raw/{videoId}/source.mp4, hls/{videoId}/{rendition}/segment_N.ts + playlists
High-level design
- Upload: client gets presigned multipart URLs and uploads chunks straight to object storage (S3); resumable by part. Upload service records completion and enqueues a processing job.
- Processing pipeline (DAG): split source into GOP-aligned chunks → transcode each chunk to each rendition in parallel (thousands of workers) → stitch → package to HLS/DASH segments (2–6 s) → thumbnails, captions, content moderation → write manifests → mark video
ready. - Delivery: manifests and segments served through a multi-CDN; origin is object storage with an origin shield. Adaptive bitrate is client-driven: the player measures throughput and picks the rendition per segment.
- Metadata service backed by a relational DB with read replicas; heavy caching.
- View counting: players send beacons; Kafka → stream processor → approximate counts (HyperLogLog for unique viewers) → displayed counts updated every few seconds.
- Search & recommendations: metadata indexed in Elasticsearch; recs from a separate ML pipeline.
Architecture diagram
Deep dives
Chunked parallel transcoding turns a 2-hour job into minutes; chunk on keyframe boundaries so chunks encode independently.
Adaptive bitrate: server produces the ladder (e.g. 240p→1080p/4K); the client's ABR algorithm (buffer-based or throughput-based) switches per segment. Short segments = faster start and adaptation; longer = better compression.
CDN strategy: popular content (top 1%) is pre-warmed to edge; long-tail served from origin shield. Cache key = segment path; immutable, long TTL.
Cost: cold-tier storage for videos with no views in 90 days; re-encode to a newer codec on demand.
Bottlenecks & mitigations
Origin egress during a viral launch → pre-warm CDN, origin shield, request coalescing.
Transcoding queue backlog → autoscale workers; prioritise by creator tier and expected views; produce 360p first so playback can start while higher renditions finish.
Metadata DB reads → cache aggressively; view counts served from the counter store, not the DB.
Thumbnails at scroll speed → pre-generate sprite sheets.
Failure modes
- Transcode worker failure: chunks are idempotent tasks; retry with a visibility timeout; a job with a persistently failing chunk is flagged for manual review.
- CDN outage: multi-CDN with DNS/manifest-level steering; player retries the alternate CDN URL.
- Object storage regional outage: cross-region replication for manifests and hot renditions; raw uploads replicated asynchronously.
- Counter pipeline lag: display counts marked approximate; no impact on playback.
Follow-up questions interviewers ask
How would you support live streaming (low-latency HLS)?
How would you enforce DRM?
How does the design change for short-form vertical video (Reels)?
Further reading
- Guide: Video streaming platformIllustrated guide on this site
- YouTube architecture (High Scalability)Articlehighscalability.com
- Scalability at YouTube (2012 talk)Videoyoutube.com
- Transcoding: how we serve videos at scale (Egnyte)Articleegnyte.com
- Under the hood: broadcasting live video to millions (Facebook)Articleengineering.fb.com
- High quality video encoding at scale (Netflix)Articlenetflixtechblog.com
- Optimized shot-based encodes (Netflix)Articlenetflixtechblog.com