The Feed Problem
A social network feed shows a user the recent posts from people they follow, ranked by relevance or recency. The challenge: at scale (Twitter/X: 300M users, Instagram: 2B users), computing each user’s feed from scratch on every request is too slow. The solution involves pre-computing and caching feeds. Two fundamental approaches: fan-out on write (push) and fan-out on read (pull).
Fan-out on Write (Push Model)
When a user creates a post: immediately write it to every follower’s feed cache. Each follower has a pre-materialized feed (a list of post IDs) stored in Redis. On feed read: just fetch from the user’s pre-computed feed. Reads are O(1). Pros: fast reads, works well for most users. Cons: when a celebrity with 100M followers posts: 100M write operations (fan-out). This write spike can overwhelm the system. Used by: early Twitter architecture.
Mitigation: use a queue (Kafka). The post create event goes to Kafka. Worker consumers read from Kafka and fan-out in parallel. Still 100M writes but spread across workers over seconds. Apply a cap: only fan-out to followers who were active in the last 7 days — inactive users can pull on demand.
Fan-out on Read (Pull Model)
No pre-computation on write. On feed read: query all the accounts the user follows, fetch their recent posts, merge and sort. Reads are slow (query N accounts, merge N result sets). Pros: no write amplification — posting by celebrities is cheap. Cons: reads are slow and expensive at scale. Works for: users who follow many people (reading requires merging many result sets); better for celebrity accounts.
Hybrid Approach (Industry Standard)
Combine both: push for regular users ( 10K followers). When reading the feed: retrieve the pre-computed feed from cache (push model). For celebrity posts: fetch their recent posts separately and merge into the feed. This is how Twitter and Instagram actually work. The threshold (10K followers) is configurable and trades write cost vs read complexity.
Feed Ranking
Chronological: simple, but users miss important posts from less-active accounts they follow. Ranked: ML model scores each candidate post by predicted engagement (likes, comments, shares, view time). Features: post age, poster relationship strength (how often you interact), content type (video, photo, text), past engagement with similar content. Two-stage ranking: Stage 1 — retrieve candidate posts (recent posts from followed accounts + sponsored content, ~1000 candidates). Stage 2 — ML ranking model scores all candidates and returns top-50. Heavy ML ranking on 1000 candidates per request at scale requires GPU inference servers or optimized CPU inference.
Pagination and Cursor-Based Feeds
Never use OFFSET for feed pagination — OFFSET scans and discards N rows before returning results. At large offsets: very slow. Use cursor-based pagination: the client sends the ID of the last seen post. Server returns posts older than that ID. Cursor = post_id or (timestamp, post_id) tuple for stability. New posts inserted at the top never shift cursor positions, preventing duplicates or gaps. Feed freshness: when the user pulls to refresh: show new posts above the cursor (new content), not a reset of the entire feed. Show “You have 12 new posts” as a banner before revealing them.
Interview Tips
- Push vs pull is the core trade-off. The interviewer expects you to discuss both, explain the celebrity problem, and propose the hybrid solution.
- Feed storage: Redis Sorted Set (ZADD with timestamp as score, post_id as member) is the standard implementation. ZREVRANGE for most recent posts. ZREMRANGEBYSCORE to trim old entries (keep last 1000 posts per feed).
- The feed is eventually consistent — a new post may take seconds to appear in all followers’ feeds. This is acceptable for social media.
Asked at: Meta Interview Guide
Asked at: Twitter/X Interview Guide
Asked at: Snap Interview Guide
Asked at: LinkedIn Interview Guide
Asked at: Netflix Interview Guide