Post / Comment Ranking System

Design a ranking system for posts (subreddit / home feed) or comments (within a post). Covers candidate generation, feature retrieval, online inference, A/B testing, and the cold-start path. The interviewer treats the model itself as a black box and focuses on the surrounding ML infrastructure. Scope Design the ranking system for comments within a single post's discussion tree: when a user opens a post with a large thread, they should quickly see the most relevant top-level comments and the most relevant replies under each branch. The system continuously ingests comment creations, vote updates, and moderation actions while serving low-latency paginated ranked reads for hot threads. Keep the scope tight. This is ranking comments inside one discussion tree — not home-feed ranking, cross-post recommendation, or full-text comment search. If the interviewer opens with a broad "rank Reddit" prompt, narrow it to the comment tree explicitly; that scoping move is itself a signal. The central insight: comment ranking is a tree-ranking problem, not one giant global sorted list. Maintain ordered child lists keyed by (post_id, parent_comment_id, sort_mode) and update them incrementally from vote and comment signals. For the first pass, assume ranking is mostly deterministic and shared across users; personalization, language-quality models, and abuse ML are secondary signals added later. Requirements Functional Fetch ranked top-level comments for a post with pagination and a selected sort mode (best, top, new, controversial). Expand a comment and fetch its ranked child replies without loading the entire thread. Create comments and vote on comments; ranking reflects those changes within a few seconds. Moderators can remove, lock, or collapse comments, reflected in ranked reads quickly. Preserve ordering stable enough that the thread does not thrash on every single vote. Keyword search inside comments, full personalization, spam classification, and cross-post ranking are reasonable follow-ups — keep them out of the first pass unless asked. Non-functional Requirement Target Rationale Initial thread read latency P95 < 150 ms for first page Opening a hot thread should feel fast Reply expansion latency P95 < 100 ms Expanding a subtree should feel instant Ranking freshness Most vote/reply changes reflected within 1–3 s Discussion UX without per-vote synchronous rebuilds Availability 99.95% Comment threads are core surface area Durability No lost accepted comments or votes Ranking inputs must survive failures once accepted Scalability Millions of posts, bursty hot threads, vote-heavy Reads and vote writes are both large-scale Ordering quality Relevant comments near top without excessive jitter Users must trust the ranking Clarifying questions to raise early One default ranking mode or multiple sort modes? → Assume multiple, with best as primary. Fully personalized per-user ranking? → Assume no for the first pass. How fast must votes affect order? → Near-realtime (few seconds), not strictly synchronous per vote. Rank the whole tree globally? → No. Rank siblings under the same parent separately. Deleted/removed comments disappear immediately? → Yes for normal users; moderators may still access them. Capacity (rough, to justify the architecture) Assumptions: - DAU: 70M - Post detail views that load comments/day: 250M - New comments/day: 120M - Comment vote events/day: 1.5B - Avg comments returned on initial load: 40 summaries - Peak factor over average: 10x Reads: - 250M thread loads/day ~= 2,900 reads/sec avg; peak ~= 30,000/sec - Reply expansion + pagination roughly double => 60,000+ read ops/sec peak Writes: - New comments: 120M/day ~= 1,400 writes/sec avg - Comment votes: 1.5B/day ~= 17,400 writes/sec avg; peak ~= 170,000+/sec Storage: - Comment metadata row ~= 500 bytes - 120M new comments/day ~= 60 GB/day raw before indexes/replication - Rank materialization only keeps hot ordered slices in cache; full history in durable stores The hard part is not storing comments. It is absorbing large vote volume while serving cheap ranked reads for hot posts without recomputing entire trees on every page view. Data Model Keep the initial model focused on the objects needed to rank siblings within a tree. Separate append-friendly inputs (comments, votes, moderation actions) from mutable ranking projections (CommentAggregate, CommentRankEntry). Post(post_id, subreddit_id, author_id, title, created_at, status[active|locked|deleted]) Comment(comment_id, post_id, parent_comment_id NULL, author_id, body, depth, status[active|removed|deleted|collapsed], created_at, edited_at NULL) CommentAggregate(comment_id, post_id, parent_comment_id NULL, upvote_count, downvote_count, reply_count, score, controversy_score, best_rank_score, last_activity_at, version) Vote(vote_id, user_id, comment_id, direction[-1|0|+1], created_at, updated_at) CommentRankEntry(post_id, parent_comment_id NULL, sort_mode[best|top|new|controversial], comment_id, rank_key, aggregate_version, updated_at) ModerationAction(action_id, comment_id, moderator_id, action_type[remove|approve|lock|collapse], payload, created_at) Relationships: Post 1:N Comment; Comment 1:N Comment (replies); Comment 1:1 CommentAggregate; Comment 1:N Vote; Comment 1:N ModerationAction; (post_id, parent_comment_id, sort_mode) 1:N CommentRankEntry. parent_comment_id = NULL represents the top-level comment list for a post. Every other parent gets its own ordered child list. Treat Vote as one current vote state per (user_id, comment_id) (unique constraint). If a full audit trail is needed, keep it as a separate append-only vote-event log. Ranking signals The exact formula is product-tunable; the architecture should support versioned, pluggable scoring: top_score = upvote_count - downvote_count new_score = created_at controversial_score = f(total_votes, vote_balance) best_rank_score = WilsonLowerBound(upvotes, downvotes) + alpha * log(1 + reply_count) - beta * age_decay Do not burn the interview deriving the perfect scoring formula. What matters more is how you store, recompute, and serve ranking efficiently. API Design REST for comment reads, reply expansion, creation, voting, moderation. Durable event bus/queue for asynchronously recomputing aggregates and ranked projections. Optional SSE/WebSocket only if the interviewer explicitly wants live thread updates while viewing. GET /api/posts/{post_id}/comments?sort=best&cursor=abc&limit=20 GET /api/posts/{post_id}/comments/{comment_id}/replies?sort=best&cursor=xyz&limit=10 POST /api/posts/{post_id}/comments { parent_comment_id, body, idempotency_key } POST /api/comments/{comment_id}/vote { direction, idempotency_key } POST /api/comments/{comment_id}/moderation { action } Example thread-read response: { "post_id": "post_123", "sort": "best", "comments": [ { "comment_id": "c1", "parent_comment_id": null, "body": "...", "score": 9821, "reply_count": 143, "status": "active", "created_at": "..." } ], "next_cursor": "cursor_2" } Use keyset cursors on (rank_key, comment_id), not offset pagination. In a ranked thread, vote changes reorder comments between requests, so offsets cause duplicates and skips. Put idempotency keys on both comment creation and vote writes. Flaky-mobile retries and duplicate submits will corrupt counters or create duplicate comments unless the write path dedupes. Internal ranking event contract (produced via transactional outbox): { "event_type": "comment_vote_changed", "post_id": "post_123", "comment_id": "c1", "parent_comment_id": null, "voter_id": "user_7", "old_direction": 0, "new_direction": 1, "request_id": "vote:user_7:c1:...", "created_at": "..." } High-Level Design Start simple, then evolve Do not open with Kafka + Redis + multiple projection stores. Start with: one comment service, one primary relational DB for comments and votes, one cache for the hottest ranked lists, one background worker that recomputes rank for changed sibling groups. That covers moderate traffic. Evolve to the scaled architecture only once the interviewer pushes on hot-post traffic and vote volume. Core idea — three concerns Durable source-of-truth writes for comments, votes, moderation. Asynchronous aggregate computation for per-comment counters and ranking signals. Materialized ordered read models keyed by (post_id, parent_comment_id, sort_mode). This absorbs heavy vote traffic without forcing every read to sort raw comments from scratch. Read path Client requests GET /comments?sort=best. Read API fetches ordered comment ids for (post_id, root, best) from the Rank Cache. Read API hydrates comment summaries from the Comment Cache. On miss: fall back to the durable Rank Projection Store (rank list) / Comment Store (summaries). Response returns only the first page + reply previews or reply counts. Reply expansion fetches the ordered child list for one (post_id, parent_comment_id, sort_mode) and returns the next reply page + cursor. Never load and sort the full discussion tree per request — rank and paginate one sibling list at a time. Write path Comment creation → store Comment row in OLTP with idempotency key → same transaction writes an outbox event → ranking pipeline increments parent reply_count, seeds aggregates for the new comment, inserts it into the relevant ordered lists. Vote update → upsert the user's vote state in OLTP → outbox event → Signal Aggregator updates CommentAggregate → Score Calculator recomputes affected sort keys (top, best, controversial) → Rank Materializer updates the durable projection and hot ordered sets in Rank Cache. Moderation action → persist action + emit event → Rank Materializer removes/downgrades the comment in public ordered lists immediately. Ordered read model layout Hot cache (Redis sorted sets): rank:{post_id}:root:best -> sorted comment ids rank:{post_id}:root:top -> sorted comment ids rank:{post_id}:{parent_comment_id}:best -> sorted child comment ids comment:{comment_id} -> hydrated summary payload Durable projection (DB/KV): rank_projection(post_id, parent_comment_id, sort_mode, comment_id, rank_key, updated_at) comment_summary(comment_id, body_preview, score, reply_count, status, updated_at) Read from the durable layer with keyset pagination on (rank_key, comment_id) rather than rewriting explicit rank numbers for whole sibling lists after every update. Implementation options: Redis sorted sets for hot threads; Cassandra/DynamoDB projection tables for larger durable views. Clean default answer: Redis sorted sets backed by durable projections. Avoid resorting everything The naive design — every vote recomputes scores for the full post and re-sorts all comments — does not scale. Instead: update the aggregate for the affected comment, recompute rank keys only for that comment, and update its position inside a few ordered sibling lists. A vote on one reply usually affects only that comment's local sibling list. reply_count changes on reply creation/deletion, not on votes — though ancestor summaries may need small updates if the formula uses subtree-activity signals like last_activity_at. A vote on a deep reply must not rebuild the entire post's top-level order. Hot-post handling Hot posts create two pressures: many concurrent reads on the same top-level lists, and many vote updates on a small set of visible comments. Protect the system by: caching the first several pages of top-level comments aggressively; batching vote deltas for a short window (~100–500 ms) before rank materialization; rate-limiting abusive vote patterns; isolating hot-post partitions so one viral thread does not starve the fleet. Short vote batching is a deliberate trade-off: dramatically lower churn / re-sort cost, at the price of slightly stale ordering for a fraction of a second — usually acceptable for comments. Notes Deep dive — why this is a tree problem Flattening all comments into one global ranked list breaks reply semantics and makes expansion awkward. Better: one ordered list for top-level comments, one ordered list per parent's children, with pagination and ranking operating within sibling groups. This aligns with the UI and keeps updates local. Deep dive — materialize-on-write vs compute-on-read Compute-on-read (scan comments, score, sort, paginate) is simpler at tiny scale but too expensive for hot posts. Materialize-on-write (update cached/durable ordered lists when signals change) is more operationally complex but makes read latency cheap and predictable. For Reddit-scale threads, materialized ranking views are the better answer. Deep dive — freshness vs stability Re-ranking on every vote instantly makes visible comments jitter constantly, hurting readability. Balance: aggregate vote changes over short windows, use formulas with inertia (Wilson score + freshness decay), and update reads within a few seconds instead of after every write — so the thread feels both alive and stable. Deep dive — multiple sort modes Supporting best/top/new/controversial multiplies projection work. Options: materialize all major modes for hot posts; materialize only best and derive less-used modes lazily for cold posts. Precomputing all modes gives faster reads; lazy computation reduces write amplification and cache footprint. Reasonable answer: precompute common modes for hot content, fall back to cold storage for rarely-used ones. Deep dive — abuse, brigading, moderation Ranking quality depends on trust in vote inputs. Protect with per-user vote rate limits, anti-brigading heuristics, delayed/damped trust weighting for suspicious accounts, and moderator remove/lock actions applied ahead of public ranking. You need not fully design anti-abuse ML — it is enough to state that suspicious votes can be filtered or down-weighted before they become ranking inputs. Multi-region and recovery Keep comment and vote writes durable in their home region. Run ranking workers in multiple regions but avoid cross-region synchronous ranking updates in the hot path; asynchronously replicate projections for read locality if needed. Recovery: replay from the event bus if Rank Materializer falls behind; rebuild Rank Cache from durable projections if Redis is lost; use outbox replay if an event failed to publish after the OLTP commit. Durability comes from accepted writes + outbox + replayable event transport; the rank cache is a projection, not the source of truth. Common pitfalls Sorting the full thread on every page view — turns hot posts into repeated expensive scans. Treating the whole post as one flat ranking list — threads are trees; keep rank updates local to sibling groups. Updating rank synchronously on every vote write — unnecessary latency and ordering thrash. No idempotency on vote updates — mobile retries and duplicate submits corrupt aggregates. Spending the whole interview debating the exact scoring formula — the ingest/materialize/serve architecture matters more. Preparation Rehearse the three-concern separation out loud: durable writes → async aggregate computation → materialized ordered read models per (post_id, parent_comment_id, sort_mode). Being able to walk this end-to-end quickly reads as experience; a slow, meandering diagram reads as the opposite. Have a crisp answer for "how do you avoid re-sorting the whole thread on every vote" — localize updates to the affected comment's sibling list. This is the crux the interviewer drives at. Memorize the read/write/vote capacity math (peak vote writes ~170K/sec, peak reads ~60K/sec) so the case for materialize-on-write is grounded in numbers, not assertion. Prepare the freshness-vs-stability trade-off (short vote batching, Wilson + decay inertia) and the hot-post protections as ready deep dives — these separate a solid answer from a shallow one. Strongest framing to land: Aspect Recommendation Rationale Core abstraction Ordered sibling lists per parent comment Matches the tree UI and localizes updates Write durability OLTP + transactional outbox Prevents lost comments or vote signals Read serving Rank cache + comment summary cache Low-latency loads and reply expansion Ranking pipeline Aggregate signals async, then materialize order Handles vote-heavy workloads efficiently Hot-thread strategy Cache top pages, batch vote deltas briefly Reduces churn, protects read latency Correctness model Idempotent writes + replayable projections Safe recovery from retries and cache loss Product quality Stable best ranking, moderation-aware updates Relevant comments without excessive jitter