User Behavior / Metrics Monitoring Aggregator

Design an event monitoring and enrichment platform for product behavior. Collect browser/mobile events, power low-latency dashboards, keep raw events for offline analytics, and enrich data with geo or compliance metadata. Requirements Track user behavior across web and mobile applications, such as ad impressions, clicks, app installs, and other product events. Provide SDKs and API endpoints for data collection. Support real-time dashboard queries with low latency. Support complex asynchronous warehouse queries for deeper analytics. Keep raw events for offline jobs. Enrich events with data such as reverse geolocation or compliance tags. Discuss schemas, ingestion, aggregation, enrichment flow, dashboard storage, and efficient query tables. Notes Scale target to anchor the design: handle over 100M events/day with sub-second query latency on the dashboard path, and treat aggregate counts as requiring full accuracy (no dropped or double-counted events). This pins the consistency story to exactly-once ingestion + idempotent OLAP writes rather than best-effort, and frames the payload as ad events (impressions / clicks / app-installs) enriched with metadata, serving both live dashboards and historical search over past data. A strong design separates the hot path and cold path. The canonical hot path is SDK/API → Kafka (partitioned by user_id or product_id for co-location) → stream processor (Flink or Spark Streaming) for windowed aggregation → time-series or OLAP store (Druid / ClickHouse / Pinot) for dashboards. The cold path forks off the same Kafka topic into object storage (S3 + Parquet) and is loaded into a warehouse (Snowflake / BigQuery) for ad-hoc analytics. Kafka is the decoupling layer: producers (SDKs, collectors) write events independently of consumer load, and multiple downstream jobs (real-time aggregation, enrichment, cold archive) can each tail the same topic at their own pace. Stream processor responsibilities: tumbling/sliding window aggregations, per-key state (e.g., session windows by user_id), and exactly-once via checkpointing. Flink keeps large state in RocksDB and snapshots Kafka offsets + state together so failures replay deterministically. Spark Structured Streaming uses micro-batches with WAL for similar guarantees. Enrichment can run inline (cheap deterministic fields like geo from IP via in-memory lookup) or asynchronously (expensive joins against slowly-changing dimension tables — denormalize into a side-input stream or a keyed state store). Be explicit about late-arriving enrichment, watermarks, and backfills via reprocessing from the cold archive. The prompt can feel closer to data engineering than generic backend system design; expect detailed questions about table layout, partition keys, and stream processing semantics. ML-engineer follow-up For ML-engineer loops the same event pipeline becomes the feature source, and interviewers may pivot to designing the training and serving systems for a recommendation model (e.g., recommending a news feed). Be ready to cover: offline feature/label generation from the cold archive, the training pipeline and model versioning, the online serving path (feature-store lookup + low-latency inference / retrieval + ranking), and the trade-offs — batch vs. streaming features, training/serving skew, freshness vs. cost, and how the aggregated event store feeds both. Preparation Prepare an event schema with event id, user id, product id, timestamp, device, location hints, and attributes; explicitly call out the partition key and why. Practice the full pipeline diagram: SDK → API gateway / collector → Kafka topic → Flink window aggregation → TSDB/OLAP for dashboards, with a fork to S3 + warehouse for cold queries. Be ready to defend exactly-once vs at-least-once + idempotent writes as the consistency story. Drill query examples: dashboard by 1-min/5-min/1-hour time bucket, per-product event breakdown, top-K active users in last 24h, and warehouse joins after enrichment. Have a watermark and late-event answer ready: allowed lateness window, side-output for late events, periodic reconciliation job.