Metrics Collection, Alerting & Click Counters

Design a real-time metrics, click-counting, or alerting pipeline that supports low-latency queries, offline analysis, deduplication, and reliable counters. Requirements Design a metrics collection system, short URL click counter, or ad / product event alerting system. Functional requirements: Ingest high-volume events such as clicks, views, impressions, or metric samples. Support near-real-time dashboard queries and alert evaluation. Preserve raw events for offline analysis and backfills. Deduplicate retries and handle late / out-of-order events. Serve counters or aggregates by entity, time window, and dimension. Scale / constraints: Assume bursty traffic and hot keys for popular links, campaigns, creators, or metrics. Alerting should be timely, but offline analytics may need more exact reconciliation. Raw event retention and aggregate retention have different storage needs. Design decisions: Durable event log vs direct writes to aggregate storage. Stream processor for real-time windows vs batch job for correction. Approximate counters vs exact counters for hot keys. Alert rule storage, evaluator cadence, suppression, and notification fanout. Notes A strong design uses an append-only event log as source of truth, a streaming path for low-latency aggregates, and a batch path for correction. For counters, avoid writing the same hot row on every click; aggregate in memory or by partition, then flush deltas. Use idempotency keys where billing or correctness matters. For alerting, separate metric ingestion from rule evaluation. Rules need thresholds, windows, dimensions, owners, silence windows, and delivery channels. Include backpressure and a degraded mode so ingestion does not fail just because notification delivery is slow. Preparation Practice a lambda-style architecture with event collectors, Kafka-like log, stream processor, OLAP store, batch reconciliation, and alert evaluator. Prepare hot-key mitigation: sharded counters, local aggregation, and hierarchical rollups. Explain event time vs processing time, watermarking, and late-event correction. Add operational metrics for the metrics system itself: lag, dropped events, duplicate rate, and alert delivery latency.