ML System Design — Predict Item Category

Build an ML system that predicts the category of a new catalog item (e.g. "Outdoor → Camping → Tents") from its title, description, attributes, and images. Cover data, labels, model architecture, serving, and evaluation. Requirements Functional: given an unlabeled item (title + description + structured attributes + images), output a predicted category path within Walmart's taxonomy, plus a confidence score and (optionally) top-K alternatives. Non-functional: latency in tens of milliseconds at write time (called during item onboarding); throughput sized for catalog ingest spikes; offline batch mode for back-filling historical items; observable per-category accuracy with drift detection. Outputs: a single best leaf-level category, top-K alternatives, and a calibrated probability per output for downstream routing thresholds. Notes Frame the problem as hierarchical multi-class classification. Two common framings: (1) flat softmax over leaf categories — simple, scales poorly when the taxonomy has tens of thousands of leaves; (2) hierarchical classifier that walks the taxonomy level by level — each node holds a classifier over its children. Hierarchical wins when the taxonomy is deep and unbalanced (which is the catalog reality). Training labels come from the existing catalog: each historical item already has a leaf category assigned by sellers or curators. Label quality is uneven — flag noisy labels via cross-validation disagreement and prioritize human review for the long tail. Features: text encoder over title + description (a fine-tuned BERT-class encoder or a smaller distilled model for latency budget), categorical embedding for known structured attributes (brand, color, material), and an image encoder (CLIP-class or a smaller CNN) for the primary product photo. Concatenate the three modality vectors before the classification head. For latency-sensitive serving, the image path can be skipped on the hot path and added via async re-scoring. Serving: model server behind a feature-store-backed feature pipeline; gateway dispatches request to TF-Serving / TorchServe / Triton. For the hierarchical setup, cascade calls — predict top-level, then route to that node's child classifier, etc. Cache the encoder forward pass when re-classifying the same item across nodes. Calibration matters because the downstream auto-categorization pipeline only auto-applies when probability exceeds a threshold; uncalibrated softmax inflates confidence. Add a temperature-scaling or isotonic-regression calibration pass on a held-out set per category. Evaluation: top-1 leaf accuracy, top-K accuracy (K=3 / 5), and a hierarchical F1 that gives partial credit for getting the parent right but missing the leaf. Slice metrics by category to surface long-tail blind spots. Drift detection: monitor input distribution (new vendors, new vocabulary) and output distribution (sudden share shifts in a category) with daily KS tests; trigger retraining or active-labeling when drift exceeds threshold. The interviewer in this round drilled on the architecture and the "how do you know it's working" answer; expect a similar push toward evaluation and drift rather than novel modeling. Preparation Drill the canonical multi-modal classification skeleton (text encoder + structured-feature embeddings + image encoder + classification head + calibration) until you can sketch the block diagram in under two minutes. Be ready to discuss the flat-vs-hierarchical trade-off explicitly, with numbers (taxonomy depth, leaf count). Practice the evaluation discussion: top-K, hierarchical F1, calibration, per-category slicing, drift detection. The reported round closed the loop here, not on the model architecture. Brush up on the two-tower retrieval pattern even though this is a classification problem — the interviewer often probes whether you would frame catalog onboarding as retrieval instead, and the right answer is "only if the taxonomy is open-ended; otherwise classification is the right framing."