ML Foundations — Optimizers, Scaling Laws, K-Means and GMM

A machine-learning foundations round covering optimizers, scaling laws, k-means clustering, and Gaussian Mixture Models. The leaked detail is topic-level rather than full question text, so treat this as a drill checklist for an oral fundamentals round. Requirements Prepare concise explanations and trade-off discussions for these topics: Optimizers: compare common first-order optimizers, when momentum helps, and how adaptive methods change the update. Scaling laws: explain the relationship among data, parameters, compute, and expected loss trends. K-means clustering: objective, assignment/update steps, initialization sensitivity, and convergence behavior. Gaussian Mixture Models: probabilistic clustering view, soft assignments, EM intuition, and how it differs from k-means. Notes This is an oral ML-knowledge round. The bar is fast, precise explanation under follow-up pressure, not coding. The same loop also included research-design rounds and an attention-coding round, so answers should connect fundamentals to modern LLM practice when natural. Preparation Write one whiteboard-sized comparison table for SGD, momentum, Adam-style adaptive methods, and their failure modes. Derive k-means and GMM/EM updates at a high level, then practice explaining the difference without equations. Prepare a two-minute scaling-law answer that separates empirical trend, compute budget, and data/parameter trade-off.