ML Fundamentals, Transformer & Regularization

Snap MLE screens repeatedly include rapid-fire ML fundamentals: overfitting, bias-variance, normalization, optimizers, loss functions, Transformer attention, positional encoding, and project deep-dives. Requirements Prepare for a verbal ML fundamentals round that can pivot quickly across: Overfitting in linear models vs deep neural networks. Bias-variance decomposition and data imbalance. L1 / L2 regularization, dropout, early stopping, data augmentation, and weight decay. Batch norm vs layer norm, including learned parameters and train / inference behavior. Optimizers and loss functions. Transformer structure: Q/K/V, scaled dot-product attention, multi-head attention, residuals, normalization, feed-forward blocks, and positional encoding. Project deep-dive: data, model choice, metrics, offline / online gap, deployment, and failure modes. Notes The expected answer style is concise and comparative. For overfitting, separate linear-model tools such as feature selection and L1 / L2 from DNN tools such as dropout, augmentation, early stopping, weight decay, and normalization. Use the training-vs-validation loss gap to diagnose overfit. For attention, write the standard shape out loud: softmax(QK^T / sqrt(d_k))V. The key conceptual point is that vanilla self-attention is permutation-invariant without positional information, so the model needs sinusoidal, learned, rotary, or relative position encoding to represent order. For project deep-dives, do not stop at model architecture. Explain why the problem needed ML, what the baseline was, which metric moved, how labels were collected, how the model was served, and what you monitored after launch. Preparation Build one-page answers for overfitting, imbalance, normalization, optimizers, and Transformer attention. Practice a 5-minute project story focused on metric, data, model, experiment, and production lesson. Drill concise comparisons: batch norm vs layer norm, L1 vs L2, SFT vs ranking-style objectives, offline vs online metrics. Write the attention formula from memory and explain every term.