Deep Learning Fundamentals: Optimization, Drift, Calibration

Deep Learning / Research screens can be pure fundamentals: gradient descent, batch vs SGD, convexity, sharp vs flat minima, bias-variance, calibration, model drift, and business framing of ML quality problems. Requirements Prepare to answer and derive the following without code: How gradient descent works; role of learning rate. Whether gradient descent guarantees global optimum. Which loss surfaces guarantee convergence to a global minimum. Full-batch vs mini-batch vs stochastic gradient descent: advantages and disadvantages. Bias vs variance and how to reduce each. Why small batches can escape sharp / shallow local minima. Generalization gap: train distribution vs population distribution. Why flatter minima may generalize better than sharp minima. Confidence calibration: definition and common methods. Model drift: definition and detection. How to translate a customer / business scenario into a measurable ML problem. Notes The expected answers are conceptual but precise. Convex loss surfaces are the clean case where any local minimum is global. SGD has noisier gradient estimates than full-batch GD; that noise can help escape some sharp or shallow basins. Full-batch uses the exact empirical gradient but may converge to sharper minima and is often less scalable. Generalization gap appears when training loss and population loss diverge; flat minima are less sensitive to small distribution shifts. Calibration asks whether predicted probabilities match empirical frequencies. Temperature scaling, Platt scaling, isotonic regression, and reliability diagrams are standard tools. Drift detection can monitor feature distribution shift, prediction distribution shift, calibration decay, performance on delayed labels, and business KPI deltas. Preparation Draw three loss landscapes: convex bowl, sharp minimum, flat minimum. Time a 90-second explanation of why mini-batch noise can help escape sharp minima while also making convergence noisier. Practice explaining batch-size trade-offs without hand-waving about "noise". Prepare one business example: customer observes worse model decisions after deployment; map it to drift, calibration, or data-quality diagnosis.