ML Coding: 2-D Convolution, Decaying Attention, Training Loop

Deep Learning and AI product roles ask small ML implementation tasks: write 2-D convolution in NumPy, implement decaying attention `softmax(q @ k + b) @ v`, and fill a standard training loop with a provided model and dataset. Requirements 2-D Convolution in NumPy Given a 4x4 input and 3x3 filter, implement stride-1 valid convolution. A = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]) K = np.array([[1, 0, -1], [1, 0, -1], [1, 0, -1]]) Expected output shape is (H - kH + 1, W - kW + 1). Start with loops, then discuss slicing / vectorization. Decaying Attention Implement: att = softmax(q @ k + b) @ v where b is based on absolute distance between token indices. Clarify whether k is already transposed; in standard notation it is q @ k.T. Training Loop Given data and model setup, write the loop: forward pass, loss, backward, optimizer step, zero gradients, evaluation / logging. Notes For convolution, the clean loop is: out[i, j] = np.sum(A[i:i+kH, j:j+kW] * K) Know the difference between convolution and cross-correlation: many ML libraries do cross-correlation and do not flip the kernel. For decaying attention, construct a distance matrix: idx = np.arange(seq_len) dist = np.abs(idx[:, None] - idx[None, :]) b = -alpha * dist scores = q @ k.T + b weights = softmax(scores, axis=-1) out = weights @ v For training loops, the expected answer is usually framework fluency and shape discipline, not a novel model. Preparation Rehearse NumPy slicing: .shape, A[i:i+kH, j:j+kW], broadcasting, and np.sum axes. Implement a PyTorch training loop with model.train(), optimizer.zero_grad(), loss.backward(), optimizer.step(). Trace tensor dimensions out loud for attention: [B, T, D] @ [B, D, T] -> [B, T, T].