Numerically Stable Entropy from Logits (Batch and Streaming)

AI Coding: Entropy from Logits (Numerical Stability + Streaming) Given logits = [z1, z2, ..., zk] (unnormalized log-probabilities). Let [ p_i = \frac{e^{z_i}}{\sum_{j=1}^k e^{z_j}} ] be the softmax probabilities. Entropy is [ H(p) = -\sum_{i=1}^k p_i \log p_i. ] Part 1: Numerically stable entropy Implement entropy_from_logits(logits) that returns H(p) and is numerically stable for logits as large as (\pm 10^4). Avoid overflow/underflow (do not naively form p then log(p)). Part 2: Streaming entropy Logits arrive as a stream. Implement a StreamingEntropy class: update(z: float) -> None finalize() -> float Requirements: Do not store all logits ((O(1)) space). Must be numerically stable, including when the running maximum changes. Example I/O (one possible local format) Input: integer k, then k floats. Output: entropy. Constraints (1 \le k \le 10^7) logits may be in ([-10^4, 10^4]) Examples k=2, logits=0 0 -> 0.6931471805599453 k=3, logits=0 0 0 -> 1.0986122886681098 k=2, logits=10000 0 -> approximately 0.0 Example Input 2 0 0 Output 0.6931471805599453