← 返回 openai 的题目列表Cross-Entropy Loss Implementation
类型:qbank
Implement cross-entropy loss from NumPy starter code, then extend it with masking and label smoothing while discussing the underlying ML concepts and the relationship to KL divergence.
Requirements
Implement cross-entropy loss from starter code in a notebook that already imports NumPy.
Extend the implementation to support a mask so excluded positions do not contribute to the result.
Add label smoothing to the same implementation.
Explain the relevant ML fundamentals while coding, then answer follow-ups about KL divergence if time remains.
Notes
The round is free-form rather than driven by a large visible test suite. The coding volume is modest, but conceptual questions continue throughout the implementation. Confirm the expected reduction, mask shape, ignored-position behavior, and smoothing convention before coding.
Preparation
Implement the base loss, masking, and label-smoothing extensions in NumPy without relying on a framework loss helper.
Rehearse the shape and reduction choices for batched predictions and sequence masks.
Explain how cross-entropy and KL divergence are related and where label smoothing changes the target distribution.