Debug a Transformer Implementation and Modify It into a Classifier

Problem: Debug a Transformer Implementation and Turn It into a Classifier You are given a Transformer implementation (self-attention, FFN, residual connections, LayerNorm, etc.) along with a unit test suite. Task 1: Debug The code contains several planted bugs (around 4). Identify and fix them so that all provided tests pass. You may change any part of the implementation (e.g., attention computation, masking logic, tensor reshaping, LayerNorm placement, dropout, device/dtype handling, parameter initialization, positional encoding, etc.). Goal: run the provided test program and make all tests pass. Task 2: Convert it into a classifier Based on the fixed model, modify the Transformer to perform classification: Change the final layer to output logits for a fixed number of classes num_classes. You may choose an aggregation strategy (e.g., using a [CLS] token, mean pooling, max pooling), but it must match the interface expected by the tests. Goal: run the provided test script and pass classifier-related tests. Constraints Follow the test script; sizes are typically small (batch size/sequence length/hidden size) to enable quick debugging. Output Submit the corrected code so the test script runs and passes.