← 返回 apple 的题目列表Implement Multi-Head Attention Forward Pass
类型:online_judge
Problem: Implement Multi-Head Attention Forward Pass
Implement a simplified Multi-Head Self-Attention forward pass.
You are given:
An input tensor X with shape (B, T, D)
B: batch size
T: sequence length
D: embedding dimension
Number of heads H, where D % H == 0
Four weight matrices:
Wq, shape (D, D)
Wk, shape (D, D)
Wv, shape (D, D)
Wo, shape (D, D)
Compute:
Q = X @ Wq
K = X @ Wk
V = X @ Wv
Split Q/K/V into H heads along the last dimension. Each head has dimension Dh = D / H
For each batch and each head, compute scaled dot-product attention:
Attention(Q, K, V) = softmax(QK^T / sqrt(Dh)) V
Concatenate all heads back into shape (B, T, D)
Compute the final output: O = concat_heads @ Wo
You do not need to implement bias, dropout, masks, layer normalization, or backpropagation.
Input Format
Read from standard input:
B T D H
B*T*D floating-point numbers for X
D*D floating-point numbers for Wq
D*D floating-point numbers for Wk
D*D floating-point numbers for Wv
D*D floating-point numbers for Wo
All matrices are provided in row-major order.
Output Format
Print B*T lines. Each line contains the D-dimensional output for one token.
Print each floating-point number with 6 digits after the decimal point.
Constraints
1 <= B <= 4
1 <= T <= 32
1 <= D <= 128
1 <= H <= D
D % H == 0
Absolute value of each input float is at most 10
Example
Input:
1 2 2 1
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
Output:
0.669762 0.330238
0.330238 0.669762
Example
Input
1 1 2 1
1 2
1 0 0 1
1 0 0 1
1 0 0 1
1 0 0 1
Output
1.000000 2.000000