← 返回 scale.ai 的题目列表Implement Multi-Head Attention (forward pass)
类型:online_judge
Task
Implement the forward pass of Multi-Head Self-Attention (forward only; no backprop).
Given:
number of heads h
sequence length n
model dimension d_model
per-head dimension d_head = d_model / h (guaranteed divisible)
input X of shape (n, d_model)
weight matrices W_Q, W_K, W_V, W_O, each (d_model, d_model)
Compute:
Q = X @ W_Q, K = X @ W_K, V = X @ W_V
Reshape Q,K,V to (h, n, d_head)
For each head:
scores = (Q @ K^T) / sqrt(d_head) with shape (n, n)
apply row-wise softmax to get attention weights A
head_out = A @ V with shape (n, d_head)
Concatenate all heads to (n, d_model) and multiply by W_O to produce Y of shape (n, d_model)
Input (stdin)
Line 1: n d_model h
Next n lines: matrix X
Next d_model lines: W_Q
Next d_model lines: W_K
Next d_model lines: W_V
Next d_model lines: W_O
Floats are space-separated.
Output (stdout)
Print Y as n lines of d_model floats, each with 6 decimals.
Constraints
1 <= n <= 20
1 <= d_model <= 32
h divides d_model
Use numerically stable softmax (subtract row max).
Example
Input
1 2 1
1 2
1 0
0 1
1 0
0 1
1 0
0 1
1 0
0 1
Output
1.000000 2.000000