← 返回 databricks 的题目列表Top-5 Most Similar Rows Using MSE Across Multiple Features
类型:online_judge
You are given two datasets (or tables) A and B. They share the same set of numeric feature columns (e.g., f1, f2, ..., fk) and each has a unique identifier column (e.g., id).
For each row a in A, find the top 5 most similar rows in B, where similarity is measured by the mean squared error (MSE) across multiple features:
[ MSE(a,b)=\frac{1}{k}\sum_{j=1}^{k}(a.f_j-b.f_j)^2 ]
Requirements:
For every A.id, output 5 rows: the matched B.id values and their corresponding MSE.
Sort by increasing MSE. If there is a tie, use B.id as a deterministic secondary sort key (ascending).
You may implement using SQL (e.g., window functions) or pandas.
Constraints (state reasonable assumptions if not provided):
|A| = n, |B| = m, number of features is k.
All features are numeric (int/float) with no missing values (or specify how to handle missingness).
Provide the query/code and briefly discuss complexity or scalability considerations for large datasets if needed.
Example
Input
A:
1 0 0
2 1 1
B:
10 0 1
11 1 2
12 2 2
13 0 0
14 1 1
15 3 3
(features: f1 f2)
Output
For a_id=1 (0,0): b_id ranking by MSE -> 13(0),10(0.5),14(1),11(2.5),12(4)
For a_id=2 (1,1): b_id ranking by MSE -> 14(0),10(0.5),11(0.5),13(1),12(1) (tie broken by b_id)