Dataset Exploration: Detect Label Noise and Choose Metrics

Problem: Dataset Exploration — Detect Label Noise and Choose Evaluation Metrics Given binary classification predictions and labels, perform data quality analysis and evaluation: Compute and report: Accuracy, Precision, Recall, F1, and ROC-AUC (if probabilities are provided). Explain why Accuracy can be misleading under heavy class imbalance and propose better metric choices. Design a simple label-noise detection method: find samples where the model is highly confident but disagrees with the label; output their indices (or a Top-K list). Propose a follow-up workflow to validate/clean the suspected samples (steps only). Input y_true: shape (N,), values in {0,1} y_prob: shape (N,), predicted probability for the positive class Optional: group_id/source field to analyze noise by source Output A dict/struct with metrics suspects: list of suspect indices (e.g., y_prob>0.9 with y_true=0 or y_prob<0.1 with y_true=1) Short explanation: metric choice + cleaning workflow Constraints You may use NumPy/Pandas; no model training required. Must consider how threshold choice affects Precision/Recall. Example test y_true=[1,0,1,0], y_prob=[0.99,0.95,0.1,0.05] suspects should include index 1 and index 2. Example Input y_true=[1,0,1,0] y_prob=[0.99,0.95,0.1,0.05] Output suspects contains [1,2] (order may vary) + valid metrics