Debug A/B Test Python Code (Metric Computation and Statistical Testing)

Prompt: Debug A/B Test Python Code (No Need to Write from Scratch) You will be given a piece of Python code that analyzes an A/B test. The goal is to read the code, identify bugs/pitfalls, and fix them. Input A user-level dataset (e.g., a pandas DataFrame) with: user_id variant in { 'A', 'B' } exposure in {0, 1} metric (may be missing) Provided code (example) import numpy as np import pandas as pd from scipy import stats def abtest(df): a = df[df['variant'] == 'A']['metric'] b = df[df['variant'] == 'B']['metric'] lift = (b.mean() - a.mean()) / a.mean() t, p = stats.ttest_ind(a, b) return lift, p Tasks Identify at least 3 bugs or statistical pitfalls and explain why they matter. Propose fixes (filtering, missing-data handling, appropriate tests, SRM checks, confidence intervals, etc.). Explain when to use a t-test vs proportion test vs bootstrap vs non-parametric tests. Notes metric may contain NaNs. Some users may have exposure=0. Randomization may be imbalanced. The metric may be skewed or zero-inflated. Output Verbal explanation of issues and fixes; optionally revised code snippets. Example Input user_id,variant,exposure,metric 1,A,1,10 2,A,1,12 3,B,1,13 4,B,1,15 Output (lift, p_value) with proper filtering and NaN handling (exact numbers depend on test choice)