Skip to content

Fix incorrect p-value calculation#342

Merged
guilhermebodin merged 2 commits into
masterfrom
rs/fix-pvalue
Jun 8, 2026
Merged

Fix incorrect p-value calculation#342
guilhermebodin merged 2 commits into
masterfrom
rs/fix-pvalue

Conversation

@raphaelsaavedra

Copy link
Copy Markdown
Member

In handle_p_values, the function wants to compute two-sided Wald p-values, but the current formula is incorrect. It currently computes a computes a probability under N(beta_hat, std_err) evaluated at Z = beta_hat / std_err.

It should compute a probability under the null distribution of the estimator, not the estimated distribution, i.e., under the null hypothesis H_0: beta = 0, the statistic is Z = beta_hat / std_err and Z ~ N(0, 1). What we're doing is Z ~ N(beta_hat, std_err).

@guilhermebodin please double check if this sounds right...

@raphaelsaavedra raphaelsaavedra self-assigned this Jun 5, 2026
@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.29%. Comparing base (0b0198d) to head (0bd708b).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #342      +/-   ##
==========================================
- Coverage   91.30%   91.29%   -0.01%     
==========================================
  Files          36       36              
  Lines        3713     3711       -2     
==========================================
- Hits         3390     3388       -2     
  Misses        323      323              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@raphaelsaavedra raphaelsaavedra added the bug Something isn't working label Jun 5, 2026
@guilhermebodin

Copy link
Copy Markdown
Member

Your fix is correct, by definition Z ~ N(0, 1). This should be the distribution where we test the p-values.

@guilhermebodin guilhermebodin merged commit 623985c into master Jun 8, 2026
7 checks passed
@guilhermebodin guilhermebodin deleted the rs/fix-pvalue branch June 8, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants