Data Science & Analytics
Search Procedures
Model how a specification universe is actually walked, rather than assuming it is enumerated. Implements the search procedures a p-hacker or a pressured agent uses (first-significant stopping, random trial within a budget, greedy coordinate descent from the pre-registered analysis, random hill-climbing, and the two-stage split-sample walk with an optional continuation rule, after Adda, Decker and Ottaviani 2020), replays each on null data to measure its false-positive rate and the inflation of what it reports, and records the path so the write-up can be checked against it. Use when asked what a realistic p-hacking session looks like, how likely a given way of searching is to manufacture significance on a specific design, how to calibrate a result that came from a sequential rather than exhaustive search, or to simulate an agent's search behaviour on known-null data.
- Publisher / source
- brycewang-stanford/Auto-Empirical-Research-Skills (found on GitHub top repositories)
- Upstream record
- brycewang-stanford/Auto-Empirical-Research-Skills
- Version
- Version 1
- Published
- Publication review
- Approved for publication
- Content fingerprint
cda7e36c073a3f09ba779c49f00aeda7b79cfdf67cd312fb3459867db47add03