Statistics and inference
An analyst checks an A/B test every day and stops the first time p < 0.05.
AppliedMultiple choice
An analyst checks an A/B test every day and stops the first time . What does this do?
- ANothing, since each daily test is valid at the 5% level on its own
- BIt inflates the false-positive rate well above 5%
- CIt lowers the false-positive rate, because more data is examined
- DIt only affects power, not the false-positive rate
The worked solution is in Premium
The answer, the full working and the one idea to take away – for this and all 1,322 questions in the bank. Answer it in practice and your working is marked, with a known mistake named when you make one.
Report an issue with this question
Of the kind reported at
More statistics and inference questions
- An A/B test compares a 10% baseline conversion with a hoped-for 11%.Applied · Premium
- An A/B test has 10,000 users per arm and a 5% baseline conversion rate.Advanced · Premium
- You adjust an experiment’s outcome using each user’s pre-experiment value of…Advanced · Premium
- You have 100 observations and a standard error of 0.4.Foundation · Free solution
- Which statement about the central limit theorem is correct?Applied · Free solution
- A strategy won 220 of 400 trades.Applied · Premium