When your A/B testing dashboard shows a 3.5% conversion rate, what it is really saying is something more like 3.5% plus or minus 0.4%. You observed 3.5%, but due to random variation in who happened to ...