When Less Really Is More: Privacy and A/B Testing

We obsess over significance and sample sizes, but ignore a bigger issue: we’re collecting too much user data - and that leads to completely unnecessary risk - no matter how complicated your test. Even so-called “advanced” testing can work with aggregated data only, reducing privacy risk while delivering the same statistical power.