Question: what are the different possible outcomes of A/B test experiments and what do you do depending on each of those scenarios?
It’s important to regularly share these findings (at least options 1,2 and 3) with everyone in the team/company to get feedback and to increase everyone’s knowledge of our customers behaviour.
What do you think? Are there more outcome options for your CRO experiments?
Recently I've seen some (often absolute) statements going around, generally in the line of "open source commerce platforms are a terrible idea". Now of course different solutions always have different pros and cons.
A hierarchy of evidence (or levels of evidence) is a heuristic used to rank the relative strength of results obtained from scientific research. I've created a version of this chart/pyramid applied to CRO which you can see below. It contains the options we have as optimizers and tools and methods we often use to gather data.