Last month I performed a simple A/B test on another (Dutch) website of me (dutchento.org) using Visual Website Optimizer (VWO). It was a simple A/B test looking to improve newsletter and RSS subscribtion rates by adding 'persuasion' (three reasons to sign up) to excising subscription links. The results were somewhat surprising to me and apparently also to the people behind VWO because they liked to use my test as a case study on their blog. Read on to learn what I learned by using split testing: A/B testing with competing goals: newsletter CTR increased by 190%, but clicks on RSS feed…. [disclaimer] I'm a happy customer of VWO, It's one of the systems I use to analyze and test websites. I like to see myself as an objective person in talking and advising you about that. If I recommend bad software you won't come back here and you won't tell anyone about me so there's really no use for me to misinform so I never will. I do however signed up for the partner program of VWO which means that if you click the links to VWO in this blogpost I get a 15% commission if you buy anything from them within sixty days. This of course will make me huge amounts of money and I will have even more incentive to blog great things for you. If you don't like that just click this plain link to visualwebsiteoptimizer.com without the referral code. Thanks! [/disclaimer]
Recently I've seen some (often absolute) statements going around, generally in the line of "open source commerce platforms are a terrible idea". Now of course different solutions always have different pros and cons.
A hierarchy of evidence (or levels of evidence) is a heuristic used to rank the relative strength of results obtained from scientific research. I've created a version of this chart/pyramid applied to CRO which you can see below. It contains the options we have as optimizers and tools and methods we often use to gather data.