13 Feb 12
21 Apr 12 7:59 pm
Split Tests are hardest when you need them most
Seems to be my day for Affilorama posts! Too much coffee?
How do you solve this conundrum of needing to test a new website (for instance with an A/B Split Test of 2 page versions) to get the early improvements you know you really, really need, when your new site doesn't get much traffic yet (meaning few sales and a very long test)?
Such a Split Test could take months to reach a decent Confidence level (like 90%). Just when you need Split Testing most, it's least effective!I have an idea about this - please comment on it
I've been doing A/B Split Tests with low traffic new websites by assigning a "proxy result" in place of actual sales. For example, you could measure the number of people who go from your Landing/Sales page to a page of background info about your product or service - since they are showing "signs of seriousness". There will be a lot more of them to sample, in a given period of time, than there will be actual sales.
You could even install something temporary and special on your site as a "sales proxy".
Of course this is not as good as a real sales test, and it doesn't test the final "last call to action" before a buy button or hoplink click. But my thinking is that it at least does indicate how to make improvements, in a more factual way, than just guessing at how to improve your pages does.
Thanks for any comments on this idea.