Just want to chat? Talk some smack. If it doesn't fit into one of the areas below then you'll be posting here (Note: Please try and find the best section before posting here)

Split Tests are hardest when you need them most

PremiumMember
jimcoe
Posts: 398
Joined: 13 Feb 12
Trust:

Split Tests are hardest when you need them most

Seems to be my day for Affilorama posts! Too much coffee?

How do you solve this conundrum of needing to test a new website (for instance with an A/B Split Test of 2 page versions) to get the early improvements you know you really, really need, when your new site doesn't get much traffic yet (meaning few sales and a very long test)?

Such a Split Test could take months to reach a decent Confidence level (like 90%). Just when you need Split Testing most, it's least effective!

I have an idea about this - please comment on it.

I've been doing A/B Split Tests with low traffic new websites by assigning a "proxy result" in place of actual sales. For example, you could measure the number of people who go from your Landing/Sales page to a page of background info about your product or service - since they are showing "signs of seriousness". There will be a lot more of them to sample, in a given period of time, than there will be actual sales.

You could even install something temporary and special on your site as a "sales proxy".

Of course this is not as good as a real sales test, and it doesn't test the final "last call to action" before a buy button or hoplink click. But my thinking is that it at least does indicate how to make improvements, in a more factual way, than just guessing at how to improve your pages does.

Thanks for any comments on this idea.

Cheers All!

_jim coe
  • 1
esolutions
Posts: 717
Joined: 28 Apr 09
Trust:
Hi,

I agree, you need a lot of data in order to conduct a conclusive split test. Some might suggest pushing a lot of ppc traffic or something, but ppc traffic acts different than other types of traffic.

Instead, I use a little tool called Mouseflow which records each visitor as they are on your site. You can then watch those interactions to see exactly what people are doing, where they are scrolling, where they are clicking and where they are getting hung up, etc.
  • 1
List not growing fast enough?
Get More Opt Ins:
http://mattgreener.com/view/arfree
 
PremiumMember
jimcoe
Posts: 398
Joined: 13 Feb 12
Trust:
Thanks eSolutions!

Mouseflow sounds good. I use an online service called (of all things) "CrazyEgg" which sounds like it does the same thing (plus heatmaps).

In the past (in spite of the cost) I've done "temporary PPC", that is, just enough PPC spend to speed up an A/B Split Test.

Haven't thought about how PPC traffic is atypical of other traffic - but that's a good thing to think about - thanks.

Cheers!
_jim coe
  • 1
esolutions
Posts: 717
Joined: 28 Apr 09
Trust:
Hi,

Yeah, CrazyEgg and Mouseflow are very similar. Mouseflow has heatmaps as well. CrazyEgg is considerably more expensive. I thought Mouseflow might not be as good due to the lower price, but I've been happy with the useful data that it has provided.
  • 1
List not growing fast enough?
Get More Opt Ins:
http://mattgreener.com/view/arfree
 

This topic was started on Apr 21, 2012 and has been closed due to inactivity. If you want to discuss this topic further, please create a new forum topic.

cron