- Dec 30, 2009
- 1,883
- 480
- First Name
- Yago
@Alexander Lau
I agree with you that on paper that sounds good, but if you read DI testing parameters "For Optimizely, in general you need 12,000 website visitors per test variation". That is a lot of traffic on each variation in order to yield the proper data, so it means that the time line is long (several months to get enough traffic on that particular change). The problem is that in the car business as our inventory changes, the OEM offer changes, model changes, availability changes, weather changes, etc customer input starts to vary with that so the intent of month 1 in our sample is different of the intent on month 2 however our results will have those together. I believe that a multi dealer A/B test that allows for a reduced time sampling yields better results to apply changes.
We added heat mapping 5-6 years ago to all our sites and almost to the last one, regardless of the color, design, banner, etc they all had the exact same patterns. We even tried, within the norms of compliance, to change consumer behavior as some dealers accepted the challenge to send customers thru a funnel of our choosing. That proved difficult. So I came to believe, an idea that I can't fully develop right now on this paragraph, that consumers follow a certain set of rules and patterns when working with a website and that these didn't change very much so we started focusing on following these patterns.
Lots of data here https://www.nngroup.com/articles/
I agree with you that on paper that sounds good, but if you read DI testing parameters "For Optimizely, in general you need 12,000 website visitors per test variation". That is a lot of traffic on each variation in order to yield the proper data, so it means that the time line is long (several months to get enough traffic on that particular change). The problem is that in the car business as our inventory changes, the OEM offer changes, model changes, availability changes, weather changes, etc customer input starts to vary with that so the intent of month 1 in our sample is different of the intent on month 2 however our results will have those together. I believe that a multi dealer A/B test that allows for a reduced time sampling yields better results to apply changes.
We added heat mapping 5-6 years ago to all our sites and almost to the last one, regardless of the color, design, banner, etc they all had the exact same patterns. We even tried, within the norms of compliance, to change consumer behavior as some dealers accepted the challenge to send customers thru a funnel of our choosing. That proved difficult. So I came to believe, an idea that I can't fully develop right now on this paragraph, that consumers follow a certain set of rules and patterns when working with a website and that these didn't change very much so we started focusing on following these patterns.
Lots of data here https://www.nngroup.com/articles/
Last edited: