AB Testing is a Cultural & Personal Choice
Ittakes a lot number of weeks of research into white papers of statistics, informative articles on application, Live blogs for examples hunting, speaking to relevant stakeholders in competition & outside companies to discuss & link it with your current situation & problem statements to finally conform on your Approach.
I did the same with AB Testing!
And in all fairness, I have to admit that this has to be one of the most Surreal experiences since I realised there was a missing piece I was not including in the whole Picture/Frame/Painting of Product Design Thought Process.
AB Testing is critical to make your Product/Campaigns/Analytics/ Development Decision making simple & obvious decision making come out with an astounding clarity once AB Testing is conducted.
Plus how important it is to be persistent & have patience with the tests (design, conducting, results) — to know when to stop & when to start again.
There are two places where conducting AB Testing is a MUST :
- Once when the prototypes of your App Key Pages are ready & you must know the initial response to them from your target consumer personas.
- Second, before doing the campaigns & scaling the features/products — knowing the intricacies details in the form of headline, text, image, theme, colour palette, etc.
A few things must have technical :
- AB Testing tool —
for beginners : VWO/Optimisely like SaaS tool ;
for mature organisations : Building the capability from scratch
for organisations in between who wish advanced readymade tool — I would recommend the hybrid way by using an open platform like Wasabi which can be customised to your requirements.
- Good Quality Team with understanding of Statistics, SaaS Tools, Consumer Behavioural Analysis
- Data Mart/Warehouse with access to Transactional Data as well as Online Behavioural Data with necessary cuts for segmentation basis Business Intelligence
A Few things in terms of Soft Power :
- Open Mind — ready to question obvious right answers (infact more if they are obvious)
- Capability to deep dive & look at intrinsic details
- Collaborate with different teams/people to check if the findings or the direction of the findings are correct — this in my understanding is the most POTENT!
Organisations must insist their people to only come to leaders once they have collaborative evidences to move in a certain direction.
It may so happen that after spending significant time also — you may not come to an conclusive evidence & one should be open & amiable about the same & not push for an evidence for the sake of it.
I must thank quite a few people for the same but most of all Booking.com to highlight that there is something missing in my thought process — the webinar by Lukas was really helpful!