Hypothesis testing during funnel attrition: examples and life hacks from KMA.biz.
Why test hypotheses and what to do with the obtained results? Today, the KMA.biz team will elaborate on this process and showcase the outcomes of their tests.
By employing hypothesis testing, we aim to draw conclusions about the viability of various creatives. Hypothesis testing assesses two mutually exclusive claims about the effectiveness of creatives (simplified: 'Approach A is better than Approach B' – and vice versa) to determine which variant yields superior results.
Disclaimer: In the tests, hypotheses were formulated, implemented, results were examined, and conclusions were drawn based on conversion rate rather than CTR. This is because Conversion Rate (CR) specifically indicates the percentage of website users who have accomplished the intended goal – submitting a request
By employing hypothesis testing, we aim to draw conclusions about the viability of various creatives. Hypothesis testing assesses two mutually exclusive claims about the effectiveness of creatives (simplified: 'Approach A is better than Approach B' – and vice versa) to determine which variant yields superior results.
Disclaimer: In the tests, hypotheses were formulated, implemented, results were examined, and conclusions were drawn based on conversion rate rather than CTR. This is because Conversion Rate (CR) specifically indicates the percentage of website users who have accomplished the intended goal – submitting a request
Offer - Frutalika
Source: MyTarget
Period: May 2022
Period: May 2022
Hypotheses
- The doctor's approach. Since the promotion is related to medical topics - WHO, we decided to test the approach with a female doctor-dietitian (slim). We added the trigger 'recommended by experts' and prominently placed the mock-up
2.Exotic Appeal + Girl. A slim, contented girl was chosen along with exotic nature, illustrating the concept of Frutalika (weight loss with exotic fruits). Visual confirmation that the product works.
Test results
Screenshots from the personal account:
As seen in the screenshot, the visualization with a slim girl in a bikini emerges as the winner.
Offer - Daoshen
Source: ads2.bid
Period: June 2022
Period: June 2022
Hypotheses
- Medical emphasis to give the offer a medical ointment vibe. To achieve this, we created a mock-up against a certificate backdrop and a mock-up with a medical badge. A similar variation involves an offer with a reward.
2.llustrating the offer's action – we depict clean feet without fungus as a result of using Daoshen.
Test results
We also faced significant approval issues, and there's not much we can do about that. Additionally, for Daoshen, we tested a GIF instead of a mock-up, but the loading speed didn't allow engaging the audience, and this approach performed poorly. We almost immediately disabled it.
Offer - Okutsin
Sources: Advertlink; ads2.bid; KMA Showcase
Period: June 2022"
Period: June 2022"
Hypotheses
1.Medical Theme. As the promotion is related to the medical field, it should appeal best to doctors. We created 2 variations with doctors. The first one includes the visualization of an eye examination by an ophthalmologist, while the second one features an ophthalmologist doctor along with a certificate.
2. Advertising approach resembling banners or images commonly used on marketplaces
Test results
The mock-up with an ophthalmologist looking straight ahead emerged as the winner
Some mock-ups were immediately disabled, as the doctor+certificate combination and the source material clearly pulled ahead. We tested them on two sources.
Offer - Weex Raspberry-Goji
Source: MyTarget
Period: June to July 2022
The task was to test background-free mock-ups (PNG) with minimal alterations.
Period: June to July 2022
The task was to test background-free mock-ups (PNG) with minimal alterations.
Hypotheses
- Provide the client with the option to choose from two. By increasing the number of packages in the mock-up, we create a sense of volume for the product. It's as if there's more of it than there actually is.
2. Exploiting the medical theme — mock-up + certificate + medical badge
Results
The mock-up with the certificate and badge emerged as the winner.
Let's summarize the test results
So far, we observe the following trends:
However, we can't definitively conclude that visualizing the result works 100% for weight loss. The test conclusion with a new weight loss promotion using the exact same visualization as Frutalika didn't work. In such cases, a retest should be conducted to expand the dataset and obtain a more accurate result.
What we find less effective:
Register with KMA! We're testing mock-ups specifically for you and providing them upon the first request.
More news and useful information are available in the Cozy KMA Chat, where you're always welcome!
- Semi-medical variations are effective: with badges, certificates, and doctors. This makes sense since the promotion mostly revolves around medical themes. Badges and doctors additionally reinforce the information presented in the promotion text.
- When using a photo of a doctor, it's preferable if they are looking directly into the camera.
- Illustrating the effect of the product yields good results – as seen with examples like Frutalika (slim girl) and Daoshen (fungus-free feet).
However, we can't definitively conclude that visualizing the result works 100% for weight loss. The test conclusion with a new weight loss promotion using the exact same visualization as Frutalika didn't work. In such cases, a retest should be conducted to expand the dataset and obtain a more accurate result.
What we find less effective:
- Visualizations on abstract backgrounds
- Visualizations with small mock-ups
- GIFs and/or videos instead of mock-ups
Register with KMA! We're testing mock-ups specifically for you and providing them upon the first request.
More news and useful information are available in the Cozy KMA Chat, where you're always welcome!