Recently, Apple introduced the M4 chip, enabling a slimmer design and better hardware. Besides being convenient, mobiles also close the gap in performance with PCs, ensuring their market dominance.
That’s why mobile campaigns are growing more important than ever. This is especially true for some local markets, like Pakistan, where almost 70% of traffic is mobile.
This particular GEO has caught the attention of our friends from Golden Goose. GG media buyers made and shared a case study. It’s a sweepstakes campaign with careful pre-planning and explained optimization logic. If you’ve been looking for a reason and guidance to tap into the mobile market, here is a good one.
Campaign Summary
Ad network: RollerAds
Ad format: Push notifications
Affiliate network: Golden Goose
Offer IDs: 15230, 11184
Vertical: Sweepstakes
GEO: Pakistan 🇵🇰
Flow: Pin submit
Choosing an Offer
Initially, two similar sweepstakes were selected for the case study:
- PK WIN CASH PRIZE 119 PIN ALL (15230): Payout rate: $0.96
- PK WIN CASH 97 PIN ALL (11184): Payout rate: $0.96
Despite the similarities, their performance varied, which you will see slightly later. Having created the campaigns in the affiliate network, our partners added the offers to Binom, their tracker of choice.
Designing Pre-Landers
Even though sweepstakes are relatively simple to promote, pin submits do require a higher financial commitment. That’s why it’s a good idea to use pre-landers to warm up the users and foster more credibility.
In total, there were three pre-landers designed: slots, spin-the-wheel, and scratch cards. While similar in the underlying concept, their performance would vary, and you will see soon how.
Pakistan has 4 major mobile carriers: Jazz, Telenor, Ufone, and Zong. For the sake of simplicity, it was decided to stick to the fourth option only — Zong. At least this was what the Golden Goose manager suggested as the most converting alternative.
The next step was to create a new campaign inside the tracker.
A/B testing typically requires just two opinions, but there were three pre-landers. The decision was made to compare the landing pages in two stages.
- First round: slots vs. spin-the-wheel
- Second round: the winner against scratch cards
Ultimately, there were three campaigns launched, based on their push subscription age. Sometimes you want to target specific subscribers. For instance, new RollerAds subscribers tend to be less ad blind. However, it doesn’t mean they are always the best option. So what were the subscription age groups?
- Recent subscribers: 0–7 days
- Familiar users: 8–30 days
- Seasoned subscribers: 30+ days
Three hours into the testing, spin-the-wheel outperformed slots almost twofold. This can be seen by looking at the LP CTR indicator.
Spin-the-wheel entered the second round and was put up against scratch cards. The results were as follows:
This time, spin-the-wheel had the double LP CTR of scratch cards. Thus, the other pre-landers were put to a halt; while spin-the-wheel was used to determine optimal zones. It also helped to understand which offer converted better.
Test #1 Optimization
Let’s have a look at the best-converting browsers at the time:
Logically, only the browsers giving leads would be left for subsequent tests. At this step, it was worth checking which Android versions performed better.
The versions 4, 5, 8, and 9 underperformed and could be safely excluded. But what about the stats for each campaign?
The campaign with the subscription age of 30+ days was profitable right away. But what about the rest? Would zone and feed optimization get those campaigns back in black?
Analysis of 30+ Subscription Days Campaign
Even though subscribers with 30+ days proved to be profitable right away, it was important to confirm the initial observation.
Zone 793592 delivered a lead, but the cost of acquiring it was too high, which is why it was in the red. It was too early to ban it, because of three optimization rules.
Three Rules of Optimization:
- The zone would be banned if it was in red at 2x of the lead’s value with no leads at all.
- The zone would undergo optimization by lowering the bid, if the zone had leads but remained in the red at more than 4x of the lead’s value. If that didn’t make the zone profitable, it would be banned.
- The zone would undergo optimization by raising the bid if it was profitable and had leads. At this point, remember to consult your ad network manager to learn about the optimal bid amount, as well as the current traffic volume.
Accordingly, zone 793592 required additional testing without lowering the bid just yet. In case of GG, they were buying 12–18% of traffic and were advised to increase their bid to 0.006. Another thing to remember is to monitor the worst-performing creatives.
The first test resulted in two underperforming creatives. However, excluding them immediately would be a mistake, as their poor performance could be attributed to bad luck. Pre-lander tests rarely reveal the full potential of the creatives. Thus, it’s crucial to run a couple of test runs and look for consistency.
In the end, the 30+ subscription age campaign received an overall increase in bidding of up to 0.005. Those feeds bringing in the leads got a bid raised to 0.006. Finally, only top-performing browsers and OS versions remained.
Analysis of 8–30 Subscription Days Campaign
The results were as follows:
Basically, there was nothing to ban. The decision was made to raise custom bids for the converting feeds.
Compared to the previous campaign, some feeds stopped being profitable. The optimization logic was as follows:
- Feeds with the leads and negative ROIs were left with a bid of 0.005.
- Their zones were raised to 0.006.
- Profitable feeds with the leads saw a higher bid of up to 0.006.
Analysis of 0–7 Subscription Days Campaign
Let’s look at the stats:
- The first two zones had voluminous traffic but also a loss of over 4x of the lead’s value. To save the profit, the bid was lowered to 0.0021.
- The third zone was excluded.
- Zones #4 and #5 had their bids lowered too: 0.0021 and 0.0025, respectively.
- All the zones that cost more than 2x of the lead’s value and had the LP CTR below 10% with 20+ clicks were banned.
- Thus, the following zones were banned: 1154615, 1151252, 1168954, 1166240, 1066120, 1169089, 847404, 1184202.
Then, it was time for the second test run. Here is how pre-landers performed:
Scratch cards performed on par with spin-the-wheel; they even outperformed it once. Still, the dataset was insufficient, which is why a second chance was given.
Test #2 Optimization
Pro tip: don’t make changes to the existing campaigns; instead, duplicate them and make any tweaks there, including bid adjusting and zone banning.
Here are the results after launching all three campaigns:
Scratch cards performed well for one of the campaigns, while spin-the-wheel was better for the rest. Thus, it was decided to make rules for that specific campaign.
Check out four days of test #2 stats:
Stats by subscription age:
As you can see, the main share of traffic came from either very old or very new leads, with a subscription age of 0–3 days. Reasonably, it was decided to optimize the ROI of these two groups first. There was an attempt to make other zones profitable as well.
- The optimization benefited the old group, boosting both clicks and leads.
- As for subscribers with the age of 30–60 days, there was a surge in clicks but no leads.
- Subscribers with the age of 0–3 days had fewer clicks (many sources were banned) but the CR got higher. The ROI was negative because the bid was raised substantially.
- Other subscription age groups didn’t change much.
Test #3 Optimization
In the third test, there were six dedicated campaigns, differing by the subscription age: 0–3, 4–7, 8–15, 16–30, 30–60 days, plus the old group. Let’s use a 0–3 campaign to illustrate what was the optimization logic.
All the zones were broken down by age zones and days. This helped to understand how the number of clicks in each zone changed per subscription age, depending on the bid by day. The whole optimization went as follows:
- The bid for 1015492, a converting but negative zone, was lowered, alas to no avail:
This zone generated the most clicks. During the first test, there were some leads but with a negative ROI. Lowering the bid by almost 2x caused the clicks to plummet with 0 leads — an obvious ban.
- For zones with positive ROI, the bid was increased, which improved overall results:
The second test multiplied the number of clicks by 5–10 times and generated slightly more leads; unfortunately, less than desirable. Such zones usually bring in the most profit, but one has to choose whether to raise the bid further or stop the efforts. Consult your ad network manager for clarifications.
- Zones with a slightly positive or breakeven ROI started to perform worse:
Such zones typically require an individual approach. As stated earlier, the bad zones were banned, leaving only the good ones with custom bids. Other campaigns were analyzed in the same way as described earlier.
Speaking of pre-landers, after looking into scratch cards, it was decided to make a dedicated rule in the tracker for the campaigns with the subscription age of 4–7 days, plus the old one to ensure that traffic went to both pre-landers, when necessary.
Scratch cards started to outperform spin-the-wheel for the old campaign. Nonetheless, it was reasonable to conduct a split test and to see whether spin-the-wheel should be turned off.
None of the creatives was excluded by this point, even the underperforming ones. The second test proved that the worst creative initially might become the next leader in the blink of an eye.
How the creatives performed during test #1:
How the creatives performed during test #2:
A closer look was taken at the lines under the star (663207), this is the second to last creative for the subscription age of 0–7.
For the sake of simplicity, the creatives themselves are not shown in the case study. If you want to get this info, register on the Golden Goose affiliate network and ask the manager there.
Test #3 Analysis
After two days of testing, it was time to look through every campaign and banned zones with a negative ROI. As for the zones with a positive ROI, the bid was increased. Additionally, the creatives’ CTR was higher than the second test, so the custom bids were lowered too. This helped to generate even more leads.
The final results of the tests were as follows:
The old subscription age saw a negative ROI during the third test, while it remained positive before. This could be attributed to the creatives not catching up at the start, i.e., there was a low CTR and the campaign started to get worse traffic. Such a campaign should have been stopped and restarted in two days, which usually solves the issue.
Campaigns with the subscription ages of 0–3, 16–30, and 30–60 saw a positive ROI. Despite lowering the bids, all the campaigns generated more leads. This was because of identifying and adding new sources to the group with increased custom bids. That’s why we encourage you to check which zone to ban daily, as well as where to raise bids.
The creatives performed as follows:
For some campaigns, removing some creatives and adding new ones was possible. While this would improve the final result, it wasn’t obligatory, which is why this step was skipped.
Final Results
Here are the stats from the tracker:
This is what our dashboard showed:
These were stats from Golden Goose:
As you can see, there was a bit of a data discrepancy (about 1%). This could be due to the following factors.
- Dashboards with three different time zones: Binom, RollerAds, and Golden Goose. Thus, the number of clicks and leads differed.
- Tracker anchoring to a specific click_id: The tracker would record the lead when the click was made; RollerAds and GG, on the other hand, record the arrival of the lead, which might happen the day after.
Quick Tips
- Follow the three optimization rules, highlighted after the first test.
- Increase the bid amount for good zones.
- Refrain from testing three offers simultaneously; proper A/B testing involves a 50/50 traffic split. Otherwise, there is a risk of getting the wrong results and subsequent flawed conclusions.
- One parameter at a time. It’s hard to narrow down the exact effect of each parameter if the changes are made in bulk. Don’t exclude creatives and zones simultaneously if there are negative results — pick one. Usually, it’s better to focus on analyzing zones first. The same principle applies to adding new creatives and raising bids — one thing at a time.
- Consult your ad network manager or RollerAds support team to decide on the optimal amount of a custom bid.
Conclusion
Huge thanks to Golden Goose, and the author of the case study in particular, for such an elaborate and thorough guide. Make sure to register on Golden Goose too, get the offers, and use our traffic to boost your profit.
Hopefully, this case study explains how to promote sweepstakes, specifically mobile offers. One has to keep on analyzing the results, implement changes one by one, and be ready to respond quickly to unforeseen events.
Always conduct A/B testing and consult your account manager to ensure you get the best results quickly. And remember, sometimes your creatives are awesome already, but the zones or bidding might be suboptimal, as we’ve seen in the case study.
