With Google Play Experiments, it’s possible to A/B test your app store creative assets in order to identify the most effective graphics and localized text for your Google Play store listing. A/B testing is a key component of App Store Optimization (ASO) that leading mobile app companies use to boost conversion rates (CVR) and decrease their cost per install (CPI).
We recently came across an interesting A/B test that CouchSurfing—a global travel brand—ran with Google Experiments. Based on our data, top app publishers in the travel industry test their app store marketing assets twice a month on both the iOS App Store and Google Play. Given the importance these companies place on testing, there are numerous insights and learnings you can pull from the tests they run to better define and develop your own ASO test strategy.
In this month’s Google Experiments ASO Teardown, we will explore the Screenshot tests run by CouchSurfing and share our data on the world’s biggest travel apps.
The Background
CouchSurfing Travel App is used by over 12 million travelers around the world to connect and meet up with locals, find accommodations, discover the best local activities, and receive travel advice. Considering the industry and what the app does, we can assume they are targeting avid, mostly solo, travelers who value connecting with locals and fully exploring the cities they visit.
As part of their ASO strategy, they conducted an experiment to optimize the first Screenshot in their Google Play store. Our data shows that among leading travel apps, optimizing the first Screenshot has a potential CVR lift of 12-32%. Hence, there is a lot to gain by testing and optimizing this marketing asset.
The CouchSurfing Screenshot experiment involved testing Lifestyle-Oriented Screenshots that evoked different travel personas. Lifestyle-Oriented Screenshots are feeling-oriented images or visuals with real-world elements that convey a distinct style of living. They are similar to banner ads and can often do a better job conveying the value of your app.
Using Lifestyle-Oriented Screenshots (instead of Feature-Oriented Screenshots) signifies that CouchSurfing wanted to appeal more to visitors’ feelings and emotions. And for good reason. We’ve found that Lifestyle-Oriented Screenshots win 64% of the time for companies in the travel industry, and when they do win, they show a higher potential CVR lift.
Image strategy will vary depending on the app and industry, but for CouchSurfing, it was clearly important to test which travel persona would appeal to potential users and drive the most installs.
You will notice, CouchSurfing kept the captions consistent across all Screenshots. By keeping captions uniform, it ensures that the image itself is the only creative asset being tested and optimized. This is essential for identifying which specific element is boosting CVR.
The Test
Based on the varying Screenshots CouchSurfing used, the rationale behind their test is clear. They wanted to understand which combination of lifestyle imagery and messaging would be most effective in increasing conversion rates.
Here are the variations they tested…
Screenshot Control (Current Version in the live store*)
Hypothesis: This variation tests the idea of creating a community while traveling. It shows an image of a group selfie that will appeal to social travelers who view traveling as a way to meet people around the world, share experiences, and ultimately form lasting relationships.
Screenshot Variation 1
Hypothesis: In this variation, the Screenshot shows a scenic destination image to portray what possible adventures are accessible with the app. This aims to appeal to adventurous travelers who are looking to explore nature’s gems.
Screenshot Variation 2
Hypothesis: In this variation, the Screenshot shows an older person in a calm environment. This aims to attract mature travelers who enjoy more relaxed activities like watching the sunset.
Screenshot Variation 3
Hypothesis: In this variation, the Screenshot showcases an urban destination with local residents. This aims to target travelers who value integrating into new cultures and exploring cities like a local.
So, which persona will have the widest appeal?
The Results
Google creates a split A/B testing environment for paid and organic traffic and calculates the results with only 90% statistical confidence. When the test ends, you can view how each variation performed and identify which one received more installs.
At the conclusion of this particular test, Variation 1: The Adventurous Traveler was applied as the “Current Version” in CouchSurfing’s Google Play store listing, which means it began to receive 100% of the traffic*. This suggests that Variation 1 performed better than all other variations.
What does this mean for you?
In some cases, this could be enough information to push a variation to the live store. However, there are additional questions one should ask when building an ASO strategy. For example:
- Variation 1 converted the best, but how can I be sure these installs came from an audience that monetizes the best? How does it perform across different audiences?
- What would the results be globally? Would the results change if tested in different countries?
- How do I apply these learnings to other marketing assets (e.g., App Preview Video)? Do people even play our Video?
- Should I put this winning Google Play variation in the iOS App Store?
Next Steps
Just as you optimize other aspects of your marketing funnel based on user behavior data, you should do the same with your mobile app. In order to grow your app store and boost conversion rates, it’s crucial to understand how users interact with your page. Without data, it’s extremely difficult to optimize or know how you can improve.
For example, based on our analyses of over 500M users, we’ve discovered that only 18% of app store visitors view the Screenshot Gallery on Google Play (for top tier apps, it’s 31%). This is because in Google Play, the Gallery appears below the First Impression Frame (i.e., below the fold). Therefore, when you design and test Screenshots for Google Play, you need to be aware of the fact that people who scroll to the Gallery have already been exposed to elements in the First Impression Frame. For that reason, the Screenshots should offer additional value and not include repetitive messaging. To create maximum impact, you might also want to test assets within the First Impression Frame in addition to Screenshots. As you can see, user data and insights are critical for ASO strategizing.
Our A/B testing platform allows you to test app store creatives in a sandbox environment before uploading winning creatives into the real store. Our platform and services have helped the world’s leading apps such as Expedia, Uber, and Google track and analyze user behavior in their app stores.
If you have questions or want to start testing with us, feel free to contact us at [email protected].
Be sure to check back next month for our next Google Experiments ASO teardown!
*Disclaimer: Since this post was written, CouchSurfing has updated their Google Play store page. Our analysis is based on their previous app store creative assets.
