If you’ve been with us since the beginning of this Ultimate App Store Test Series, you’ll know that all great app store tests begin with a solid hypothesis that is transformed into effective creatives. At that point, targeted traffic is sent to the specific creatives you’ve made to trial hypotheses for accuracy.
Check all other articles in our Ultimate App Store Test Series:
Part 1: Building Hypotheses
Part 2: Creative Design
Part 3: Driving Traffic
The only thing we haven’t covered yet is how to analyze the results of your tests. But don’t worry, that’s exactly what we’re covering in this article, the final stretch. Combining these four pieces will get you started on the route to App Store Testing supremacy which you must remember is an endless circle that keeps spinning.
Analyzing test results is the last step in the app store testing process — and arguably the most important. If you can’t decipher what your tests are telling you, you’ll never be able to make improvements to your app store pages and propel your company forward. But, as you can learn from the image above, this is a never-ending cycle. Only by implementing the results you got into your next hypotheses and moving this process constantly, will you start to see real improvement.
Keep reading to learn what good analysis looks like, the types of data you should be able to pull from your tests, and more. Let’s dive in!
Conversions Aren’t Everything
Newbie ASO managers tend to think that conversions are the be all and end all. On the surface it makes sense, if you can convert more product page visitors into users of your app, your company will grow, right? Actually, that’s not entirely true…
Let’s pretend that your free app gets 1,000 downloads a month. Of those 1,000 downloads, 100 of them are high-quality, meaning that 10% of your user base purchase a monthly subscription of $4.99, leading to $499 in monthly profit.
Now, what would happen if you boosted quality downloads by a measly 2% to 12%, but your conversions shrank by 10% in the process? Would that be good for your business?
Let’s run the numbers: since your conversion rate has dropped by 10%, your free app now receives 900 downloads a month. Of those 900 downloads, 108 make the subscription purchase of $4.99, which translates to $538.9 a month.
As you can see, company sales went up — even though conversions went down. This proves that a high conversion rate isn’t everything.
What Does a Good Analysis Look Like?
“Okay” you’re thinking, “if boosting conversions isn’t my only goal, what else?” Great question! The analysis phase of the app store testing process is all about understanding why you received the results you did. That way you can replicate and improve them in future tests.
Which assets worked well and which didn’t? What messaging enticed app store users to engage with your product page? Are there specific elements of your page that cause users to drop off at high rates? These are the kinds of questions you need answers to.
A great analysis is also tied to business questions, and answering which creatives converted the best for each audience you were testing. For example, discovering that creative X converted the best for ad Y and creative Z converted the best for audience type 1 etc.
When you have these insights, you can use them to form new hypotheses to test, and create a long-term strategy that fosters consistent growth in a constantly changing industry.
Four Types of App Store Test Data
There are countless data points you could track and analyze. But the most important ones all fall into one of four categories, which we’ll break down in a minute.
But first, it’s absolutely essential that you understand the two kinds of users that will visit your product page…
App Store Users
Every Apple App and Google Play store user can be segmented into one of two segments: decisive users, i.e. Decisives, and explorative users, i.e. Explorers.
- Decisive users: Users who only view the first impression frame (the area of your app product page that can be viewed without scrolling) before deciding to install or not. These kinds of users make decisions quickly — usually in three to six seconds — and account for roughly 60-70% of all app store users.
- Explorative users: Users who take their time to explore app store product pages, sift through screenshots, watch videos, read reviews, etc. before installing. These kinds of users typically spend six to twelve seconds on a product page and make up about 30-40% of all app store users. Explorers are often higher-quality users.
Once you understand how your users interact with your app store product pages, you can use the information gleaned from your tests to optimize them more efficiently.
App Store Engagement Data
As we mentioned, when it comes to app store engagement data, there are four categories you need to pay attention to.
Keep these metrics in mind as you analyze your test results:
1. Screenshot Engagement Metrics
Your screenshot gallery is arguably your most important asset in terms of conversion. This is because most Apple App Store and Google Play store users rely heavily on screenshots when deciding to install apps and games. In fact, screenshot optimization can boost CVR by 28%.
In both app stores, nearly 100% of users view one to three screenshots. But very few users take the time to look at every screenshot in the gallery.
Because of this, we suggest carefully analyzing screenshot engagement metrics to learn how many of them your audience typically views and which ones drive installs. Then, try frontloading your gallery with your best images so that they’re always seen by your users.
2. Video Engagement Metrics:
The ASO community is split on the importance of app store videos. Some in the industry believe that videos are the only reliable way to boost conversion rates, while others claim that videos do nothing but lower conversions for their apps. Who’s right?
The answer is.. it depends.
The truth is, the specific audience you target, the country they’re located in, and the quality of your videos have a direct impact on the success of your videos. This is why we strongly suggest you view your video engagement metrics.
Are your users watching your videos all the way through and downloading your app afterwards? If so, look to optimize your videos. If a high percentage of users watch the video, lose interest and drop from the page, consider removing it from your app product pages and including more screenshots instead.
3. Review Engagement Metrics:
Your app’s reviews and ratings help build trust with new users. They’ll look at what other people have to say about your app and think, “hey, if they liked it, maybe I will too.”
It’s important to realize that reviews in the Apple App and Google Play stores operate differently. Here’s what you need to know:
- Apple App Store: Apple doesn’t display app download metrics to users, so the number of reviews helps them determine if an app is popular or not. Additionally, reviews in the Apple App Store are located at the bottom of the page, making it less likely that your users will actually read them.
- Google Play Store: Google gives reviews their own widget. The Play store also highlights the top and most critical reviews for easy viewing. Because of this, Google users tend to read reviews more regularly than their iOS counterparts.
Take the time to understand how your users interact with your app’s reviews. Then do what you can to optimize this experience for them.
4. Page Engagement Metrics:
Finally, we suggest looking at your app’s general page engagement metrics.
How do users interact with your page? Do they focus on the first impression frame or do they like to scroll? Do they care more about screenshots or reviews? How many of them actually read your app descriptions and what do they do after reading?
If you can determine how your target users interact with your app product page, which elements entice them to install, and which make them click away, you’ll be able to use your analytics to boost meaningful downloads and revenue.
All of these engagement metrics will tell you about your users. Just remember, it’s not enough to know which app product page performs best. You also need to know why. That way you can build a reliable long-term strategy.
How to Analyze App Store Test Results
To properly analyze your app store test results, you need to uncover two key insights: how Decisives interact with your product page, and which product page elements cause Explorers to download and/or drop.
Fortunately, these are both easy to figure out with Storemaven!
Our team will create an exact replica of your app store page in a sandbox environment that will allow you to easily track and analyze user behavior. You can then switch out icons, create new screenshots, adjust descriptions, and more to see what your audience responds best to. And you won’t even have to worry about missing out on installs. Users will automatically be redirected to your real app store page if they decide to download.
Once you’re set up with Storemaven, take a look at your Decisive users and ask yourself questions like, “How many dropped after seeing the first impression?” and “How many people watched my app’s video?”
You’ll also want to investigate your Explorative users to determine their behavior. Ask yourself questions like, “Did they explore my app product page because the first impression was enticing or unclear?” and “Do they consistently install or drop after a certain screenshot?”
These kinds of questions will help you get to the root of who your users are, what they want to see in your app product page, and why.
Moving Forward
Once you’ve run tests and analyzed the results, you need to do something with the information you’ve uncovered. If your hypothesis was correct, update your app product page. Then adjust your understanding of your audience to suit. If it was incorrect, leave your product page alone, but develop a new hypothesis to test next.
Testing should always lead to fresh insights and hypotheses. Think of it this way: every test is like a brick in a building. They each build on top of each other until, eventually, you’ve created something substantial — an app that consistently converts high-quality users.
Run Better App Store Tests Today
App store testing is a never ending cycle. When you do it right, you’ll generate more hypotheses to test, which will generate even more hypotheses, so on and so forth. Plus, user preferences are always changing, forcing you to adjust your strategy on a regular basis.
The good news is, you now know exactly how to run effective app store tests, from planning hypotheses to making creatives to driving traffic and analysing results.
That’s it! Well done on completing the Ultimate App Store Test series!
Looking for help during the testing process? Use Storemaven’s App Store Analytics Platform! Our products and services are designed to give you more accurate test results in less time and for less money. Request a free demo today to learn more about our offerings.
