In this episode, our two Mobile Growth and Pancakes Podcast hosts – Esther Shatz, Storemaven’s VP Consultancy and Product Marketing, and Jonathan Fishman, VP Growth and Marketing, crossed to discuss the new Google Play Console capabilities announced in May 2022: From the boost to Custom Store Listings, Store Listing Experiments, and LiveOps, what it exactly means for developers and marketers and how to start using and consider them in your mobile growth strategy.
Check out all the other episodes of Mobile Growth & Pancakes here.
To connect with Esther and Jonathan:
Listen to the full episode:
You can also listen on:
And please – rate & review Mobile Growth & Pancakes:
“This announcement is about more powerful tools falling into our hands that we can start working with to continue the revolution of making the app store a world that’s optimizable, that you can see how scalable it is, and lead to stronger performance metrics.”
Esther Shatz
Key takeaways:
- Custom Store Listing is a tool that helps you with localizations and narrows down your audiences within different regions. At their annual conference, Google I/O 2022, the company announced you can have up to fifty Custom Store Listings.
- Though LiveOps existed before in the Google Play Store, Google announced that they are expanding it. Also, LiveOps is taking the form of something similar to in-app events.
- The pros of Google Store Listings experiments include more transparency, control, and experiment opportunities. But the danger is that it is easy to make mistakes in your setup based on your desired outcome rather than the validity of the data you get. A test is only as good as the data you put into it. If you don’t understand data science or statistics, you may set up an invalid experiment that will damage your performance metrics and affect your organic traffic.
- Regarding privacy, Google added into store listing pages an area that explains what data each app is collecting, what is sharing, and how it is being used. Just make sure that you fill in your privacy form for Google by July 20th.
Transcript:
Esther: Hey, welcome to Mobile Growth and Pancakes. You guys know when TV shows do those joint episodes, crossover episodes like the Simpsons and Family Guy? That’s basically this episode because I am joined by alternative host Jonathan Fishman. Hello, Jonathan.
Jonathan: Hello, hello. Can I be the Simpsons?
Esther: Yes, you can be the Simpsons. I’ll allow it. All right, Jonathan, we’ve got some big changes that have happened in the industry that I think we should discuss. Do you want to kick us off?
Jonathan: Yes. Last week Google did its annual conference Google I/O 2022. They basically announced a few really exciting features that they’re adding to the Google Play ecosystem. I would say one, in a nutshell, they basically called and raised everything that Apple did with iOS 15.
Esther: I like the way you phrased it. I think let’s start off with custom store listings, which they’ve always existed. They’ve been a tool to help you with localizations and really narrow down your audiences within different regions. Now, they’ve announced that you can have up to 50 custom store listings, the same idea as custom product pages, where it’s URL-based, so you can put them with whatever flow and wherever you want to go which is incredible. Obviously the more tailored we can create messaging the better obviously. What’s your take, Jonathan?
Jonathan: I think that first of all, it’s really funny that they did the 50. They thought about–
Esther: It’s a nicer number though than 35. 35 sounds so small now compared to 50.
Jonathan: They knew that they were doing 35. They say we’ll do 50. It has no meaning because you don’t need 50 custom store listings. Probably you can get a lot of value just from starting to implement it with a handful or a dozen. I would say that it’s different because actually, Google owns Google ads. Apple owns Apple search ads, and it’s integrated with CPPs.
Esther: Yes, but you don’t get to scale on search ads the way you can scale with Google ads. That’s the bottom line, is Google ad network, you can spend your whole budget there and still get traffic. Apple search ads, there’s a limit. There’s a limit to how much growth is going to come from there.
Jonathan: Exactly. I think that with Google ads, and of course they’ll integrate custom store listing with Google ads even in the presentation that they did on Google IO, they showed a screenshot it says, “Target by Google ad campaign.” It’s still not out there for developers. It’s going to roll out soon probably. What really starts to pick my interest about that is how they’re going to integrate it with UAC which is a really important source for Android.
Esther: I think the big potential here is since 2020 basically we haven’t been able to analyze pure organics in Google because UAC mixes in with explore and mixes in with the search. If you can create a custom store listing, you’re talking about the integrations with UAC, if you can create a specific page that targets search users, all of a sudden there might be the option to separate out the organic search and the paid search again, which is incredibly powerful for developers. Everybody’s been missing that feature for a long time.
Jonathan: For sure. How do you think the developers or marketers should start thinking about their first custom store listing? It’s the same methodology as they have been using on the iOS side with CPPs?
Esther: Look, overall the goal is the same, which is to tailor your messaging and create really strong funnels. First of all, I think on both sides something that maybe developers aren’t doing as much as they could is to also look at CPPs and custom store listings as a really strong way to measure your campaign performance, especially on the Apple side CPPs. Because you have a URL that you’re actually able to see analytics for, you can start using that in your ad campaigns regardless of the creatives that you’re using almost just to be able to bring back the measurement side and understand how this campaign is performing further down.
For Google, as they are working on their privacy changes and as that’s going into place, it’s also something important to keep in mind. From the strategy side, I think you do want to take a similar approach. Just make sure you’re thinking about your audiences as individuals. We know that Google audiences aren’t the same as Apple audiences. Doesn’t matter what the flow is that brings them in. They’re different sources. They’re different people, there are different demographics that build them up. Start really playing around, and see what works. Do a small test to start. Create a couple of CPPs, and a couple of custom store listings. This is going to confuse me for a long time. The CPP and the CSL, let’s call them.
Jonathan: CLS.
Esther: CSL, custom store listing.
Jonathan: CSL.
Esther: CSL.
Jonathan: It probably means–
Esther: Like DSL but CSL.
Jonathan: It probably means something horrible. It’s like maybe a medical abbreviation.
Esther: We should look this up-
Jonathan: Yes.
Esther: -before we start using it. I think really start to play around. The example that Google gave was if you’re a recipe app and it’s Thanksgiving time in America, create one that’s turkey-based, and it’s different holidays somewhere else, but I think even looking at genders is always a good example. Especially if you’re a dating app, you want to target a different flow for males and females, depending on what people are looking for. For gamers, you want to look at which type of gamer am I going after.
If I’m an IP app, I want to create a custom store listing that’s more centered around the IP, and then I want to create another one that’s maybe more centered on the genre for users that I’m trying to pull in from competitive apps that don’t share the IP. Start off small with these really concrete demographic differences, and then start to iterate and grow from there and see what we can come up with.
Jonathan: Awesome. This is a custom store listing, a really exciting addition to the Google Play site. The second thing that they announced, it’s LiveOps. LiveOps existed in the Google Play store. It was out there for quite a lot of game developers. They really announced that they’re expanding it. It’s taking the form of something extremely similar to in-app events.
Esther: I think first of all, if you have more than 100,000 monthly active users, apply for the beta for sure. Don’t wait if you can get in. Honestly, I don’t really think there are any negatives to LiveOps. It’s just a powerful opportunity. According to Google, it’s been leading to a 5% increase in day 28 monthly active users, a 4% increase in revenue. There’s really no downside. It’s definitely worth trying if you can.
Jonathan: The way this is going to work for those of you who are less familiar with in-app events on the iOS side, you can basically list an event that’s happening within the app. It could be a competition, could be new content that you’re featuring, or could be some special content for a limited time. It could also be offered, which is a new type of event that Google Play announced there. Which is any type of deal, any type of discount that you’re offering for a limited time, which is great. One really great thing here is that we had some time to analyze the impact of in-app events on the iOS side.
Because it’s been out there and adopted by, I would say, around a third to half of all top 100 developers in each category. That’s one analysis that we did recently. Basically the impact on both searches, because these events get you indexed, and at the Google Play side, for sure they’re going to index all the text elements within your in-app events, the title, the description, so they come up for specific keywords on search, and it gets featured a lot. On the Google Play side, there’s an event tab that is completely dedicated to events.
Esther: Google straight out said it, if you’re using LiveOps, there’s a ton of different featuring placements that you’re able to get just by having these ops, you can show up in pretty much every flow on the play store by using the LiveOps events which are incredibly powerful. I think one thing that is worth thinking about, as well as Google, is starting to really roll out more with deep linking and understanding of how to work with that.
For LiveOps specifically, you put a deep link into a specific page within your app. Instead of sending users to a default flow, you can actually customize the LiveOps flow from seeing that event and taking users into where it’s actually relevant, which just brings us closer to the dream of the web which is tailored flows, customized perusers, and what brings them in.
Jonathan: Amazing. This is out and it’s the second phase of the beta. It’s bigger, and every large developer can apply as long as you have a great product because they’re going to review the product itself. As you said, 100,000 MAUs, that’s the-
Esther: Global MAUs, 100,000.
Jonathan: Yes. Go ahead and apply. You can Google it and find the application form. Another thing that they announced was, and this has been rolling out for a couple of weeks quietly, new changes to Google store listing experiments and the way that you set up a test.
Esther: I think there are some pros and cons to what’s happening here. First of all, the biggest pro for me as there’s more transparency now in experiments. Until now, there’s been this metric called scale installs, which it’s an imaginary metric. It’s not something you should ever really take seriously, because what happened until now is basically you didn’t know how many people saw each page. You didn’t have the metrics to understand how many people installed.
What you saw was this projection of if I send 10% of my users into an experimental page, then scaled installs would be whatever rate that 10% of users were installing. We multiply that by 10 to see what it would be with 100%. The reason this metric doesn’t work and that process doesn’t work, we’ve seen it, you’ve probably seen it in your UA campaigns as well, the second you scale an audience, your numbers change.
Whatever a 10% of your audience was showing would never be the same as if 100% of your audience were seeing it. If you’re using this metric of scaled installs, it’s an imaginary metric. It doesn’t mean anything in real life. Now, Google is actually giving you the numbers, how many samples are coming in, and how many installs you’re receiving, which means there’s room to work with here. If we’re using the right data science tools, we can create really smart experiments and come up with numbers that we can use and take insights with that we can rely on.
Jonathan: I think, first of all, I have to say that for me it’s surprising that Google went this way because in statistics, in the world of A/B testing, there’s always a war or an argument between the Bayesian camp of using Bayesian statistics and using frequentist statistics that is basically the more traditional way to A/B-test. Google has in a lot of its products, basically, they came forward and–
Esther: They were the innovators of Bayesian. They were like the–
Jonathan: I think that in Google Optimize, which is their flagship optimization product for the web, they only do Bayesian statistics now. Basically, they went out and had an argument that using frequentist methods, which is what we know, the confidence level and the minimum detectable effect, and calculating a sample size beforehand, and that methodology, and then running a T-test, of course, after you collect enough data. They said it’s just not really a good method for A/B-testing digital experiences, for many different reasons.
Here, they basically decided that in Google experiments, you can choose your desired confidence level, how accurate you want the test to be, and you can configure the minimum detectable effect, which is in simple words, basically just how sensitive you want the test to be, that if there’s a difference, how small the difference between the performance of each variation should be, that the test should detect.
It’s basically another configuration that controls how accurate or sensitive the test. The higher the minimum detectable effect the less sensitive the test and the less sample size you would need. Then basically what they did on the user experience of using the Google Play experiments dashboard is that you can basically control how big the sample that you would need to conclude the test.
Esther: I think this is where one of the dangers really comes in, because at the end of the day, your test is only as good as the data that you put into it, which is, we’ve seen it before there are some people who will say, “Look, I need to finish this test in a day. I’m sending 2,000 users today and I’m taking the results,” and they end up really damaging their performance metrics because it wasn’t a representative sample. One day does not tell you what your overall audience is like. A user who comes in on Sunday does not have the same need from a user who’s coming in on Wednesday midweek.
From this side, I think this is where the danger comes in. If you’re not really somebody who understands the data science side and the statistical side, you run the risk of setting up an experiment that’s not going to be valid or true to form once you go live. Obviously, you’re affecting your organic traffic as the most valuable traffic you have. If you’re creating a change that you shouldn’t have created, it’s going to be incredibly damaging. We don’t want that to happen. I think in general, the idea of adding more transparency and control is amazing.
That’s what we want to see happening because eventually, it will lead to better tests and better decisions. Right now, if you go into the platform, I think it’s very easy to make mistakes in your setup based on your desired outcome rather than the validity of the data that’s coming in. That’s where I think developers need to be really cautious and make sure that they’re working with strong data science and people who understand testing to be able to actually get results that we can rely on.
Jonathan: It’s tempting also for a marketer to get these tools. Basically, it’s just really tempting to configure everything to lead to really low sample size, to a test that would be concluded in a couple of days, not even a week. Then you can go into the rhythm of just running a lot of meaningless tests to satisfy some, I don’t know, some wrong KPI that some personal set up. I call it a fake testing culture when they’re just running a lot of tests without– They’re trying to optimize the number of tests that are running and not their quality.
Esther: I like that. That’s true.
Jonathan: It’s a really tempting capability to put in the hands of marketers that are working in that kind of environment.
Esther: You know that phrase, “Work smarter, not harder,” my life motto?
Jonathan: Yes.
Esther: I feel like test smarter, not more. Lower the test frequency, as long as you’re creating tests that really are meaningful, that you can learn from.
Jonathan: Exactly. We just talked this morning about this. We have our data scientist working across us right now.
Esther: We bring him in here, but it’s difficult for English speakers to understand what he’s saying. He’s too smart for us.
Jonathan: Yes. I think, and it’s really early days, here that given this added flexibility and using data science expertise, you can really have the result of having way more accurate tests on Google experience if they’re set up correctly.
Esther: Yes, agreed. Let’s talk about privacy for a minute because that’s what everyone in this world talks about. There’s nothing new here. We know the Privacy Sandbox side Google announced that they’re going through similar changes to Apple, but they’re doing it in a much more slow, data-driven, experiment-driven way. There’s still nothing major that needs to be focused on this second. Nothing is going to change overnight.
One thing though, that is being added that developers should make sure they’re aware of is similar to an Apple where you have the data section on your store listing, on your products page in Apple that explains what data’s collected and how it’s used. Google is also adding this into store listing pages, an area that explains what data each app is collecting, what is sharing, and how it’s being used.
Look, we know this is not something users are super sensitive to. We’ve run quite a few tests on Apple of changing certain levels, of understanding how much this bothers them. The answer is it doesn’t bother them. Most users don’t even look at it. It’s not something you should obsess over, but you need to make sure that you fill in your privacy form for Google by July 20th. Otherwise, you will lose your store listing presence until you get it done. Do fill out that form. My advice is, just be really honest about how you’re using the data, and don’t worry too much about optimizing it for what you’re afraid users might see. Just say how you’re using it, fill out the form, and it should be fine.
Jonathan: For sure. Cool. We need to wrap up in a couple of minutes, but another thing that caught my attention when we talked this morning was thinking about it as a revolution. What do you think about it? Because Apple started a revolution of basically, we call it splitting the app stores in two and really tailoring the experience for paid audiences or organic audiences, and really breaking it down to really different segments of users. What do you think about this announcement and how it relates to that revolution?
Esther: Sure. Apple really was a revolution. It was so much a revolution, and this is true, Jonathan can validate that, my water broke the day Apple announced-
Jonathan: That’s true.
Esther: -the changes to iOS 15, I went into early labor because it was such a revolution. Look, this announcement is not a new revolution. It’s not something that’s putting anyone in a panic. It’s the next step of the revolution. It’s taking the concepts from iOS 15 and leveling them up a bit. Doing something that, first of all, we have a lot of data to really analyze and understand a bit more of a measured approach and learning, and we as developers and as the growing community has a lot of learnings to take with us from iOS 15 that we can now implement here.
I would just say we’re in that stabilization period of the revolution where things are still revoluting. What would be the word there? This shouldn’t be causing panic. It’s honestly, for me, it’s just more excitement. It’s just more powerful tools falling into our hands that we can start working with to continue the revolution of making the app stores a world that’s really optimizable and that you can start to really see how this is scalable and will lead to stronger performance metrics across the board.
Jonathan: What’s really nice about it for teams that have been really leveraging and leaning in to everything that Apple announced on iOS 15 and started releasing shortly after, is that the methodologies are similar. It’s a different audience, it’s a different ecosystem, but the methodologies are similar. For those teams that have these methodologies in place and are really ramping them up, they’re going to enjoy that value really soon.
Because on the iOS side, I saw studies and analyses for teams that saw millions of dollars of value generated by leveraging in-app events correctly. I saw uplifts on custom product pages that are really significant, both on search ads and on different networks such as ironSource. I’m really confident that there are millions of dollars of value to make here by leaning into these changes and adopting them.
Esther: Jonathan, what’s your favorite flavor Pancake.
Jonathan: We don’t have any guests to ask beside us.
Esther: I don’t know. I don’t know this about you.
Jonathan: Did you ever say–
Esther: We’ve known each other for five years, we’ve never spoken about pancakes.
Jonathan: Did you ever say what was your favorite flavor?
Esther: I don’t think I have, but I could.
Jonathan: Mine is weird.
Esther: Go, tell me.
Jonathan: I don’t want to say.
Esther: You have to say it.
Jonathan: Its like just lemon
Esther: Lemon Pancakes?
Jonathan: No, the pancakes are normal. You just take a half a lemon, squeeze it on top of the–
Esther: I don’t want to hear this. It’s an abomination.
Jonathan: Listen. No, it’s very good.
Esther: I’m sorry. This is offensive.
Jonathan: You just squeeze half a lemon on top of the pancake and put a sizable chunk of butter on the pancake. It’s really, really good.
Esther: Wow, I’m repulsed.
Jonathan: I didn’t want to say.
Esther: I don’t know what to say. You shouldn’t have said. This was a terrible thing I’ve learned about you.
Jonathan: What do you like?
Esther: I have two favorite types of pancakes. They’re similar. I like chocolate and fruit on my pancakes. One is a Nutella banana. That is a classic combination. The other is a white chocolate blueberry. Also very, very good. Much better than lemon. Don’t do that. Don’t tell people this.
Jonathan: Lemon is a good thing. It grows on you.
Esther: I disagree.
Jonathan: It’s what happens when you start–
Esther: This is why you guys have to optimize your custom store listings because you have freaks like Jonathan who add lemon on his pancakes. My God, that requires a custom. That’s an audience to look at. I don’t know what to make of that.
Jonathan: I’m a one-man audience.
[chuckling]
Cool. We need to wrap up. This was a really short episode that we hope that you enjoy just on your way to work, commuting, whatever you’re doing, doing dishes, or whatever people do when they listen to podcasts. If it got you up to speed with everything that Google announced, so you can share it with your team and be the smartest person in your company.
Esther: Happy optimizing
Jonathan: See you.
Esther: That was Mobile Growth & Pancakes. To find out more about Storemaven and how we can improve app store performance, visit storemaven.com. Then make sure to search for Mobile Growth & Pancakes in Apple Podcasts. Spotify, Google Podcasts, or anywhere else podcasts are found, and click subscribe so you don’t miss any future episodes. On behalf of the team here at Storemaven, thanks for listening.
