In this episode of Mobile Growth & Pancakes, Esther Shatz is joined by Chang Chen, the Head of Growth & Marketing at Otter.ai. Chang discusses unconventional ways to drive growth, such as advertising without targeting and user personalization.
Check out all the other episodes of Mobile Growth & Pancakes here.
Connect with Chang and Otter.ai here:
Timestamps
01:05 – Chang’s and Otter.ai’s Introduction
01:55 – Optimizing for market penetration and user education
03:00 – Measuring user education
04:20 – Zero targeting
04:55 – Otter.ai’s future campaign targeting
05:30 – Chang’s experience with zero targeting
06:30 – Surprising user groups
08:00 – Implementation of personalization at Otter.ai
09:45 – Personalization demographics
12:10 – Measuring personalization success
13:45 – Personalization challenges
15:15 – The importance of qualitative research
17:10 – Difference between qualitative and qualitative research
18:40 – Quickfire questions
You can listen to the full episode here:
bCast player:Or listen on:
Apple Podcasts
Spotify
Google Podcasts
“If we were just look at lookalike audience for targeting, we would be limiting our growth”
Chang Chen
Key takeaways
- Otter.ai is an AI powered application that helps people take meeting notes. It captures every detail, so you stay engaged in your meetings
- According to Cheng, remote work during the pandemic spiked user acquisition numbers for Otter.ai
- AI is relatively a fresh idea for the users, and Otter.ai is constantly working on educating the users on how to attain maximum benefits from AI
- To evaluate user education, Otter.ai adapts to a qualitative approach. They talk to the users to see how they’re viewing the app to note their opinions, locate their problems, and evaluate if targeted pain points are being catered to
- For feedback, they take user behavior data through surveys and match it with user engagement data
- Chang plans both targeted and broad strategy campaigns for Otter.ai’s paid spend
- Otter.ai is not the first place Chang has tried zero targeting. She has also used this strategy in previous roles and saw really strong growth from it
- Combining “zero target” advertising with qualitative research will surprise you as user segments you were not thinking about at all could find value in your product. You will find users that are very creative and will find creative ways to use your app
- Chang initially thought only people who do a lot of meetings would use Otter.ai, but now people are using it for things like voice memos, presentations, podcasts, etc.
- Personalization is done by designing different onboarding flows for each target segment. For example, while targeting people from a certain state, we will have an icon picture for that state in the ad, and we will also have the same picture following in our onboarding flow as well the questions and emails
- Otter.ai’s current record for different split tests is: 100. They had 100 different versions of their onboarding flow running simultaneously.
- If Chang could take Otter.ai growth back to square one, she would experiment more with qualitative data and have transparency with the customers by talking to them and understanding them
- Qualitative data is essential for directional growth as quantitative data only answers the ‘what’ question’ For example, it tells us that certain people drop off or prefer one feature over the other. But they don’t know “why.” This is where qualitative data comes in – to answer the ‘why.’ You have to talk to the users and understand their workflow, to understand what motivates them to start using your product and to come back
- One tip Chang would give other mobile-growther’s is – understand your customers, know how your product is making your customer’s life better’, and solving their problems
- Chang’s favourite mobile growth resource is: grow.com
Full Transcript:
Esther: Thank you so much for joining me today. I know you run growth and marketing at Otter.ai, do you want to introduce yourself a little bit? Tell us a bit about the app and the product and yourself.
Chang: Sure. My name is Chang Cheng, I’m the head of growth and marketing at Otter. Otter.ai will help people to take out meeting notes. Otter automatically helps you to take every detail, every action item for your meetings. That’s so you can be more engaged for your meetings. Now that all of your data is working from home, they’re working remotely, now, we’re actually seeing that a lot more people are starting to use Otter and starting to use Otter to take meetings and start to use Otter to take their meeting notes.
Esther: Amazing. I can tell you how much I hate taking notes during meetings and that pause for the typing where you hear everybody trying to catch up and losing place. Really sounds very, very cool and very necessary. Before we jump right in, we’re going to talk about a couple of strategies that you guys have implemented over the course of this podcast, but first, it would be great to get an understanding of when you talk about growth and how you’re measuring mobile growth, what are the main KPIs that you’re looking at? What are the metrics that you’re really looking to optimize for?
Chang: Right now, we are trying to optimize for penetration into the market, and we’re also trying to educate the user and to get users to start using the app. Because right now taking notes using an AI technology, this actually is something new. Actually, we need to teach users that, “Hey, now that you have a better way to take notes.” Right now, we’re really trying to educate end users and really trying to penetrate into the market, especially now that we’re trying to create something new. The market penetration and market education are the top two things that we are trying to optimize for.
Esther: How do you understand if you’ve been successful in that education? Is there a metric that you’re looking at to say, “Yes, we’ve gotten through to this user, they understand what to use,” or is it just qualitative at this point?
Chang: It’s a combination of both. We do talk to the users and try to see how they are using the app and how they are viewing the app, if there is anything that we can improve for them, and if there are any pain points that we were trying to solve. Therefore, the pain point that we are trying to solve, did we actually successfully solve for them and do we have anything that we can improve? On the other side, we are also looking at a lot of our user behaviour data and trying to see that for each segment for users, do they start to use the app? After their first use, we also send a survey to understand how happy they are.
After their first use, we’ll send a survey to understand how satisfied they are and also try to understand if they’re going to come back. From that perspective, we also continue to see our users’ engagement data as well.
Esther: Got it. Can you give me an idea of the rough scale of the market that we’re looking at right now? How many maybes installs do you guys have approximately, doesn’t need to be an exact number?
Chang: I don’t think I can share the exact number, but we do have a few millions of users. Now that we’re aware, we’re seeing tremendous growth as well.
Esther: Amazing. Makes sense. Super, super in line with technologies that are relevant for today. I guess that brings me to an interesting strategy that I know you guys have been trying, which comes to do with the targeting. Generally, in acquisition, I’d say most commonly, you see a lot of people building look-alike audiences in different areas like that, and you guys tried it a little bit differently. Do you want to tell us a bit about that?
Chang: We did try a look like [unintelligible 00:04:47] extend, but I do think that when you are targeting user lookalikes, you are actually bringing buyers into your targeting. Especially for a hyper growth company, we’re creating a category that we don’t really have any traditional good or don’t have any too many users that they already have the habit. Even with our large user base, we’re still looking at a way larger potential that we can grow into. The users that we acquired right now, they may be attracted by some aspect of our tools, but a lot of them, they may still not know us.
For example, previously, we ran a campaign that attracted a specific occupation and specific location of users. Those segments of users, they showed really high-quality for us, but it might not be the only segment for users, and that will become loyal users for us. If we just use lookalikes, that means that we are just emphasizing on that specific segment. I think for a lot of our product, we actually have a much bigger market that we can go after. If we just use a look-alike, we’re actually limiting ourselves in growth.
Esther: Basically, you’re saying that even though the performance metrics for a look-alike campaign or for a specific audience, you’ll see strong performance metrics, but you’re doing it within a limited body. Meaning, especially for an app like you guys, you’re revolutionary, you’re new, and you don’t know the full potential of your audience, you might be deceived by the fact that you have these strong metrics and actually prevent yourself from finding out who the next strong group of users would be.
Chang: Yes, I totally agree with that. Especially for the hyper growth companies that the early adopters may or may not be the next growth market for you. If you just use lookalikes, that means that you are just finding out more users from the same segment to your existing users, and your existing users may only be a very small fraction of your market.
Esther: Do you think that would change, let’s say, a couple of years down the line when you’re not in the same hyper growth stage and you have maybe a more established audience, would you say at that point, you’d go back to more look-alike or targeted campaigns, or would you continue in this more broad strategy?
Chang: I think we’ll continue to do both, meaning that we will still try to find out the users that we already know and we are trying to get more of those, while I always believe that there will always be new potential markets that we can penetrate into.
Esther: What brought you to deciding to try this? It’s pretty revolutionary, kind of goes against the play book, I think that a lot of marketers are looking at, did you from the start decide to use your targeting or was there a trigger that drove you to experiment with it?
Chang: Actually, this is not the first time that I used this strategy. I have been using this strategy for my last company as well where we did see a really strong growth from there. The more interesting thing is that when you combine the qualitative research, you will actually be able to identify– Sometimes, you’re going to surprise yourself and you’re going to find a user segment that you have not thought about at all. You will find users really creative and they will find creative ways to use your app.
Esther: Do you have an example of a surprising market or a surprising user group that you uncovered in this way?
Chang: Previously, we were thinking that it’s only people who will have a lot of meetings that they will use our app, but we then realized that we have a lot of interesting user cases. People can actually do voice memos using us. They can actually send the voice messages using us. A lot of people, they’re just starting to use Otter for different conferences. We also see a lot of podcasters, they started using us to record their sessions and to transcribe them, to prepare them to transcribe their podcast or to transcribe their podcast sessions into written content so that they share them on their website or where they can share on social media as well.
Esther: That’s completely separate to a meeting, that’s a completely new used case. That’s very cool. Okay, I’m going to shift a little bit into another strategy that has worked well for you guys, which is something I’d say that the industry has definitely been talking about for a while, which is personalization. Tell us a bit about how far you went with personalization for Otter. What did you guys start to implement?
Chang: The examiner gave [unintelligible 00:09:48] that they snatched it for Otter so at my last company, we have been focusing a lot on that and we had each targeted. With each targeting, we actually have a different onboarding flow. For example, when we’re targeting people from certain states, then we will have an icon picture from that state in our ads and we will also have the same picture there following all the way through our onboarding flow as well. All our onboarding flow, onboarding questions and onboarding emails, they will also be personalized based on the targeting. Sometimes, it can be a location targeting. It also can be gender targeting and the ad can be a different use case that targeting as well.
Esther: How do you figure out what kind of– For geography, it’s pretty straightforward, right? You’re looking for landmarks from the location that people are in. How do you decide what kind of personalization elements you’re using for other demographics like gender, like age? How do you know what content to show them? Do you experiment with that or do you have an idea ahead of time?
Chang: We have some initial ideas and we have done a lot of experiments. The highest record, we are running more than 100 different versions of our onboarding flow.
Esther: Wow, all at the same time?
Chang: Yes, they’re all at the same time and for different targeting and some may be testing against each other.
Esther: How far do you go? If I’m a female from Boston aged 30, am I seeing something completely different from a male from Boston aged 30 than a female from California? How many layers deep are you going in the personalization?
Chang: All these things are dynamic enough. We have totally different flow for different use cases and for different states, they will have different graphics and it’s with different gender. Sometimes, we’ll have a different graphic and different copy and sometimes, that we offer the same thing because by previous test, the difference in copied data ends up in a different sign-up rate or word conversion rate. It depends, but I think the majority difference is coming from different used cases.
Esther: It’s clearly a huge effort, 100 different flows and testing and figuring out exactly what works for users. How do you understand if those efforts are justified by the outcome? How are you measuring the success of personalization and if it was worth all the effort that you’ve put into it?
Chang: At the end of day, we want to acquire pay users. We have different flows and targeting different funnels. We have low-targeting users trying to drive users for install to sign up, then we have low trying to drive user phone sign-up to payment. With different flows that we look at, the new flow compared to the control and trying to understand them and when we calculate the catch, then you’re able to see movement on the catch. With the improvement of catch, then you will be calculated how much money that we’re actually saving for the company. Then you’ll have a very clear idea of the ROI.
Esther: Do you ever get to a point when so much personalization in the initial stage and the ads and the onboarding flow, do you get to a point in the product where maybe you haven’t personalized all the way through to the end and users actually get a negative experience having been accustomed to that, or do you make sure that every step of the funnel along the way is personalized, or does it not even matter to them? Once they’ve had that initial positive experience, they don’t need to see it all the way through.
Chang: We found from Day 1 that we didn’t have everything. We started with the very top of the funnel. We didn’t see negative reviews, but we started to think about, “Hey, if the presentation top of the funnel really works, what would happen if we applied that to a deeper funnel?” If we do that for our activation, for our conversion, are we going to see more users? As more users started using the app, are we going to see more users become paying user? It’s that idea that’s driving us to implement more and then to experiment more. It’s that idea that’s really driving us to try to implement personalization all the way through.
Esther: Awesome. If you were to start, whether Otter.ai, whether Mile IQ, wherever you were, if you were to start growing the app again, you were coming back to square one and presenting with this task of you need to create growth, what would you do differently now?
Chang: I think we would do more experiments. We will do more qualitative research as well. Previously, I was really believing in data. I threw in a lot of different ideas, we were trying to run a lot of experiments, but we didn’t talk enough to our customers. We didn’t really have too much qualitative research. If I do it all over again, I will talk to more of our core users and really try to understand them.
I think that that will actually save us a lot of time to run the experiment and that will give us a better understanding of what exactly to test and how we can create a better experience for each segment of users.
Esther: Okay, awesome. Now, just one last question on that. When you say qualitative research, how much do you break down the difference between qualitative and quantitative? Meaning, it sounds like you were saying qualitative helps give you a better idea of experimenting, and maybe quantitative is a way of measuring, but I know you also measure through qualitative sources as well. Where do you put the balance? How do you understand where it gets too subjective or where you’re focusing so much in the numbers that you lose the vision of what you’re supposed to be doing?
Chang: I think the quantitative data will tell us what. We were going to see from data the conversion rate, we’re going to see from the data what are some popular features and how often that users use them, but we can’t really understand why. We know a certain percentage of people will drop off, but we don’t know why. We know that for certain segment users, they will use Feature A more than the Feature B, while for the other segment users, they may use a feature B more. From the data alone, you actually can’t understand why.
You have to talk to user and then to really understand how they’re using the app and why. You really have to talk to user to understand, other than the product, do they use anything else and to really understand their workflow to understand, what’s the motivation for them to start to use your product? What’s the motivation for them to come back? I think we also need to understand, before they use the product, what they are using and really understand the pain point that we are solving for them. In addition to that, we also want to understand what’s the user’s workflow to see how we can be more integrated as well.
Esther: Makes sense. Now, for the quick-fire round, for questions that we ask everyone here on the podcast, first, if you could give just one tip to somebody who’s entering the world of mobile growth marketing or aspiring to be a mobile growth marketer, what would that one tip be?
Chang: Really understand the user and really understand how your product is solving problems for them, and how your product is making your customers’ life better.
Esther: What’s your favourite resource for mobile growth? Is that blog, newsletter, or a site?
Chang: I have been reading growth.com a lot.
Esther: Who is the person in mobile growth that you’d most want to have lunch with and why?
Chang: You.
Esther: [laughs] Thank you. I want to have lunch with you too. We’re a bit far away, but a virtual lunch, we can definitely do.
Chang: Yes, that would sound great. When we went out, you were four people in the same city. [crosstalk]
Esther: Nobody is meeting for real-life lunch.
Chang: We still only have virtual lunch.
Esther: It’s true.
Chang: That will feel the same.
Esther: It can be dinner for me and lunch for you.
Chang: Yes.
Esther: Connected to the other meal that we didn’t talk about, but what is your favorite flavor of pancake?
Chang: Pancake, I would say blueberry.
Esther: Good choice. Very good choice. Amazing. Chang, thank you so much for sharing that with us. It’s super interesting.
Chang: Thank you.
