Mobile Growth & Pancakes #3: User-Centric Feedback Systems with Sarah Main

On episode three of Mobile Growth and Pancakes we sit down with Sarah Main, Director of Product at WeatherBug, to look at their unique feedback system and discuss the significance of partnerships.

In this episode of Mobile Growth & Pancakes, Esther Shatz is joined by Sarah Main, the Product Director at WeatherBug. Sarah talks about her role at WeatherBug, their unique feedback system, and the significance of partnerships in her industry.

Sarah has a diverse set of responsibilities in the company; apart from being in charge of the product organization, Sarah also contributes to marketing, customer support, and design. Weather Bug has over 15 million users across desktop, Android, and iOS.

Check out all the episodes of Mobile Growth & Pancakes here.

Connect with Sara and WeatherBug here:

Timestamps:

02:35 Introduction to Sarah and WeatherBug
03:28 Discussing KPIs
04:53 WeatherBug’s scale
05:20 The process of collecting feedback
06:44 How feedback is used
08:57 WeatherBug’s partnerships
10:35 What does partnership success look like?
14:00 Once partners are shortlisted, what’s the next step?
20:24 Going back in time
21:45 Quick-fire questions

You can listen to the episode here:

Or, listen on:

Apple Podcasts
Spotify
Google Podcasts

Key takeaways from the episode: 

  • WeatherBug’s development process is extremely data-driven. Their feedback-based system helps developers understand how their app is being used, this data is then fed back into the product development process. 
  • WeatherBug’s core KPI is the “retention time of an average user”.
  • Sarah uses partnerships to fuel growth, one such example was their partnership with Arity, a mobility analytics company. This partnership allowed WeatherBug to provide users an in-depth analysis of how their commute would be impacted by a change in weather. 
  • The success of a WeatherBug partnership is simply determined by the volume of users visiting that partners section of the app.
  • Excellent communication is the key to a strong partnership. An honest and synchronized collaboration could save the relationship from potential issues. Maintaining an aligned schedule and running all updates past your partners can reduce friction and increase the likelihood that that partnership will thrive. 
  • If Sarah had the chance to go back and do things differently, she would do more user based A/B testing.
  • Sarah believes that an aptitude for learning is essential to thrive as a mobile growth professional. She advises growth marketers must always learn.

“Absorb everything, then don’t be afraid to commit to something you’re not certain of”

Sarah Main

Main Transcript:

Esther Shatz: Today we have Sarah Main from WeatherBug. Sarah, could you introduce yourself a little bit?

Sarah Main: Hi, my name is Sarah. Well, first of all, thank you so much for having me really appreciate being on this podcast. I’m the director of product for WeatherBug. We are a weather consumer app that delivers the weather to all over the world. I’m in charge of the product organization, but we also operate as a very lean team. I also manage all of the marketing, customer support, and the product organization, including design underneath me.

Esther: Amazing. Yes, you only have a small set of tasks as I hear it. Not a lot at all. [chuckles] We’re going to dig into some of that the specifics of your strategy, but first, if you could let me know what are the KPIs that you’re looking at when you’re optimizing. What’s the north star of what you’re aiming for?

Sarah: Totally well, we’re always looking to grow our user base. I think we always want to look at the end-user at the end of the day. I think that’s the most important of growing your user base across all of our different platforms and also understanding how do our users tie into all of the different platforms they’re on.

The one thing also to give you a little bit of background about WeatherBug is we’ve been around for 20 years. We’ve actually started when we were the pre-installed apps on desktops for Windows. A key part of our understanding is also how do we create that user journey as the users going across applications, whether it’s from desktop to mobile, to net connected TV or on other different devices.

Really retaining that user and touching them on different platforms I would say it’s very, very key for us, but also retention is how long is the user spending inside the app. We want to make sure that yes, they’re coming into our application, checking the format, the forecast, but what else are they looking at? How can we make sure that not only they coming, but we’re attaining those users and making them spend more time within the app?

Esther: Do you have specific behavioral KPIs or it’s more day seven usage, monthly average users?

Sarah: Session length would be key, session length would be the key, definitely it’s how long was it in the app whatnot. Then obviously DA use and MA use are very key important for us.

Esther: Awesome. Just so we can give an idea of scale and of course share only what you’re comfortable with. How many monthly users about, are we talking about, or how many downloads for your mobile apps?

Sarah: We actually have over 15 million users across all the applications and those are primarily desktop and both mobile platforms for Android and iOS.

Esther: Amazing. Let’s go ahead and get a little more understanding about a cross funnel work. I know feedback is something that is critical to the way you’re working. Can you tell me a bit about how you currently get feedback from your users?

Sarah: Totally, definitely. I would say that is one of the most key things to us is understanding how our users use the app. Like I was telling you, so we’ve had users for– I’ve been with us for the last 20 years. We have very vocal users telling us what they love and what they don’t like. Rather than always just building new features and figuring out what we need to build next and how we can keep up with the competition, we also prioritize what the users are saying quite a bit, so we can integrate their feedback into other tech deck or new things that they’re looking for.

We put them into an internal feedback system where they can email us saying things they don’t, and they do like, and we categorize that each week on the top-ranking complaints and top-ranking requests. Then that goes into the product pipeline directly. Feeding in from customer support underneath my team into understanding, “Okay, if this is what our users are saying.” and then also alerting us in case something goes wrong or something’s not working. We actually know in real-time because users are going to complain right away. If something’s not showing up or something’s not working as it’s supposed to be.

Our users can actually help us understand our QA process to a very, very quick, real-time understanding, and then filter that into the tech prioritization to understand what we should be fixing or working on between the large scale projects.

Esther: Are you able to tie that feedback a little more top of the funnel. When you’re looking at growing into a new audience space, do you have that ability to incorporate feedback or is it focusing on your existing base?

Sarah: Right now it’s our existing base, but the nice thing about it is because they are so vocal, they can also request what platforms they’re looking to be on. I would really want you to, hopefully, you’re on this new TV or you’re on a connected– let’s say, Alexa. If we are getting that feedback from our existing applications, it can actually shape the ongoing product roadmap in the future to understand what they’re looking for.

Also, it really comes in handy too, when they’re looking for different data sets or looking for different features. If there’s something really particular to them, whether they’re looking for a forecast or something to do with travel or boating, that really actually helped shape our understanding, “Okay, this is what they’re using our app for and this is where we could improve and this is what we could do better.”

Esther: Amazing. I guess it could also tap you into a different user base at the end of the day, if there’s an alternative use of the app that you weren’t thinking of as your main use case, you have a group of users now to tell you, “Actually, we can use this and we do use this for this purpose.” I’d imagine it benefits your reviews as well. If you have any, user base you’re responding pretty real-time to their requests. You don’t get that the slew of reviews you see when a popular app or at least a new version of, Oh my God, why didn’t they ruin it one star.

Sarah: Totally. I think that’s actually one key to the way that we operate as organization as well, is we really care about the mission of providing safety and environmental intelligence to all of our users. We really put a lot of due care into our users. All of the emails are actually responded by customer support team rather than just a bot, rather than just something that’s just an automatic reply.

We actually be helped every single user that we work with, diagnose their issue and get to the bottom of it and help them solve what’s going wrong rather than just saying, “We’ll get back to you. I’m sorry for the inconvenience.” Its really is important to us to making sure that we show them that we are going the extra length to making sure that we help fix their problem.

More often than not usually the angry customers just want their voice to be heard actually then come back and reply with a more positive review saying thank you for being how understanding and how quick you got back to me. It really does show in the care of the users as well.

Esther: Oh, I totally agree. All right. Let’s shift gears a little bit from feedback into partnerships. I know that that’s a central focus for you and using partnerships to grow. Can you describe that a little bit?

Sarah: Totally. I think we’ve always been a B to C company, obviously looking at the way that we can deliver the weather to our consumers, but in the last year or so, we’ve actually been opening up, how do we grow our user base and functionality was leveraging partnerships. One partnership, in particular, you can look into our application it’s called the commuter section. We partnered with a company called Arity, which delivers telematics to consumers. They are the technology branch outside of Allstate that’s providing driving data to understanding how to better prepare and better drivers on the road.

When we partnered with them, we realized that we could combine weather data and driving data and create a feature that would allow users to come in and understand, okay, how is weather impacting your commute? How can we understand about our quick use so you can leave on time or understand that how it’s going to be impacted the next day.

We’ve been looking at how do we partner with different businesses and enterprises to create new feature sets and new data, to allow users to really understand how weather’s impacting all different aspects of their day to day, rather than just opening the app and saying, “Okay, is it going to rain the next hour or so?” I’d say that’s really important to us is understanding we can deliver more value and more safety on so many aspects of what they’re going to be doing every single day.

Esther: How do you measure the success of a partnership like that? How do you know that you’ve actually really hit onto something that’s working well?

Sarah: Definitely. The one thing about it is we can actually see obviously how many users are going to that specific section in the application. The way that you look at our app, obviously we have so many different data sets. We measure obviously what are the different areas in the app of the users going to, and first and foremost, the users like obviously want to check the forecast and they want to understand if they’re looking at maps, how the weather is impacting your day-to-day, but we can see then obviously on a ratio, how many users are then actually going to the different datasets of the, of the app and not being obviously now the commuter section.

We can evaluate that we had an assumption, how many users we thought we’d go, and how many users would use it. Then obviously now over time, as it’s growing, we do that as a benchmark of success. If we reach a certain amount of users that are actively going back to that section every single day.

Esther: It’s more than just that initial click and that they’ve explored it. It’s how much this has actually become part of their routine usage?

Sarah: Totally. That’s the thing is also the one thing to note about the partnership with Arity is when you’re in that section, once your driving has been calculated and you’ve done a certain amount of trips, we can also then start predicting the common routes you have and also saying, “Okay. It looks like you drive to this location.” We can tell you that it’s going to be pouring rain overnight, or there’s going to be a snowstorm at night. You should probably leave early so then you aren’t caught in traffic.

Then that way, combining the telematics and can driving, whether we can understand, okay, you users want to know how we can better help prepare them when they don’t even know what’s going to be coming in the future.

Esther: That is amazing. I’m imagining now the night before you go to bed, instead of having to figure out that you’re wake up and wonder if you’re going to be an hour late, because that at least where we are, we don’t have snow here, but Israeli drivers shot down in the rain, like two drops of rain and everybody’s like, Whoa, the roads are crazy. Just out of curiosity, you do proactively alert them like, “Hey, this is your normal drive, something’s going on there,”?

Sarah: Yes, I’d say that is actually in the pipeline. It is not that feature is not released yet, but that’s exactly where we’re going to, because the end of the day, we just want to make sure that our users can be the most equipped for any situation, and also giving them that heads up notice, depending on their route the next day, they can be aware that, okay, something’s going to change in the forecast.

Esther: That’s super cool. That’s genius. There’s a lot of avenues that it looks like you guys are always listening to your users, you’re always looking for opportunities. How do you create that shortlist of ideas? How do you narrow down on, okay, this is a partnership we want to pursue. Let’s see what we can do with this out of the bank of things that you’re coming up with?

Sarah: Totally. Well, when it comes to products, we all have hundreds of projects in the pipeline of understanding some different directions we can go. There’s always that brainstorm, that ideation. However, the one key for WeatherBug and the way where we look at partnering with different companies is we always want to make sure that our mission and values align with them. We’re providing the same expectation what we’re giving to the user.

First and foremost, with Arity, is providing that driving intelligence to create safety on the roads. Very hand in hand with us creating weather intelligence, environmental intelligence, keeping you safe in all aspects of your lives. We always have different partnerships that are on our table. The number one thing that we look at is we want to make sure that we have an aligned vision of how we view our users, and how we’re leveraging that data. When we create those solutions, we’re aligned in what we’re expecting to get out of the users as well.

Esther: What happens next? You’ve got your idea, you know what you believe is going to impact your user base the most powerfully, what’s the next step? How do you actually make the partnership come alive?

Sarah: Totally, it definitely comes down to great communication between the partners and an expectation where you want to go, from a product side, you definitely pick on which direction you want to go. Then we’ve confirmed, okay, this will be the next feature build, and it’s the next partnership where we can deliver aspects.

It goes back and forth, setting those clear expectations, saying that communication and then really putting that understanding what the data looks like in a product and us designing a potential feature or a potential new section within our app, and then going, okay, does this work for you? Does this align with your goals? Does this align for us? Then obviously setting those KPIs, what we think we can hit once that partnership goes live or goes live within our app.

Esther: It sounds like you guys are, both through the feedback loop and through partnerships, that you’re incredibly user-focused in the way that you’re creating your product roadmap and the way that you choose what to do. Do you find ways to pull that in the marketing side as well? Or is it just more of a natural flow of we’re focusing on the users, so we’re just giving them what they’re looking for?

Sarah: Totally. Definitely focusing on the users for sure is one thing. That also flows into how we promote our app and whatnot. In the marketing standpoint, a lot of our marketing is actually very organic, rather than paid ads or all around social media. Also we have a fantastic partnership with a company called the live storm chasers, who actually go around the United States chasing after storms, but are directly communicating to our users through Facebook and through Instagram and talking them about what’s going on, how the weather is going impact them, and really connecting to the users one-on-one.

We definitely connect that personality and that connection to users to our marketing strategy and really trying to connect the users, rather than trying to have this very kind of more of a serious approach. We want to go very more personal approach when it comes to social media. Users can connect and reach out to us and also submit what’s going on in their lives. What are the different videos and photos that you’ve seen on your day-to-day? Really bringing that user-centric approach ties in the marketing to the product as well.

Esther: Do you have seasonal trends outside of just holiday season, whatever your weather app, and are you seeing spikes when the weather gets bad? Or is it just static the year-round?

Sarah: Without a doubt, it’s definitely that’s one of the challenging things about the job is we get more users when the weather gets worse. I never want to impose bad weather, but you definitely can see that when weather spikes you see different usage because obviously users are coming into your app to check what the storm is going to be like, what’s the impact in my location. Anytime there’s high weather, you’re going to see a high spike in users.

Esther: Do you have any ways of tapping into that, from the way you manage whether it’s marketing in the social media side? Do you have a strategy that aligns with that, or it’s a natural flow of how things go?

Sarah: Definitely, I’d say you can definitely align up a strategy with anything with social media, but also product push marketing. Think about how you could engage with your users directly through your app. You can also send out notifications to users in times of high weather to then push it to the users’ homescreen, and then they go, “Oh, I should check my weather app, let me know what’s going on my location.” Thereby driving more users in the app with a higher session time.

Leveraging the tools of connecting to the users outside of just them opening the app, really you can tie into the weather trends because you know that users are going to be concerned about the weather that impacts their location. Giving them a heads up will just automatically let them know what’s happening and drive them more in the app for more usage.

Esther: Now, I’d imagine you also get you mentioned most of your traffic is organic outside of the social activities you guys are doing. I’d imagine there has to be a word of mouth factor here of if you have this kind of level of communication with your users that I’m imagining a mom pinging their kid, a kid pinging their mom, hey, make sure that you are downloading this, this just helped me, you should be using it as well. Do you have a way of tapping into the organic trends outside of your efforts in social networks? Or is that something that’s just amazing, great, it happens, well, thank our lucky stars that we got these users?

The one thing is we do get a lot of, I’m going to say super users, we get a lot of users that love and share our app through word of mouth. We do tap into it by understanding their product feedback in a positive sense of reaching out to them and connecting in that sense and on how it’s doing, how it’s done.

I would say probably we do leverage social media the most in the way that we can connect and understand what they love about our app so much and how we can grow that further. I would say it does more from the organic side, more than often or not.

Esther: You’re mostly focusing on ton social media from the organic side, it’s less about paid advertising, and it’s more about the outreach and a natural community engagement?

Sarah: Correct. I’d say the big thing that we want to create is that community, because we do find that our users are quite vocal, and they can give us quite good feedback without having to spend a lot of money. Our social media has been quite around that a community approach and making people feel like their voices are heard and also understanding how they can incorporate into our product roadmap.

Esther: Amazing. Do you have an example of a time where maybe the user feedback led you astray? Meaning you had a really passionate voice from your users, and it actually didn’t align with what practically happened afterwards?

Sarah: Yes, definitely. I think that comes down to also when you’re also with feature requests. You need to understand, we’ve got over 15 million users. When you have 200 feature requests for something, that’s still not even an increment of the larger user base. There was one time we did create a feature out of ongoing weather events that was happening. We said, we thought we should create a new section for it.

Unfortunately, obviously, it was very timely in which the users wanted to see something to do with natural disasters in the application. However, it wasn’t very representative of the larger user base at hand, because it was a small fraction of users that had requested it. We did invest a lot of time, resources in building that out, but it didn’t reflect to the remainder of the user base and how well that’s actually getting used.

Esther: Amazing. Now, let’s say you get to flashback, rewind a little bit. You’re starting out WeatherBug brand new, what do you do differently this time around?

Sarah: I would do more A/B testing. Before anything goes live, I would definitely put more things in users’ hands to understand two different directions you can go in, and allow the users to provide more feedback. That’s the one thing is we did a lot of user testing outside the application. However, we didn’t do a lot of AB testing in app. Letting the data speak for itself, even though you might have an assumption or you might think what’s going on due world current events, definitely pushes you in one way.

If you put a lot of the information to users’ hands, especially if you have a large user base such as 15 million, you can gather a lot of data very quickly, and help that steer what you should do. If I was to rewind three years ago, I’d definitely stay focused on the data and really hone in and listen to the users from a data perspective.

Esther: 100% it’s one of the biggest challenges is, it’s critical to listen to your users and of course, they know what they want, but sometimes it’s really challenging to self-report on what you think you need. Something can sound like a great idea in your head and then in practice when you’re actually faced with the two different options of how you’re going to be using, one naturally works much better. I hear you there.

Sarah: Definitely.

Esther: Cool. Now it’s the quickfire round of questions. We have got everyone who joins. Yes. The first one is if you could give one tip to somebody who’s entering the mobile growth space? It’s their first foray in, what’s the one tip you would give them?

Sarah: Aptitude to learn. Definitely one of the characteristics of looking at people entering into any new job space or you have experience or not, is really have an aptitude to learn to try new things. Don’t be afraid to fail and say yes to everything. I think that’s really shaped different growth paths and understanding how you can contribute to a growth strategy or a different part of the team is, come in with an open mind and just absorb everything.

Don’t be afraid to commit to something that you don’t know, because I really think that that can be applicable across all different aspects of organization, but especially product and probably the growth space as well because times are changing so quickly that come in not thinking you know everything and just be ready to learn.

Esther: Do you have a favorite growth resource?

Sarah: Let me think, I do quite like reading or listening to Recode podcast with Kara Swisher, might not be specifically related to growth but I do like listening to Kara Swisher’s podcast quite a bit as well as the new one that’s emerged recently it’s called The Morning Brew. It’s a newsletter plus a podcast and they’ve always got really good tips just on emerging tech and understand the whole tech space. Amazing, check it out.

Esther: Who is the person in the, if we look at the mobile industry, the growth industry, product industry, let’s imagine corona is not a thing dictating our behavior. You get to take one person out to lunch to hear what they have to say and learn from who are you taking out?

Sarah: Good question. Wow, you put me on the spot there. Let me circle back into that. I’m going to look up the last few people I’ve read about and I’ll let you know who that is, in one second. I can’t think and on top of that you putting me on the spot. Let’s go to the next question.

Esther: It’s all right. Okay, the next question is the last one and it’s the most important one, which is, what is your favorite flavor of pancake?

Sarah: Oh, I would say raspberry, white chocolate but also with Canadian maple syrup because you can’t not have Canadian maple syrup. It’s definitely much different. Raspberry, white chocolate pancakes for sure.

Esther: Very much respect to your pancake opinions. I married a Canadian just to make sure I’d get that authentic feel of the maple syrup. I don’t know if we mentioned but the podcast is called Mobile Growth and Pancakes. That’s why this is the most, most important question we have. One last thing, where can people find you if they want to hear more.

Sarah: I would say the best place for me to reach I was actually on Twitter. I’m actively on Twitter all the time. Feel free to also connect with me on LinkedIn. It’s not an issue. Twitter, I’d say is probably the most interaction that I get with tech people, tech discourse, it’s probably very, very quick and less formal as well.

Esther: Awesome. Okay, Sarah, thank you so much for joining, for sharing, for really giving nice, concrete examples for us to learn from. Thank you.

Sarah: Awesome, thank you so much. Do you want to do the person as well-

Esther: I was going to say you can do the person and we can circle it back in.

Sarah: Definitely. Got you. I got one.

Esther: All right. Ready? Who is the person in this space that you would most like to take out for lunch and why?

Sarah: The person that I would want most would like to go out for lunch would be Tim Ferriss. I think the way that he interviews candidates, especially founders and people in the space. He’s got a very good approach to understanding what they find important, how they contribute to their company, and what is happening in the tech world. I also do like their approach that it’s not always just tech growth. He’s got an understanding across so many different disciplines and how it contributes to world economics and the way the world is changing. I think he’s got a fascinating view on the way the world works.Esther: Good answer.

 

About Esther Shatz
Esther is a UX nerd with a natural aversion to sunrise, morning people, and birds. After consulting on website UX for clients like Disney, Walmart, and Skype, she moved into the world of mobile and now obsessively searches for new App Store tricks to share with her clients and random passerby. Hobbies include coffee, hula-hooping and making soup.