Metrics to track community health

Ankita Tripathi: Hi everyone. This is Ankita and I would like to give a small brief about what I've been doing until now before I begin the talk. So currently I'm building developer UGC programme at Google. I'm working as a community manager in developer relations, kind of also handling the community, the product and everything. So today when I will be talking about the matrix, I'm going to bring in different perspective with respect to product and as well as community because that's what I've learned in the few months that I've joined Google. I started my career as a tower developer. It was seven years back when I started it, and then I switched to become a content developer. Then I switched to become a technical writer.

And simultaneously I was also handling a lot of communities personally. And now I am working as a community manager here. So the topic that we are going to cover today is tracking matrix, like the health of your community.

And it's not going to be boring, I'm pretty sure about it because people say that we know everything about tracking what we need for measuring our community health. But here I will bring something very different because I know that when you are measuring community health here are actually talking about how do you keep your community members happy if they are enough involved in your community, if you are giving them enough topics to talk about and everything. So today I'm going to talk about real life examples, some community stories, three different community programmes that I had run until now. Then we'll talk about the real matrix that what exactly do you need to track. And third point will be, I hope that you go away with some of your own metrics and decide what you need to track for your own community programmes.

So let's start with the first community programme that I had been running for almost two years.

It was a programme, not a programme, it's a community of community managers in India, which is called Community folks. And I, along with other members was handling the entire community programme. So managing the members, managing the content, welcoming them and everything from that part. We have been doing that and that community is one of the biggest successes in my own community life, which also landed me today at Google as well. And the other community programme, other two community programmes in fact, which I'm going to talk about is what I've been running in Google itself. So one of the programmes that we recently ran was called 30 Days of Google Cloud. And now this programme was for the Google Cloud community in India.

We had around 300 campuses where we had to run this programme for. And what I realised that the Matrix, what ran for this programme was entirely different.

What we manage for a community, like I talked about community folks. So the matrix was entirely different for that. And the third thing that currently I'm building is developer UGC programme, which is called Dev Library. And again, we handle a community of developers here who have written or built any project using any of the Google technologies, be it flatter, fire base, angular Android, or machine learning. So they can submit their projects, they can contribute, they can inspire others with their content. This was a very different kind of a community.

Again, when you analyse these three are entirely different sets of communities which I was a part of and being a part of them, I realised that the matrix were entirely different for each programme. So this is what I had in mind today. What I realised is when I was talking about dev library, I thought that I was managing a community of developers while I didn't realise that they were actually the users of the product.

So they were not exactly community members. So what one we should realise is that the users of your product are not your community members. Community members is an entirely different set of different set of people who are actually interacting with the community, who are also giving their own inputs for the community and running those programmes. So this was one of my takeaways from these things. Now I'm going to divide this into matrix that are good to measure and matrix that are healthy to measure.

So what everyone says is that you should not track numbers, but while when you work for a brand community, numbers are highly important because that's how you run up the ladder, right? You have to show numbers to your own team, to your own seniors, to your own company and show that this programme, this product is worth investing in. So as a lot of people might know that Google launches a thousand products in a year in two years and half of them shut down.

So it's very important to show numbers and that's why these metrics that are good to measure for developer products especially here is first of all have some OKRs for your community. I have seen this like in personal community programmes as well. I had seen that we do not really have a vision for our community. These are the three OKRs that I need to measure and that is how it's going to define the success of my community. So what here we defined in the brand community, what we defined was at most we had three OKRs that this is a demographic that we need to target.

This is the developer community where we need maximum extraction for and what exactly do we want out of this community. So these were the three OKRs that we tracked. Second, we focused more on monthly active users than the daily active users.

This is again, something that I learned in these last three, four months is that why is monthly active user count more important than the daily active user count is because suppose if I put out a tweet in a day and suddenly it got viral, I will get a maximum traction on that day. My analytics will be increasing with the number of people viewing my site coming onto the site. But that is just for a day. Just for a day, maybe two, that's it. But what exactly tracks our community, our programmes, our products is how are you retaining that number?

If your monthly active users are still the same as your daily active users, that means that your community is doing well. The third part, again, the most important part is content. So whatever content you put out for your community, and this is I think common for brand community, for personal communities, both the communities that the content that you put out should bring value.

And if that is performing well, you know that a community metrics is doing good. And the last part, which is a personal stat that I was tracking for my own community programme stats is how many developers are we reaching? And I'm going to show you some numbers in the upcoming slides. So for a recent cloud programme, I think the three okas that we had, we reached almost 90% of them. We had almost 80% of active community members.

So if I say that we reached out to three 50 campuses, I would say that almost two 80 of them were active for entire 30 days, which is a huge success for any community programme. We reached out to almost 400,000 developers out there who wanted to build something using Google Cloud. We had almost 60,000, 6,000 thousand registrations as well. So these numbers show that developers out there, the community of developers that we have been running, which is called the GDSE programmes, we call it Google Developer Student Clubs, is something that was doing very well and Google Cloud was something that they were really interested in.

So these stats helped us knowing that. Now the second part is this was matrix that are good to measure, which I say are good to measure in brand communities, but the matrix that are healthy to measure and this healthy part of it, I calculated because the community that I've been drawing personally, which is called community folks, all the measures, all the metrics that we have been tracking, there were not numbers, they were never numbers. And what we kept tracking was how is the engagement going on? What is our community age?

I calculated the community age with the old and new members percentage ratio. How many old members do we have? How many new members do we have? And the community diversity, like how many countries, how many regions are we covering? What is the gender ratio in our community? And also the community member types because this, I read in Rosie Sheri's newsletter, I'm a huge fan of hers.

So I read that in her newsletter that workers are not your enemies because somehow they are contributing to your community by being just there and reading out whatever you have put out there. So since then we have been tracking all these things.

And again, I'm going to show you some things. What we realised with community folks that is the personal community is that whenever we never focused on numbers, like some people say that this is a good metric to identify that you're having a good community, which means if you have an increase in community members, that also means that you are handling a good community, which is personally, which was never something that we tracked for community folks. So if I want to show you some stats for cf, k, ccf, K, short form for community folks, we have been running this community for almost three years.

And even then we just have 800 members. And it does not mean that we do not want to scale or we couldn't scale. It's just that we have a very specific agenda when we try to capture people and you have to fill a form, you have to show that you're actually interested in communities and we select who we want inside a community. And especially speaking, the initial members of our community was the hardest part and I think was the most beneficial thing that we ever did because those were one-to-one interactions that we had with the community members. And by that, when we started building CFK, I can tell you that there are almost 50 members who are the oldest members of our community, who are still active, who are still active after three years.

And I think those people, those community members bring the maximum engagement, maximum value, and they drive the community with them.

So I think this is one of the most important factors that invest if you want to bring really good matrix to your community if you want to maintain community health. So please invest in your initial members who you try to bring into your community. For cf K, we have conducted almost 50 online and offline events. And almost always the turnaround number was huge. If we had a hundred registrations, almost 50 to 60 people used to turn out for the events, which is a good number considering the whole pandemic situation as well. And there were always active tags on social media. So if you see Twitter, if you see LinkedIn, there are so many active members from our community who keep on tagging us on social media as well.

So which shows that they are getting something out of our community, which again shows they're happy, they are engaging, they're contributing and everything.

So this was some matrix that I had from community folks. Now, how can you ensure that you are maintaining community health? So what we did for both brand and personal communities is that we used to share feedback forms if, and this is not just a form, but this is more of a personalised form where we are actually asking them personally that what changes would you like to see in the community? Is there something that you want to know more? Is there something that we should change or tweak? The second part is having one-to-one conversations with the community members. Now, this is a very difficult part because when you have a huge community, more than 2000 members, it's very difficult to have one-on-one conversations.

But what we realised is picking up at least five to 10 members, maybe biweekly, can help you at least get to know if you're running on the right track.

So sometimes when you try to scale your community, you forget what was vision that you started your community for and even if you're doing right or not. So it's almost always good to have one-on-One conversations, I can't tell you for this dev library taught as well, this was a platform where we wanted users to feel engaged. We wanted them to form a community of their own product areas. And I did one thing that I had one-on-one conversations with at least 20 members from one product area, so suppose 20 members from Android. And I wanted to know what will make them submit their content or what will make them talk to other community members. And those brought out such huge, what you say, life-changing suggestions, which we also incorporated. So I think having one-to-one conversations is always good to go.

Third is incentivization. Now incentivization always does not mean handling out swags or community something, but it's always good to also incentivize them personally.

So maybe tagging them on Twitter for this community member did this, or we did this with CFK that we used to run a post and whoever had commented, we used to tag them in a blog post saying that this was a suggestion given by this community member or this was a comment given by this community member. So these are very small personalised things that we try to incentivize. And again, that ensures that the community members were happy. Fourth point is constant influx of useful content. And I think this is the most important part. How much are you bringing?

What good quality content that you're bringing into the community that also requires you to read more, study more and bring that engagement in the community. So content is always like the king, the queen, everything. So try to bring out good content. And last is interactive topics. I think this was one of the things that again helped us in the community was not just keep having community discussions. Sometimes if you brought out some personal topics, what is your favourite music? What is the playlist that you are listening to this week? So even those small non-community topics brought out the maximum engagement.

And again, the community members felt a lot connected. So these are the few things which you can help and ensure community health.

This was one of the tweets that I found from David sps that an easy way to tell if a community is healthy is when new members introduce themselves. Does anyone respond? One point for emoji responses, two points for welcome replies, and three points for personal responses. And this is an activity that you can run for your own communities. That what exactly, and what is the maximum number of responses that you get for the welcome messages. One thing I can tell personally for community folks that is the personal community is that I think we always have at least welcome replies.

So two points for us.

I designed this small case study. I don't know if we would have time, but I wanted to know that this is a very small example, which is something from a real life example actually. And I wanted to know what would your approach be? So suppose there's a community of ML experts who has recently hired a community manager. Their main aim for building a community of ML experts was to bring in diversified people and eventually help them become ML experts. Now the community manager starts scouring the social media route to drag people who are asking for ML help. Now for me, the pros and cons of this CMS approach was that pros were they were very clear about their OKRs, that they want diversified people and want to help them to become ML experts.

And also the second point was value driven community. But the cons were that the first step they took was to acquire members that took the social media route, which was not a very ideal way to do it.

And the second was they were aiming for a larger community. So all they cared for was numbers, like a huge number of people that they can just taken into the community. So I wanted to ask this question, what would you have done differently? And I don't know if you will have time for this or I can check, but I can wait for 10 to 20 seconds if there are any responses, but I have anyway designed, so I don't know if I can even see the answers. So what personally I would've done differently is when you bring out value driven content out. So suppose if I'm an individual who wants to create an ML expert community, what I will do is the first step is try to create ML content online.

Now, when people start reading it, they start understanding it. It's easier for people to connect with me now acquiring ML experts as first members. I will also reach out to those people who have been ML experts in this industry for a longer time so that when the new members come in, they will have someone to guide. And that's another second thing that you can ensure that the community runs and retains those members. And the third is asking the members what they expect from the community and focusing on retention more than acquisition. So I think that was it because I think my time is up. So any questions, any comments? I'm so happy to take them up.