Made by
Made by
Microsoft’s Sarah Thiam spoke at DevRelCon London 2019 about tracking community metrics without crossing the line into creepiness.
Sarah: Hi everyone. Nice to meet all of you. My name is Sarah Thiam and I’m a Dev Rel Program Manager at Microsoft. I’m actually based in Singapore so if you heard a really big “whoop” during Elisha talk yesterday, that was definitely me. Whoo, Singapore!
What I want to talk about today, and I think Christie did a really great introduction of how I came from marketing. It’s been a journey from marketing jumping to dev rel. It is really, really different and coming from a world of lead tracking, one of the things that I realized in dev rel is that it’s important and I see the value of it, as Mary and Steve talked about yesterday during day one. But at the same time it’s done in a very different manner and there are a lot of sensitivities to it that I went through a learning journey to learn and I wanna share with all of you as well so that you can build on top of that.
So, today the talk is about how to report on community relationships without being creepy about it. And if you can see, it’s a little bit of a IKEA theme over here, ’cause I’m gonna give you some do-it-yourself tools on how you can build your own program for your dev rel team. Because every team has different priorities and different needs as well. So one quick question, and a show of hands in the room, how many of you yesterday during the whole conference have tweeted a really quotable quote on the screen and saying like, “Oh, awesome quote”? Show of hands?
[several audience members raise their hands]
Great, so that’s what I prepared for. But what I wanted to say was, dev rel reporting can be creepy. This sentence will morph over time through this presentation so don’t take it as a hard fact and don’t shy away from it. One of the reasons why it can be creepy, and I think we talked about this a lot yesterday, is around the business value that, as dev rel professionals we are paid to do a job. And at the end of the day, those stakeholders, in order to continue their investment, will continue to ask us, “What is it exactly that you’re doing? Can you please help us understand? What is the business impact you’re bringing back to our business for us to continue to invest in your travel, your budget, your swag, and your stickers?” I think one of the things that happened for us in dev rel reporting is that there was a lot of hesitation for it. Realistically my team and I faced it as well, and I think there were questions around how exactly are we gonna format this, and how do we avoid commoditizing community connections?
As much as possible as a program manager one of the things that we talked about was how do we humanize the process that can be effective for our advocates out there? When I say advocates I mean the people in my team that are actually focusing on the technology and going out into the field and connecting with the community.
I want to make a point that dev rel reporting can be creepy but more importantly if you manage it the right way, dev rel reporting just needs to be sensitive, and it can actually be done in a good way. I’m going to talk a little bit around my role itself. Another really important insight, and I really like this slide a lot. And I’m going share a little bit around what my role is, because you might be thinking, what is this additional title to dev rel program’s manager, what is this program manager piece in dev rel?
What we do focus on, besides doing a really great job of events and putting awesome stickers is also to think about the skill and adoption of our advocates’ content, and the technology, and enabling them to have more streamlined and optimized processes to take product feedback and improve the technology. So, that’s where your program managers come in. We are the people behind the scenes, you don’t get to see us, we actually do a lot for connecting the dots. And so, one of the things on the top of the mind for us is how do we come up with that kind of process to report our team’s value while they’re out there doing the great work that they do back to other stakeholders within our business who might not understand what their role’s about? And also, I think I wanted to share that when I came to dev rel, you know from Christie’s introduction that I’m new to dev rel.
I’m going to share a bit about the perspective that I had when I came into creating this process. When I came into the team, it was probably about months one or two. And they said, “Okay, you are from marketing, so you must know storytelling really well. So we’re gonna task you with this task on creating a process that works for our whole team of really, really, really experienced people in dev rel on how we can capture that community connection.” To be fair, there’s a lot of research that I’m doing right now on the back end, it’s about month 10 for me now. So, I’m doing a lot more research and next time around I hope to be able to show some really good references. Probably read Mary’s book or something like that. But also, where I’m coming from in this talk is really a process that came from the ground up.
It was something started from scratch. Think of it like a guerilla process making technique, where you go out into the field and just ask people, “What is it that is important to you?”What is it that you are comfortable or not comfortable with? And, how do we create a system for you that works and that you want to use and that adds value to you and is not just busy work?”
I’m going to share your takeaways for today. Okay, first thing that you’re gonna get to take home is the thought process of building your own dev rel reporting program. Again, this can be very different in every organization. And, the second thing that you’re gonna get to take home is my learnings. It’s a simple process, but also I’ll share the realities of what I went through so that you can try and look out for them. You might still stumble on them. But, at least you’ll have a heads up. And lastly, what other dev rel professionals think. So, surprise, surprise – we are all dev rel professionals. So, one thing that I’m gonna keep open is that if you hit the My Twitter page, the top pin tweet is actually an open discussion, it’s a live poll on sli.do that you can key in your considerations of what you think a reporting program is going to be about. ‘Cause, as you can tell from these two days, it’s actually a really hot topic in dev rel right now. I’d like to hear an aggregated opinion that we can all share with each other here on what we think is important and what needs to be included.
Okay, so I’m sharing the first part of my process. Super ground breaking first step, listen. I think, oftentimes, even though I’m new to the team, listening is something that is easily forgotten. When push comes to shove and everything is happening really, really quickly, it’s hard to remember to listen. It’s hard to remember when you are being tasked with a certain thing and expected to deliver, it’s hard to remember to listen to your teammates around as well. ‘Cause again, we function as a team, we don’t function as a individual. Listening is going to be really important.
When we first brought up the topic of reporting, there was a little bit of hesitation. There was a little bit of confusion, and there was a little bit of just not knowing what it’s about. People were not super comfortable about being open to learning this process. The first thing that we did is to empower them with the opportunity to build a program, to have a program that was built by them. We reached out with a lot of feedback questions and asked them, exactly what is it that you’re not comfortable about, what do you think the reporting needs to entail, and what do you want to be able to put down into a form so that we can record what you’re doing? So, two things that we heard, and these are my learnings, I’m gonna do both at the same time.
These are the learnings that came in. I think these two really came up tops. One was on privacy and one was on authenticity. So, I’ll share a little bit more about that. For privacy, it was concerns around if you get a D.M, a direct message on Twitter, that’s one-to-one and personal. But, it is a really, really great quote to say, “Your team has really helped us with our technology, “your team has really helped our community, “it’s been great and it’s been awesome.” And, you match that to your stakeholders, pressing you every day and being like, “Can you explain to me exactly what you mean by advocacy?”, “where exactly has the favorability of your program increased? I wanna see proof, and I don’t wanna just see the quote itself, I wanna see the person that said it because it has to be someone credible as well.” It puts you in an awkward spot of, do I share this personal piece, even if it’s just internal for our team, just to showcase the value that we’ve brought?
And then, another question that we also faced, often, was “Do you have a list of your top community influences, or top community connections? I want names, I want emails, I want Twitter D.M.s, LinkedIn profiles, everything, all there on the sheet.” And the more that you collect is gonna be able to show the number, the amount of impact that you have. And, “Oh yeah, also include all the impressions on their Twitter handles so that we can count the total and it will cross a million,” or something like that. So, that comes to the point where you decide, do I share that list? Isn’t privacy a big issue? And, how far can I go to violate the privacy in order to do well at my job, or to be able to prove my impact? So, this is where the reporting program comes in handy, because when push comes to shove and you’re stressed out, you wanna have a reporting program that you can fall back on that is able to respect people’s privacy.
The second thing is authenticity. So, on this piece, the questions that came in from our team out there was really a concern around commoditizing community relationships. And, I think dev rel is a really, really interesting industry that is very different from any other industry I’ve been a part of. It is not sales, it’s not P.R., it’s not marketing. And, it’s something as a business, but it’s also very tied to human relationships. It’s a career, and I’m sure all of you are friends in some sense, and you’ve got to know each other really well, and it takes a lot of trust for a dev rel advocate to be able to do a good job. ‘Cause, only will trusted people be more open with their feedback and be more invested to give you better engagement and better feedback for you to improve that technology, that then goes back to them. So, it’s a symbiotic relationship. So, that’s really important.
But, once you start reporting on it, and if you say, like, “Oh, I’ve done 10 community connections”, “I’ve done 10 trips, duh, duh, duh, duh, duh” it becomes a number. And, by making these things a number, are we dumbing it down to something that is so much less than the meaningful relationships that we make? So, that was a big concern as well.
The next step was on acting on feedback. So, as program managers, whoever it is, if you create something, you have a responsibility to listen, but you also have an accountability to act on the feedback that you will receive. You’re gonna receive a bunch of feedback, and it’s gonna be really redundant if you just leave that aside, or you don’t demonstrate how that has directly connected into you building your program. So, acting on feedback was really important.
The two things that I talked about earlier on, I’m gonna share how we acted on those. Oh, yeah. So, on privacy, on the piece of privacy, we put in a baseline requirement of how privacy is key. I’ll give an example, and you can always choose your own for your own team. Privacy is key meant that we decided as a team to only focus on publicly available information. So, only things that people have posted up on their public Twitter sites, if it’s not privatized. Or, they posted it on a public Facebook group and not a closed Facebook group, regardless even if the Facebook group has 5,000 members, it does not mean that it’s okay to share that content if it’s not publicly accessible. So, that was a big piece.
I think on privacy, another big piece, and a good reference point for us was to use the GDPR standards. So, GDPR, for those who don’t know it, is a European standard for privacy. Of course, I’m in London, so all of you know it. But, in Asia, it’s not super obvious most of the time. But then again, it pertains to every European citizen as well, so with the world becoming more globalized, you never know if a European citizen is in your community piece. So, it’s always a safe standard to go by, and you can actually use this as a building foundation for you to build up an additional requirements that you wanna add to your privacy filters. So, we made sure that those two were in place, and we tried our best to make sure that our reporting process was able to cater to that. For authenticity, how we responded to that, and acting on that feedback of how we were afraid to commoditize community connections, we made sure that it was really important to defend dev rel authentic behavior. And when I say that, I actually mean in two ways.
The first way is being able to provide the right metrics. So, metrics drive behavior. If something is celebrated in your company or your team, it’s going to influence the way that people act moving forward. So, you wanna make sure that you have the right metrics in place. Now, for example, if you have a metric that says the number of people filled in a room at every dev rel engagement that you do. Sounds good at first, and might make sense for the first few times, but slowly, some people, depending on the team, or depending on how stressed people are, might end up making it a checklist, and might end up making it just a numbers game. So, they’ll know how to game the system, not saying that they intentionally do so, but if I can book a room and I can invite more people than developers to come into a room and it becomes 150 rather than just 50, real programmers and real developers that will get value out of this, then that decreases the value of your engagement and you don’t want it to go unseen in your program. So, that’s really important to have that. When we say different authentic dev rel behavior, we meant to have metrics that are qualitative and quantitative at the same time. It’s still balanced and we’re still working that out, but how do we measure the amount of engagement that we’ve done together with the quality of those engagements as well? It could be true storytelling, it could be true, a more qualitative progress that you keep a record of over time while meeting the privacy mentioned above. But, those are things to think about.
The second way to defend authentic dev rel behavior is that the other concern we had was that, if we have a reporting process, it’s going to end up pigeonholing a few of our activities into only about three types. So, I’d probably say like… And, these are the most common types, and I’m not saying that you’re wrong, but if you have events, community, and online blog posts, then if we don’t… If we’re unable to go and double click into that and to open it, keep it more open-ended to how people can identify a Venn diagram space in the middle of two categories and come up with something really creative and new to engage the developer community, according to their different changing needs, then we’re always gonna be pigeonholed into those three things only. And, it’s just gonna be a blog post conversation, an events conversation, and you wanna be more than that. And, by keeping things open-ended – one great example that came from my team was having a social movement start on Instagram for developers. Right? So, that actually didn’t come under any of those categories but it was something that, because we kept it open-ended to be able to celebrate things that are new and creative as well, that was actually how we managed to keep the reporting process a bit more open, and encourage more authentic dev rel behavior. So basically, let people continue doing as they are doing without feeling pigeonholed.
And lastly, is to optimize. So, optimization is around how you know you’re going in the right direction, which is great. You know your reporting program’s going in the right direction, but it’s still pretty rudimentary, because you’re only meeting the baseline requirements at this point. How do you then take that and optimize it a step further so that you can not only just meet requirements but also lead the conversation? With a reporting program. I know, that’s like the dream for all program managers. “I can finally lead a conversation “and not just have people throw things at me “and be like, ‘fix it!'” Sorry, very real.
So, the last piece is around how we optimized it a little bit more. On privacy, we optimized it in a sense of being more community focused. So, at step one, we talked about how privacy is key, so we won’t record any non-public information. But what if we also moved away from individuals as a whole? And, we just moved our reporting to us focusing on the community only? Which is great because in a sense you’re leading the conversation on how it’s more important to focus on that community development and growth regardless of the members that change. And, the members will always change, which is very natural and fine. If a community leader that you’re in contact with moves on and somebody else takes on, you’re still able to support that person, because the community is the thing that matters, not the individual. And, this is also great in the sense that you’re encouraging a potentially toxic behavior of focusing on a few exclusive individuals, and you’re being even more inclusive in the developer environment, which is great.
The other piece on authenticity and how we optimize on that is to focus not just on what you did but the learnings that came out of it. Having a majority of your reporting be about the learnings and the insights that came from it, so that as a team you’re centered more on an objective of improving developer engagement. Which then ties directly to what your advocate’s intents are actually out there in the field. You’re not actually changing the way that they behave at all, zero. So, you’re actually directly tying in your reporting mechanism to feed into what they are focused on, which is improving developer engagement. And, not just for them, but also how do you share these learnings with the rest of your business to say, “Hey, as a whole company, “this is how we can better engage developers”? And, it’s not just what we did, but it’s more important that the learnings and the best practices that came out of it.
That is the three step process. And, super ground breaking process: Learn, act on feedback, and optimize. When I did the slides, I was thinking, “Oh, I came up with this really great model and it’s gonna be a new thing.” But then I looked at it at the end of it, and I was like, “Oh, actually that’s pretty common sense.” But then, again, it’s really important to remember all these things. And, why I named this “A Not So Final Product” is because, as with anything in the developer ecosystem, things are always gonna change, things always need to refresh, and this is the only way that will keep relevant, and to keep improving. So, it’s not final. There’s a lot more research that can be done. There’s a lot more input that all of you can give to this discussion. Whether, at listening on my Twitter, or amongst yourselves, on how we can improve dev rel reporting as a whole. And, I super look forward to what you all have to say, as well. So, that’s the end of my talk. It’s gonna feel like this most of the time when you try to build out your programs, trust me. But, keep going. Yeah. Sooner or later, the table will get built. All right, thank you.