Success metrics as narratives

Numbers have long been the defacto way of describing success. In dev rel we often count “butts in seats” for conference talks and workshops, number of badge scans at booths, and attendees at parties. But numbers don’t tell a full story, such as whether the event was actually worth the time of the folks that were sent to it. Numbers are not the storytellers we need in dev rel.

In this talk, Wowza Media’s Director of Developer Relations Amara Graham describes the use of narratives before, during, and after events to get a complete picture of dev rel event success.

Transcript

So, I’m super excited to be here today. Welcome to my home. And I was going to say sunny Austin,Texas, but it’s cloudy Austin, Texas today. Either way, I’d be staying inside.

I hope you are too, but today I’m going to talk about success metrics as narratives. And I want to start off by saying that I don’t hate numbers. In fact, I really like them. They can be quite comforting. I’ve been a lifelong athlete, so some of them are really important to me. But I also want to make sure that folks understand that success is different for different people and how we communicate that success across our program or across our company is really, really important.

So, let’s talk about numbers. My science teachers were very clear about numbers needing units. And what I mean by this is we have this lovely little metric conversion charts so that I could, you know, be inclusive to our global time zones here. But numbers were really important in science when you were doing different equations, you were trying to calculate things, but you couldn’t just end with, you know, okay, the answer is five, well, five whats, and importantly, five whats in what unit of measure?

Are we measuring in grams? Are we measuring in milligrams? Because there’s a big difference between those two. So, it’s really important to have numbers associated with a unit, not only of measure but what they’re actually, you know, trying to present and what they’re actually trying to convey.

So, as my science teachers liked to say, “No units, no points.” As I progressed farther in my career and did classes at the university level, suddenly it was like, “Well, if you have the correct units, we’ll give you like half credit.” Because that was also really hard to find at certain points. But the fact of the matter is a number without a unit just doesn’t make sense. If I tell you five, you’re going to be like, “Five what?”

Is five good? Is five bad? Who knows? So, numbers need units. So, what looks successful? So, I’ve put together kind of a list of things here motioning to my second monitor over here, which I’m sure, you know, no one cares about, no one can see.

But what looks successful and most importantly, what looks successful to you? So, this is a list of things that I pulled together from my previous experience. Some data points, some feedback that I received from anything from meetups to workshops, to conference talks, to booth duty. And these are some of the things that happened over those times. Are these successful?

I will kind of give you a spoiler and say, yeah, these are all successful data points. Success is obviously, different as you’re going across these. Some, you know, have dollar amounts. Some say, you know, that it was just for the price of pizza and drinks. But I really, really want to hit home that success looks very, very different for everyone.

And more importantly, success looks different as you’re going across your DevRel activities. So, if you’re doing a conference, it’s going to be very different than a meetup. If you’re doing a conference talk, it’s going to be very different than the conversations and engagement that you have at booths. And I think sometimes we distill everything down to, you know, a handful of numbers.

We present that back up to our upper management and we say, you know, “Look at the great things that we did,” and that can so quickly be misinterpreted. Another item that I can put here as I, you know, mentioned that I’m a lifelong athlete, I played water polo in college and I have one goal to my name. But the really important context there is that that one goal was against a team that had three times more people than we did and I scored that goal against their first string goalie.

So, for me, that one goal for my entire water polo career was so important because it was so incredible that my tiny little team against this giant team that could put in like a whole new swath of people every couple of minutes, that was incredible. Like I shouldn’t have been able to do that. And so, some of these numbers that you see here, you know, it doesn’t tell you how many people were at that event representing our company.

It doesn’t tell you how many hours were put into making that thing successful. You don’t really get the full picture here. You just get kind of like the little bits at the end that say, you know, “Oh, I guess this was good,” or “This looks good.” And people start interpreting it as it moves farther and farther away from you. So, standing room only is great, but were you in a small room?

Were you in a big room? Does it really matter? Was everyone standing? Who knows? So, all of this to say, all these numbers have units, which we talked about in the first slide, but we need to have qualifiers and other language around them to make them really make sense. Because presenting this, especially to upper management, they’ll go, “Great. Now compare it to your other events.”

Well, if you just came back from a conference and then before that you did a meetup and now you’re doing something virtual, it may look very different as you move across those different activities. So, what am I saying here? Numbers are not effective storytellers, especially by themselves. So, it’s unrealistic to assume that numbers have denotations, so that’s your dictionary definition, even though they’re well regarded as fact.

We have a lot of folks that are sharing numbers, especially today in the world that we’re living in, we’re presenting numbers and we’re saying, “Look, this is so important. Here’s this big number.” But again, if we don’t present it with context and we don’t present it with that situational awareness, numbers are not going to gain that themselves.

The previous slides when I was talking about, you know, okay, the answer is five, the number five, well, what does that mean? Is five good, is five bad? Numbers are open to interpretation and they especially have connotations. What I mean by this is if you think of statistics or you think of percentages, most people will tell you depending on, you know, their field and their interest, that a percentage should kind of be tracking towards either a 0 or tracking towards 100.

So, you either have everything, 100%, or you have nothing, 0%. But that’s not really true. In some cases, you may be working on improving a particular percentage to be within 20% or 30%. Like we talk about meetup attendees. We talk about, you know, converting them from the registered folks to the folks who actually attend.

And a lot of people who run meetups will tell you that their conversion rate is somewhere in the ballpark of 10%, 15%, 20%, maybe even 30%. Now, it would be great if we could track towards 100%, we could do things a lot easier, like scheduling how much space we need, or determining how much food we need.

All of these things would be easier, but ultimately, the goal shouldn’t be to push that to 100% because you’re going to do a lot of work for maybe not as much return, but again, you’re presenting this number out. You’re saying, you know, “Hey, we had 30% of our registered attendees convert over to being actual attendees,” and someone who doesn’t really understand that situational awareness doesn’t understand that context to see, “Hey, we typically only see around 10% or 15%, so 30% is an incredible number.

You don’t get that context when you’re just presenting that number. So, you have to keep them really, really tightly coupled so that you’re not just presenting out a number with no context, you’re presenting out a number that has historical significance. It has that context.

And most importantly, when you’re presenting these numbers, you need to make sure that they continue to have their context because they can become dangerous if they’re misinterpreted or if the historical context is not there. So, having these things kind of merged together and presented in the package where you have everything together, you have the numbers, you have the language, you have the historical data points, all of this becomes a much more powerful story than just a single number or a single set of numbers for a particular event.

While I was doing some research to try to figure out like, “Could I find some sort of diagram or something to kind of talk about how, you know, numbers aren’t effective storytellers,” I did do like a quick Google search and was reminded about story arcs. And I remember learning this as a kid, but I don’t remember the graphs or the visualizations being so incredibly different across like my academic career.

I think all of the story arc graphs looked pretty similar. And if this is what we’re teaching kids in school, I can understand why they’re confused. But you can see, I didn’t add one because I was like, “This is weird.” And it’s even weirder if you take it on from like a mathematical perspective because this is not typically how you join lines.

But I thought it was interesting. Completely off-topic too. So, let’s get back on topic. Let’s talk about language. So, I hear that Americans love to use the word, amazing. Everything is amazing. “My wedding was amazing. My honeymoon was amazing. My vacation was amazing.”

They’ve kind of ruined the word amazing. And if they haven’t word ruined the word, amazing, they’ve certainly ruined, awesome, because that’s another one that’s I think is incredibly overused. And you can see here, I pulled up thesaurus.com, found some synonyms for amazing and awesome came up. So, you can’t even use those interchangeably. Pick a different word. But the point here is that we can use language and we can use a number of different terms within whether it’s English or your native language or the language that you do business in.

There’s a number of different terms that mean something similar or almost the same, but they might convey a slightly different meaning when attached to a number. “We had five people, fork our repo.” Isn’t that incredible? Okay? That’s a little bit better than “We had five people fork our repo.” Okay?

Well, do we want more than five people? Do we want less than five people? Is that a good thing? Is that a bad thing? Again, we start adding some of that language together with our numbers and we tell a little bit better story than we were than when we were just using numbers themselves. So, adding this language and taking a look at how we’re using language can help us understand a little bit better about our developer relations activities, especially when we start talking to our different folks within our teams and get their perspective about a particular event or activity and see how they describe it and see how it was impactful for them and the engagements or the connections that they made.

And that to me is a lot more important in helping define what was a successful event or not than just saying, you know, “Hey, as a team, we talked to 200 people.” Look, excuse me, 200 people is great, maybe for an awareness piece, but what about something more engaging? What about something that gives us a little bit more feedback than just, you know, “Yeah, we talked to 200 people.”

Great. Now what? So, I tried my best that I could to rip out all of the language that was in my presentation around event-specific things. I know most folks are not focused on events. You might be doing things that are virtual, you might be doing, you know, webinars or virtual workshops.

So, everything from here on out instead of events, I’ve just transformed it into the happening. So, the happening for you could be any sort of DevRel activity. It could be writing a blog, it could be doing a conference talk, it could be hosting a virtual workshop. All of the things that we don’t have to travel to do.

Or even in some cases, if you’re trying to, you know, review some historical information, historical being, you know, the past year or so, or even more to understand, you know, is this an event that when we do start traveling again, we do want to invest the time, money, and effort to go back to? So, from here on out, you’ll see the happening, which is, you know, insert your DevRel activity here.

So, before the particular happening, again, webinar, maybe it’s a meetup, maybe it’s a conference, who knows. So, before this, you want to make sure that you’re evaluating previous iterations. And I know for some of you, you’re probably kind of rolling your eyes and going, you know, “Well, duh.”

But for some people, that’s not really the case. They’re saying, you know, “Hey, we’ve got this full schedule. We don’t really have time to go back and see, you know, did it make sense for us to attend this function, or did it make sense for us to be involved with this activity? Previously we were asked, we didn’t say no. So, we’re just going to continue doing it because it’s kind of an easy win for us.”

But I think it’s really important to kind of go back and evaluate, you know, what you did in that particular happening. Does it make sense to continue doing that? And for each one of these happenings, I want to stress that you should be setting expectations and setting an event-specific goal. Even if you’ve gone to the same conference over and over again, every quarter or every year, and maybe it’s only really changed location, you still need to set a goal specific to that event because they’re going to look different year over year or session over session.

And then the most important question that I ask, especially when coming into a new company, I’m only a year into Wowza, actually, I think a year as of tomorrow, which is crazy. Does this align with our target developer persona or does it align with some sort of new functionality that we’re releasing or a new programming language functionality that’s coming on board that we want to be able to, you know, support and encourage?

All of those things roll into this idea of, you know, do we want to continue pursuing this investment or is this something that maybe it was cool a couple of years ago, but it’s kind of morphed and changed or we’ve morphed and changed and it doesn’t really align with us and our goals anymore? And that’s totally okay.

We live in a very agile world, going back and iterating on these things is really important and it keeps your developer audience engaged and not, you know, bored or not saying, “They’re always here. So I’m going to go talk to someone else because I know I can catch them at the next event or I can catch them next year.” Kind of keeping that fresh perspective is really important.

And I want to stress too at this point, this is where the majority of the work is for the leader. So, whether you’re a manager, a director, just a lead on your team, this is where you need to start shaping your idea of success or your idea of failure.

So, for the individual contributors, while it is important to have these things in mind, I really think this is something that the person who is leading the effort, whether it’s a program manager, or like I said, a manager or a director, this is really where we’d like to see them kind of setting the expectations and helping the individual contributors understand, you know, what are the expectations here?

What are the goals here too? So, some examples, and I’ll do this with the rest of my slides here. And hopefully, you can see where this is going. So, before the happening, you can get feedback from your team or the folks at your company that might look like this. I’m not going to step through each one of these, but I do have this nice little like emoji-coded thing that we’ll walk through with some of my thoughts when I hear things like this.

So, I’ve kind of distilled these down to be as sensitive as possible. I don’t want to throw anyone under the bus. I don’t want to do a finger-pointing exercise here. But this is all feedback that I’ve received over the course of my DevRel career about different things that we’ve done within DevRel. Whether it’s events, whether it’s blogs, whether it’s partnerships and co-marketing activity.

These are some of the things that I’ve heard. And you’ll see like the top one, “We’ve always gone. We always sponsor. It’s a key industry event for us.” Okay, cool. That doesn’t really tell me a whole lot about, you know, are we going to be successful taking our DevRel team there? It doesn’t really tell me a lot about, you know, why we’re engaging, who we’re engaging with, anything along those lines.

But it’s important feedback to hear to set your expectations and to start determining, you know, what would your goals be there? So, again, not going to read through all of these, but I did want to show you kind of my emoji code for these terms. So, when I say, “Evaluation needed,” I’m not saying that this is something that we don’t want to be a part of.

For example, some of the other ones on here for evaluation needed, “I’ve never even heard of this before.” So, maybe it’s a brand new community, maybe it’s a community that’s very like regionally specific. All of these things just mean that I want to ask more questions about this effort. I want to make sure that we’re going in and the folks who are organizing this or who are offering us this opportunity, they understand what my goals are and what I look for in success.

So, for me, I’m looking for, you know, can we engage with folks? Can we get folks hands-on with our technology? And depending on how they set this thing up, whether it’s, again, a blog opportunity, maybe it’s a conference, I want to make sure that the organizers are aligned with my needs as well. And understand, you know, kind of where I’m coming from.

So, this is where I ask a lot of questions. It’s not a no like if this is the first time something is being done, it’s definitely not a no, but I do kind of proceed with a little bit of caution and make sure that I can ask a lot of questions and get a lot of feedback between now and the happening. The handshaking emoji.

So, this is an alignment opportunity. So, at bigger companies, particularly you can have events that are sponsored by your corporate marketing team, or you can have guest blogs or other things that are sponsored by a team that’s maybe a couple steps away from DevRel. And these alignment opportunities are really great because you can win those internal advocates and kind of win friends, as I would say.

These are important to participate in. But again, setting the expectations that you might have a little bit different audience than you usually do from your other like DevRel-specific activities. And that’s okay. I think it’s important to be a team player. And it’s important to invite other folks into kind of your DevRel understanding and continue pitching this idea of DevRel because it’s still new to some folks.

As we’ve been onboarding new people to Wowza, there’s folks that I continue to run into that have never worked with a DevRel person. So, having opportunities like this where you can partner with them and say, you know, “Hey, it doesn’t really make sense for us to specifically go to this particular activity or this particular conference, but it might be cool if we have like a technical developer-centric booth or a developer-centric workshop to see if we can tap into the brains of the technical folks that are going to be at this particular industry event or something like that.”

Lastly, here, red flags possible. So, anytime someone says, you know, “We have to be here.” I don’t love that. If we have to be here, that’s not really setting us up for success. That’s saying, you know, “Hey, I’m a warm body in a seat,” or “I’m just standing here because we need to be here.”

That doesn’t really push my message and my goals forward for developer relations. Is it important for other folks of that company to be there? Probably. Is it important for DevRel to be there? Probably not so much. So, I approach these with extreme caution and make sure, again, that I have the opportunity to talk to the organizers to make sure that we can figure out a way that we can present this so that everyone kind of wins and we don’t just have to be there.

Lastly, “Is this a community-backed activity?” For me, I don’t want to say that this is like an absolute no, but it’s definitely shaping up like this. If I have to even ask if this is a community backed-activity, the chances are there’s no community behind it. And for me, I don’t really think that’s going to be a successful DevRel effort for us.

Again, that’s kind of more of my opinion, but that’s one of those things where I’ve noticed like, if you’re just kind of sponsoring this vague thing for this vague group of people and there’s no community behind it, the idea of really getting any level of engagement beyond just awareness and just the interactions that you have at that particular activity at that particular time is not really great.

So, during the happening, you should be asking questions like, “How are you feeling about the expectations and goals?” And this can be, you know, an internal monologue if you’re there by yourself. This can be, you know, questions to your team on day, you know, two or three of the particular happening as you’re setting up or as you’re getting ready to continue going.

And then do you need to reset anything? So, this may not be as well-suited for every happening. So, like for example, webinars or a format like this, I can’t see any of the screens. So, if halfway through we lose, you know, my audio or people are just really lost, they’re asking a lot of questions, I can’t see anything. I have no idea what’s happening.

But for workshops and conferences, this works really well because you’re able to pull the audience and you’re able to say, you know, “Hey, is everybody finished step four or five?” If they haven’t, it’s totally appropriate at that point to level set with your crowd and say, “Actually, instead of moving on to the next steps, let’s get everyone through the first couple of steps so that we can have a more productive and more successful use of our time, rather than shoving everyone through every single step to be like, ‘Okay, we completed it, we’re done.'” So, what does some of that feedback look like?

And my team that’s been out with me at conferences knows that I ask some of these questions where I’m like, “Hey, how do you think this is going?” And I offer my feedback as well, say, you know, “Hey, I think this is X, Y, and Z is happening, blah, blah, blah.” So, some of the things that I’ve talked with folks during happenings are, you know, “Where is everybody?”

“This is awesome.” “Lots of great technical questions.” “There are a lot of job seekers here.” Or, “I’m a little surprised at the questions that I’m getting.” So, I have slightly different emojis for this particular slide. But maybe you can kind of tell by my inflection, what is kind of happening here. And again, there’s no numbers associated with these.

So, when I say, “Where is everyone?” We could have, you know, had 200 people come by our booth or we could have 50 people signed up for our webinar and they might all be sitting on the webinar ready for us to start. But if we were expecting 500 people to come to our webinar, or we were expecting many more people to come by our booth, “where is everyone,” is kind of the appropriate response.

So, to emoji-code these, “where is everyone” is a red flag that’s possible. Depending on how things are set up, depending on, you know, if there are other conflicts in your community where maybe you have a competing event or a competing thing happening at the same time like you’ve decided to do a webinar at the same time maybe your competitor is doing a webinar, that may fracture the audience.

So, where is everyone? They are splitting their time between your webinar and your competitor’s webinar, things like that. The DevRel Happy Place. I love getting great technical questions. And I love being halfway through a happening and saying, you know, “This is awesome. This is fulfilling. This is where I want to be. This is where I feel like I’m providing my most value.”

But the fact of the matter is we’re not always in that space. We’re not always able to say, you know, “Oh my gosh, this was a great event. This was wonderful.” And then the last two, again, evaluation needed for, you know, I’m surprised at the questions I’m getting, is this an appropriate place to be, may need to level-set there after the happening. So did you meet your expectations and goals that you set at the beginning?

Would you do it again? And this is where those numbers start to creep in. So, be really, really careful. So, a lot of the feedback that you see here again, is things that I’ve provided after events. So, you know, “I’m bored. I’m not sure why we were here.”

Again, with my little emoji codes, I’ll go ahead and flash those up for us. “The organizers for this event are hard to work with.” That to me, it’s really not worth it. There’s so many events and things that are going on. There’s so many webinars and things where you can get in front of folks. Working with hard organizers is just…there’s a point where you’ve got to just like sever your ties and move on.

Not seeing the right kind of developers or saying, you know, “DevRel didn’t need to attend this event, we should just send marketing.” Again, I want the best use of time for my developer relations folks. And if we’re not getting those technical questions or we’re not seeing the right developer audience, we’re going to just move on. So, what should you do?

So, I truly believe that using words and evaluating at every stage is incredibly important. So, before, during, and after doing kind of this check-in with you or with your team, combine it into one comprehensive look so that you can evaluate the entire happening if you will. If you have a large team or you’re sending a large team to an event or a large team is participating in a workshop, you can collect feedback and run it through sentiment analysis.

If everybody’s responding in kind of a negative manner, that may be worth digging into and understanding, you know, what happened. If it’s things like technical difficulties, we get it. But if it’s things like, you know, “We spent six hours and we talked to four people that were having an incredibly meaningful conversation with us.” It’s not so good. And again, along the lines of, you know, these numbers creeping in, you want to share your goals and expectations, and you want to make sure that you’re gathering baselines.

So, if you’re running a webinar now, today, that’s very different than you running a webinar in September. We have a lot more folks staying at home. We have a lot more folks attending other webinars during normal business hours. So, what we’re experiencing is probably not a great time to gather a baseline, nor is it to compare to your baseline that you had in September. And I think the most important thing that I have to say is stop letting other people, organizations, teams, what have you, define your success.

So, if you say, “I want to put together a ratio, a range, a percentage to show, ‘this is what I think is successful, this is what I’m tracking towards,” again, pulling all of that together so it’s not just a number, it’s the context, it’s the language, it’s the other things around will make you a better storyteller in your DevRel experience and make you a better…it will make you a better successful storyteller in presenting that information in the way that you want it to be presented, not how it’s going to be co-oped as you’re just sharing numbers without units or language.

So, that’s all I got today. I know that I have some time afterwards to answer some questions. So, I’ll cut my screen share down so I can actually see everything again. But if I can’t get to your questions or you have some later on, please feel free to connect with me on Twitter. I’m pretty active there.

And if you’re interested in Wowza and all things live video streaming as are kind of important right now, there are some links to Wowza stuff as well. ♪ [music] ♪