Made by
Made by
Tristan Sokol
In this talk, recorded at DevRelCon London 2017, Tristan Sokol lays out how to make the most of Stack Overflow to support your dev rel program. He also touches on some useful analytics functionality, and other ways you can make using the platform a smoother, more efficient experience for your developer community
When I originally planned this talk out, I had sort of structured it all around some of these advanced concepts that I and my team had worked on at using Stack Overflow to manage our developer relations program. But you know, talking to people today, I realise that maybe some ground rules would be a little more helpful. So I’ll be talking a little bit about the basics, some of the stuff that I think is probably opinionated in to how we use Stack Overflow to support our developers, but hopefully we can get everybody on the same page. Some cool stuff you can really measure, particularly around the Stack Exchange Data Explorer, so if you’ve never used that, look forward to learning more. And some topics about cool stuff that I personally haven’t done, but I think you can have great fun with as well.
So, who am I? I work at Square. I’m on our developer relations team. Up until last week, I was the only person on our developer relations team. Now I have a couple of friends. You can see in my profile that even though I may be in the top 4% this year, I still have a paltry 2400 reputation. My tags are mostly revolving around questions that are asked about Square things, and I’ve also given some advice in home improvement and the Cooking Stack Exchange. So, yeah. So hopefully most of you are generally familiar with the concept of Stack Overflow. It’s a question and answer site, particularly just geared towards developers. The Stack Exchange networks have expanded into other concepts as well, like cooking or home improvement.
But Stack Overflow, by far the most popular and, you know, possibly the number one or number two most popular developer site, depending on how you conceptualise those things. The idea of 51,000 developers looking up answers right now is pretty impressive I think. And so, just to give you a couple of the basics about Stack Overflow, people ask questions and people answer them. You know, it could be different people, could be people all from the different communities.
One of the ground-level concepts that I think is very important to understand is this idea about tagging. And so tagging is like one of the most basic metadata features you can apply to a question. Hopefully this is sort of old hat for a lot of you. But a lot of my opinionated aspects of this talk are all about having tags for your products. So in general, tags should represent individual products that you support. If you are a…perhaps not as well-known in the developer spaces as Google where you have Google Vision API as a tag, and then Google Drive API as a tag. We just have sort of a one over-arching tag for all of our Square things, the Square Connect tag. So that might be a good strategy for you as well, depending on your size. But essentially, tagging is very important to all of your questions.
And then the other basic level Stack Overflow concept there is this concept of points. On Stack Overflow it’s called, “Reputation,” so you can get points for being better community members, basically. There’s a lot of different ways you can earn and lose it. Basically, you get points whenever you ask good questions or answer questions that the community likes. And then you lose points whenever you do things the community does not like, which as a person supporting developers on Stack Overflow, you are at risk of doing all the time. But there’s also ways to farm reputation if you really want to, and there’s a lot of documentation on that should you be interested in that domain.
Reputation really isn’t very important except for the benefits that it gives you. And so these are a little bit small, I know. But sort of the ground level, you start with one, and then as you work your way up on the reputation scale, you get the ability to vote questions up and down. This is an important way to signify questions that you, who are probably monitoring some tag of questions that come in related to your projects, can sort of verify the quality. And you can support content you think is good or you can support content that you think that your friend in documentation wrote. You kind of have a way to bubble up the things that are most correct answers for your community.
And then voting down is also a way to actually push away content that may be unappealing. So you know, a lot of times community members on a totally unmoderated space can answer questions that seem not really in the best good faith into answers. You know, you might have a question that asks, “Oh, you know, I’m trying to use Square as my payment provider. How do I do this?” And then someone says, “Oh, you know, here’s two other APIs you can use instead.” And you know, you may want to vote those up and down depending on your position or employer.
One of the funny anecdotes…you know, at 50 rep, you get the ability to comment. And so if you’re not familiar with answering Stack Overflow questions, basically when you answer a question, you’re supposed to have an answer there. But a lot of times, people ask questions and they don’t really include enough information about the question to actually answer itself. So you need to comment that, and say like, “Hey, can you give me this additional piece of information?” We had some people join our API support team who had never used Stack Overflow. And so them not having enough reputation to actually comment made them fairly useless to support this community until they were able to farm enough reputation to do this. So it’s something you need to consider whenever you are moving into the space I think.
And then some of the most important ones, I think, for someone who wants to really effectively manage a community here or support a developer community there is really all about being able to edit questions and answers. So I have enough reputation to where I can edit willy-nilly your questions and answers and no one checks that. And that is great because you can add links to documentation to things, you can edit things for grammar and spelling, and I’ll talk more about that in a bit. And apparently you get access to Google Analytics if you get a lot of reputation, but I don’t know anybody who’s done that.
One of the things I also wanted to sort of highlight here is that it’s very important to guide your developers to Stack Overflow only in times that it’s appropriate. We have a lot of things that push developers to Stack Overflow, and a lot of times they ask questions that are not so Stack Overflow related. And community members do not like that and it’s not really a great support experience. You know, if this is someone onboarding into your product, and then they, you know, get, “Oh, I’m supposed to ask questions here,” and then they get all sorts of sad comments and down votes and their question gets closed.
You can sort of figure out where in your developer relations morals you should walk the line about how you support things like this. But one tip I do have is that if you answer your questions very quickly, generally people who are more moderator-focused are looking at questions that are unanswered. And so if you’re able to answer your questions in your community quickly, you can hopefully mitigate some of these bad experiences. So now that you have some groundwork about what Stack Overflow is and how to use it, how can you really use it best to support developers?
The first one, I think, is probably somewhat obvious, but here are some tools that we use to actually tell developers about the tag that you use. So you want developers to sort of all be focused on one area in Stack Overflow. And so some of the things that we do is that we actually include a link to our Stack Overflow tag at the bottom of your documentation page. We also have this great, you know… If some merchant, this is sort of the merchant-oriented support flow, but if you’re able to say, “Oh, I need help,” and you happen to pick developer APIs, Stack Overflow is the first one that you see, and it goes directly to our tag. Emailing support’s kind of the other option because we don’t really have a way for developers to call us and get good support. So we sort of put those developer-oriented products, so first and centre for support options. And then we also have a support page in our documentation. And we, of course, reference what kind of questions you should ask on Stack Overflow, so really the more technical questions. Whereas the more proud, conceptual questions about, “Oh, I’m trying to build this integration. Does anybody have any tips about is this a good product market fit,” that’s best left for another community that you may support.
Question maintenance I think is probably the number one job of somebody who works on developer tool or API and wants to have developers with their product have a good experience, things like updating old questions. We release new products all the time. I’m sure the things that you work on release new features and products. And so when a developer asks a question, “Oh, is this possible with your product,” and the answer is no, eventually, the answer might turn to yes, and you need to update that question. Because really, you might think that Stack Overflow is a question and answer site, but really, it is a piece of scalable documentation that the majority of the people looking at it are people looking at it months after the question was asked, years after, possibly.
And so if you have broken links, if you have things that are inaccurate or factually untrue, things that have changed, or product renames, those are all things that you want to try to edit as quickly as you can so that you keep that information correct because all of your Stack Overflow posts have a ton of Google juice. And so you want to keep that stuff up to date. So I mentioned some of these.
And then this is an example of one of the pieces of content I edited personally. So we had a post three years ago when our first APIs were created, before our developer platform really started, and it was like a side project for one of our engineers. And it was about sandbox accounts, and we didn’t have any sandbox accounts at that time. And the question was, “No, we don’t have any sandbox accounts.” But then as we produced sandbox accounts for developers, editing this question was very important because this is…we’ll get to it later, but this the most viewed question that we have. So this question, out of all our Stack Overflow questions, has the most views, and so it’s a piece of content that we should keep updated and correct.
Re-tagging is another question maintenance concept I think can be really useful. So if you search for Square on Stack Overflow, you’ll find square root, you’ll find square brackets, you’ll find Square, maybe the API. You might find that. If you search for Square API even, you’ll find Foursquare API, you’ll find our API a little bit. But if you search directly for our tag, then you can find all the questions that relate directly to our product. And one of the things to help you…that we use to help you take questions that are asked, that aren’t maybe tagged correctly, is to use Google Alerts. And so I get alerts whenever somebody asks questions about our APIs. And if it’s not directly related to our Stack Overflow tag, you know, I try to re-tag it to get that into all of our good data analysis.
Editing, another core concept. I’ll just sort of go through some of these. You know, people are on Stack Overflow from all over the world. And, like I said, these are pieces of scalable content that people find generically, or generally from scalable search. And so if you don’t have good SEO on your questions, then people are gonna ask the similar questions. Because generally, I think all the developers in the room probably know, when you find a question on Stack Overflow, it’s because you searched for some problem that you had with an API, or with a tool, or with a product. And so if you don’t edit those questions so that they have better SEO and really mention the products that you use, and have all the good keywords in there, then you’re just gonna have a lot of repeat questions as opposed to sort of funnelling those into your scalable content that you already have.
I’ll give you kind of an example of a question that I edited, and you might feel that it’s good editing. You might feel that it’s too much editing. So we had a developer ask this question, and you know, you can kinda tell that maybe this developer is not a native English speaker. You can see that there’s maybe some opportunity to add some better formatting in terms of you know, the grammar and spelling. And also, you can see that even though they’re asking a question about PHP, they’ve added this great detail about they know how to do it in JavaScript, which may be great for them to say, but it’s not really relevant to the question.
And so you know, in editing I made the title much more understandable. I sort of edited the formatting of it so you can see some of the sort of functionality that Stack Overflow has in formatting your questions, added some keywords like the PHP SDK, so when somebody else looks for, “Oh, how do I get error detail messages with my PHP SDK on Square,” all these good terms are already built into the question so they can hopefully find that more easily. I also added the PHP tag in case developers in the future want to see things just sort of related to PHP and Square.
And the final piece of advice, generally, is just answering your questions. If you wanna support developers in a community like this, you either need to ask the questions yourself, have somebody on your team answer the questions, figure out ways to help community members answer those questions. And to help you do that, I will dive into some of the tools that we use to make sure our questions get answered and our community gets supported well.
Stack Exchange has built-in email notifications. This is something that we use on or team. So you can sign up for basically a query. So we have queries about Square and Square Connect, our tags. And basically, whenever a question gets asked, we get an email about that and our supporters get email about that. So they have a notification that there’s something that they should look at, too, and answer.
RSS feeds, you know, Stack Exchange supports all sorts of RSS feed technology. That’s actually how we get our questions into Slack. So if you have an internal chat tool, this could be a good way to get notifications about questions, and then we can discuss them right here, see who’s gonna answer them, and sort of get a consensus about the answer before we have you know, three independent supporters kind of working on them independently.
And then the last tool that I haven’t used a ton, but I think you’ll have great success with, is the Stack Exchange API. This is something that is very powerful, but possibly too powerful because all of the needs that we have are already met with some of the built-in integrations.
And another thing I wanted to mention was Nicholas from RedHat, our comrade in arms, probably in this room, he’s built this cool Slack integration with Stack Overflow. So you definitely can check that out if you need a more robust solution than the RSS feeds. Oh, and then the other cool thing that Stack Exchange offers is the mobile app. So I personally use this if a developer is, you know, sort of a lot of back and forth on commenting and they have a comment that I can quickly respond to. You get a notification and you can just reply to that comment very quickly from your phone, which is nice.
So you know, the tag info page about your tag is somewhere where you can actually get a lot of interesting data as we sort of shift our focus to data. Here, you can see not only some of the related tags, so you can see that a lot of developers using Square or having trouble with Square, depending on how you look at it, are interested in PHP, and iOS, and JavaScript, and you know, Android, and you know, C#’s way down there in comparison.
So you can kinda get a good ad hoc understanding about either how large your audience is in these different groups or how much trouble these different groups are having with your products. You can see some of the all-time contributors and the recent contributors. This is not only a good place to possibly identify people who work on your team, but also is a place where you could source community managers, right? If you wanted to have somebody who was a little more scalable, maybe, you know, maybe you pay them five bucks a question that they answer and you don’t want to answer those questions yourself, you have somebody right here who has already demonstrated value in your community there and would probably be willing to write a blog post for you, engage with you in their local community, you know, all the other things that sort of scalable evangelism can take hold of.
And then now I want to jump into the world of data science. This is originally sort of what I thought would be most interesting to people, but it’s one of the more advanced topics on Stack Overflow. So if you’re not here yet, I think you should have a lot to look forward to.
So this really focuses on the Stack Exchange Data Explorer which, if you’re not familiar, is essentially a SQL console for Stack Overflow. And so I’ll just show you some of the stuff that we’ve done with it. You can look at your number of questions. It might be interesting just to export your list of questions out to another service or an Excel sheet so you could graph them or sync them up with some other data, right? You can see that over here in the bottom left, the X-axis is months and, you know, Y-axis is question volume. We’re slowly getting some you know, increased question volumes as time goes on. And this slide’s just weeks.
But you could use this data to possibly compare yourself to a competitor. So you know, you could think about Square, we have lots of competitors in the ecommerce space. We might look at this data, look at their data, see how, you know, how does our question volume compare. How good are we doing in support to compare to some of our other people? We might compare you know, how does our question volume in this space compare to like the…all the questions for JavaScript’s. You know, if we look at some of these more macrotrends, we can start to identify possibly areas that we can improve, or you know, possibly areas that we think we’re doing very well in.
Response time. So far, I haven’t met anybody who’s measured this on Stack Overflow. And with this graph I hope you can identify when I started working at Square. This is something that is a, you know, metric that’s, I think, very popular in the support world, right? So you have time to first action and time to completion, so you can measure that on Stack Overflow with a pretty simple SQL query. And you can see that since having an increased focus on Stack Overflow, our median response time has gone from somewhere in the, you know, maybe 4000 minutes, which is a few days, probably, into the, you know, under a couple hundred minutes. So within a couple of hours, developers can get a good answer to their questions.
And then you can view that data on a weekly level, or you can also just sort of measure that in a holistic number. So like the JavaScript tag, questions on JavaScript, in median, get answered within 15 minutes, which is kinda crazy when you think about how many questions there are that get answered in the JavaScript world because there’s just so many of those questions.
Another thing that we really take a lot of measure on is question views. And so, you know, you can sort of pull those out as a list. You can measure that as sort of an all-time… You know, I assume you measure sort of documentation views. So you know, this is another piece of content scalable documentation. So measuring those views over time I think might be able to serve you well as you justify your efforts in that domain.
You can see that…so in this graph down here, the X-axis is the date the question was asked, and then the Y-axis is how many views it’s gotten. And so even though in our previous graph we saw that Square’s getting more questions asked in the last couple of years, really, our views isn’t changing so much compared to when…you know, some of our earliest questions have the most views. And that really speaks to sort of the Google juice power of Stack Overflow. You know, a lot of the times developers can be interacting with this content is not within the day that the question was answered, not within the week. It’s possibly years after when they search for something and they happen to find this content.
To justify that case a little bit more, this is the views, same graph, just for JavaScript. And so you can see JavaScript has a lot of questions, but you can see that there’s sort of a linear trend here as the… You know, even the questions up until 2015 haven’t gotten as many views as some of the questions there have been in the past. And so really, it’s that ongoing SEO factor that leads to adoption of your questions.
You can even do some fun queries like seeing the most common words in your questions. I didn’t filter out any of these filter words, but you can compare that to some of your competitors and see are people asking more about how they do things or errors that they get. You can see that, you know, we have this sort of brand name, Square Connect, and so that comes up a lot in our questions. Our URL is squareup.com and so the word squareup comes up a lot. And so developers actually believe that our product is called Square Up, which some of you might believe that. We’re called Square. And so it’s interesting to see how that is sort of pervasive in our community. And so it’s kind of an interesting thing you can monitor in real time.
And then like I’ve been talking about, comparing our question volume to people you might compare your company with or even just related tags I think can be very useful. We use that internally quite a bit. And doing competitive analysis, like I said.
Some of the sort of advanced topics in the last couple of minutes I have at DevRelCon Tokyo, one of the sponsors there was this company called Teratail. And they claim to be the Stack Overflow of Japan. So depending on your users, Stack Overflow may not even be a great use case for you. Maybe you should look into other, similar exchanges that have the resources that your developers actually need. I think a whole talk could be made about seeding your own questions and answers, which is something that is totally within the rules of Stack Overflow, though you have figure out a good way to make it not seem spammy and make it seem worthwhile to your end users, but it’s something you can do.
Stack Overflow ads. This is something that we tried out. It may work well for you. We tried it out. It cost five grand to get a sales rep about it, but could be cool for you. One of the really cool things that we want to experiment in the future with is creating…so you can basically create and own a Stack Exchange Network. So if you have a cool product like Nexmo up here in the front, Nexmo could have their own Stack Exchange Network similar to how Magento has their own Stack Exchange Network. And so there, the community’s all about Magento, and so you don’t have to worry as much about non-developers or maybe novice developers getting their questions really down-voted to eternity when… The whole community there is all about Magento and then you can ask more freeform questions, maybe not as technically oriented. So if you have a large developer base, basically, it takes a lot of activity and developers to get through Area 51, but if you have that, then that, I think, is an excellent option. Facebook has that, a couple of other developer programs do that.
And lastly, one of the cool techniques that I haven’t experimented with because I’m still very stingy on my Stack Overflow points, but you can offer bounties up. So you can basically give extra points to questions. And so the idea there, I think, is all about…you could see the question about, “Oh, how do I use my new API with this product?” And then you put a big bounty on it. And then there’s a lot of bounty hunters out there on Stack Overflow, and so then you have a developer that is incentivised to go try out that new product, see how it works with integration, answer that question in detail and in depth. And then you just give them some Stack Overflow points, which you earned very easily because you’re answering all these questions, and then you have not only a piece of…it’s kind of like a blog article about…a mini blog article about you know, how to use your API, your product with some new technology, but you have a developer there that’s also tried it out themselves and been able to have that experience. And hopefully that’s a learning situation on both sides.
So I want to thank all of you. You can reach out to me on Twitter if you really want to. I would love feedback on this talk. I know it was very fast-paced. I also make a lot of blog posts on Medium. You can see me on GitHub. This presentation and a lot of other ones about Square can be found online, and so all the links to the different SQL queries and things are all linked into the presentation. So if you want to go in there and…all you have to do is basically paste in a different tag. And you can see, you know, how many views per month does your tag get, how many views per month are PHP developers asking questions on Stack Overflow, things like that. And then for those of you who have traveled several thousand miles to get here, I also am now running the SF Developer Advocates’ Meetup group. And so if you’re an SF, I would love for you to join that. And that’s it. Thank you.