Made by 
Made by
March 6, 2018
Events and Marketing Coordinator at Hoopy
Alex Salazar
Making sure organisations get value from their money is a central priority of any CEO. In this talk from DevXCon San Francisco 2017, Alex Salazar, founder and former head of Stormpath, presents a practical guide to the interwoven relationship between developer relations, developer experience and return on investment.
So no pressure. All right, so, yeah, that’s how we’re gonna do it. So look, all of us are here because at some point, you know, some executive in your organisation, the CEO, the VP of Marketing, VP of Product, somebody went full Steve Ballmer and decided that like developers were gonna be critical to the business strategy that they were going to make this big investment and out of that big investment they were gonna get this big return on that investment. And that’s really important because a lot of people in developer relations and developer experience, they’ve been tasked with going out and getting developer love but it was never clearly articulated to them that they’re gonna be measured on return on investment.
And that’s really important because as you’re doing your work at some point there’s gonna be this tough moment at the executive staff level where they’re gonna look at an expense report or the financial statements and ask, “Hey, what is all this money we’re spending on developers? Are we getting a return on that investment?” And if you don’t have a good answer to that then it’s gonna be very hard for you to continue getting resources to continue making the investments you need to make to be successful, and in some cases, you and your team’s jobs may be at jeopardy.
And so I’m Alex Salazar, I’m the former CEO and founder of Stormpath. We built an identity API and we’re recently acquired by Okta. I now run their developer platform building the same exact product. And I’m gonna take you through a few topics to kind of really articulate ROI around developer relations and developer experience and first, we’re gonna talk about how CEOs really think. And now it’s not just CEOs who think this way but they’re really the one person in the organisation who has probably the most holistic view of the business. And then we’re gonna talk about how to think about return on investment for both developer experience and developer relations so that you can build a successful DevRel and DevX program in your organisation. And be confident that you’re doing the best thing for the business.
At the end of it, my hope is that you’ll be thinking about DevRel and DevX in a much more effective way and in a way that you can articulate to anybody else in your organisation who doesn’t really understand developers, who doesn’t understand developer relations-developer experience. So let’s first talk about how CEOs think. A CEO’s perspective is actually really simple. They see the world in three dimensions. Everything that they’re doing, everything that they’re pushing, everything that they, you know, evaluate or prioritise is measured along three axes.
The first one is driving revenue growth. Is what we were doing going to drive the revenue, the top line further? The second is cost reduction. Is what we’re doing gonna make us more efficient and help drive the bottom line? And the third one is, is what we’re doing enabling us to be more predictable or help us reduce risk? Everything a CEO does falls along these three axes. Developer relations and developer experience also need to fall along these axes.
So from a CEO’s perspective, there is no such thing as developer relations or developer experience. What they have is a developer go-to-market. They are taking their products, their services whatever it is that company is there to do and they’re trying to promote it, distribute it, sell it, whatever it is, through the developer community, through the developer audience. Developer relations and developer experience are simply components of an overall go-to-market strategy.
From a developer’s perspective, developer relations is just marketing. It’s just really highly specialised field marketing for a really hard to market audience who hates being marketed to. And developer experience is the product. At the end of the day it doesn’t matter how fast your API is or all of the amazing features that your product has if the developer can’t use it, if he can’t easily figure out how to do what he needs to do in your API that features it doesn’t exist.
So at the end of the day as the CEO is doing his job he’s gonna take everything that’s happening in the organisation and he’s gonna have to ruthlessly prioritise every single minute of every day. He has scarce resources. He’s got really challenging metrics that his board is pushing on him. Every executive in the organisation is telling him that their project is more important than your project and everybody in the organisation who is not in developer relations is looking at your organisation thinking, “That’s not a good investment of our resources.”
Developer relations, it’s really fuzzy, we got an enterprise sales team that’s gotta go do their job. Right? We’ve got this large marketing organisation that we’ve got to go fund, right, we’ve got an engineering team that we need to double because we’re falling behind on roadmap. Why are we investing in developer relations? Why are we investing in developer experience? And so at the end of the day, the CEO’s job is not to play favourites. A good CEO’s job is to look at everything that’s happening and say “Where am I getting the most ROI and where can I keep putting more money into to get more scale?”
Okay. So with that perspective how do we build a successful developer experience and developer relations program? How do we get it right? And not just how do we get it right because we’re gonna build developer love but how do we get it right so that the business is getting value, so the business is getting a return on its investment, so we can continue to invest and our company continue to grow.
Sorry, you can’t do this. So everybody looks at the next, you know, Twilio or Stripe or Heroku and says, “Oh, my god, they’re doing that. Let’s go do that. That’s so cool. It worked for them, they’re publicly traded, they’re growing so fast, their stock is trading at such a high multiple. We’re gonna go copy what they’re doing and we’re gonna be successful.” It won’t work. And the reason is they’re a different company than you. They have a different product than you. They’re solving a different problem than you. They have a different budget than you. They have different competitors than you. Everything about their business is likely very different from every part of your business and if you copy what they’re doing you will fail. So let’s do it the right way. Let’s actually think about what is our strategy? What are we really trying to do with our developer go-to-market?
Well, I’m sure there’s a lot of different ways of doing this but I’ve only seen three models be successful for developer relations. I’m sure there’s more. I’m sure someone will grab me afterwards and be like, “You miss one.” But the three that I’ve seen is you’re either trying to sell your product, which is what Twilio does, what Stripe does, what we now at Okta with our Dev platform are doing, trying to sell your product through the developer. Or you’re trying to build an ecosystem and this is what Apple with iOS is doing, this is what Google with Android is doing. You’re trying to get developers to build on top of your product to create value for your product. The more apps that exist on your OS the more valuable that OS is.
Or the third one, which is not as common, we’re trying to see a little bit more of it lately is I already have a product, it’s already being sold successfully, we’re already in a lot of accounts and I’m now trying to deepen the engagement for that product in the organisation. Box with their platform is a really good example. They’re building at APIs in a developer engagement model not to sell more Box but to embed those seats that they already have deeper into the organisation. So the organisations are just building workflows off of their core product. You have to pick which one you are because which one you are is gonna drive every other part of your program. If your Apple doing what Twilio does it won’t work, if your Twilio doing what Apple does it’s a really bad idea.
The next step and I see this a lot especially from people who are new to developer relations is you’re going after developer. There’s no such thing as like a developer. Right? There are millions of different kinds of developers. There are senior developers, junior developers. There are frontend developers, backend developers. There are data scientists. There are mobile developers. There are architects. There are people working in Java, .Net, Node, you know, Python, Ruby. Depending how you look at your market, depending what matters to you, you need to slice them differently.
The next thing is once you figure out which developers you care about which are the ones you’re really going after. Well, what do they care about, right? You can’t deliver your program developer experience and developer relations if you don’t understand what they care about. A Node.js developer who is working on Heroku has a very different set of concerns than an enterprise .Net developer working in a purely Microsoft stack in the data center. And the sooner that you understand who you’re targeting the more focused and the more bespoke you can be in how you reach those people.
Similarly, where are they? Or if you’re going after, you know, more junior frontend developers that aren’t even one or two years out of school hackathons might work for you, maybe. But if you’re going after very senior enterprise architects you’re not gonna find them at a hackathon, right? You might not find them at the events that you’re gonna find a mobile developer at. You know, you might not find them in the same community, the same forums, the same content, they may not read Hacker News or maybe they do but understanding who they are is gonna help you figure that out.
And once you have a really good sense of who your personas are, which developers are you really going after then you can start to put together a plan of how you go after them, what features you build for them, what kind of experience you’re gonna cater to them, the depth of your documentation. This is the hard one for each of one developer relations, I found this to be a problem with almost everybody I’ve spoken to who works in developer relations. We’re all really focused on driving developer love. We want to drive community. We want to give back, right? We want everybody in open source to love us. And that’s great I mean that’s a hallmark of a successful developer relations program. But at the end of the day, you need to be sober about the fact that this has to tie back to revenue somehow? Maybe not explicitly but at some point, you have to be able to tie it back, otherwise, it would be very hard for you to argue that you’re a good investment for the organisation.
So a general question is how will your company extract value from all the work that you’re doing. Maybe it’s direct revenue because you’re Twilio and Stripe and you’re generating revenue off of people using your service or maybe it’s implicit because you’re building brand awareness for some, you know, for the broader company or maybe you’re Apple and Google and people are now consuming more of your OS which is driving more service but somehow you need to be clear on how you tie back.
For many companies, the developer go-to-market is not the only go-to-market they have, right? They might have a field sales team, they might have a really, you know, a really built out ecommerce self-service model. They might have any other form of partnerships. They might have resellers. Understanding what those are and how your developer go-to-market ties into them is critical.
I’ll give you an example, so at Okta we have an enterprise sales team, field reps, all over the country, hundreds of them. We need to be really clear on where developer relations sit, right? And how it fits into that, right? They’re dying to know what events and conferences we’re talking at so that they can start driving their customers that they have relationships to to those events and show them that we really care about this community.
Conversely, we want to make sure that as organisations, as large organisations like Goldman Sachs and other companies are to come in to us through our developer relations programs that we’re doing a clean handoff if somebody is interested in talking to sales, right? Making sure that we’re not just simply two different parts of the organisation that don’t talk to each other, there’s no Chinese firewall where one organisation trying to hand working it like in a coordinated dance. This is really powerful if you want your program to be successful.
This is also a really important one. You need to be focused on cost. And people in tech often forget the importance of cost, right? A lot of us live off of venture capital, you know, we were…if you’re doing developer relations somebody has probably said this is really critical and they’re giving you a ton of money but at the end of the day you have to be really, really careful about how you’re spending that money, right? Which programs are working? You know, when you’re flying all over the country, all over the world to give talks, which are the ones that are actually driving impact. For developer experience when you’re building up all of this SDKs, all of these libraries, all of this documentation, what’s impactful? What’s working for your customers and what isn’t?
And if you’re doing metrics correctly, and we’ll get to that, figure out what’s, you know, you really have to figure out what’s going on. And if something is working do more of it. And if something is not working stop doing it immediately. Now, this is really hard in developer relations. It’s really hard to measure it. And everybody says “Well, you can’t do it.” You can do it. I think Adam is giving a talk a little later to talk about it, so I won’t steal too much of his thunder. But you have to find ways of tracking it. But here is this thing it’s really easy in developer relations to see a blog post and say, “Oh, my god, we…Goldman Sachs saw this post on Docker and they came into that post and six months later they bought our product. Therefore, we should write more about Docker. Well, was it a fluke? They just happened to come in through Docker? Do you expect JP Morgan and Bank of America and all the other banks to come in through that same post or similar content?
You need to know these things to figure out where to invest your time and money because DevRel resources, one, are really hard to find. They’re really expensive and chances are you’re really busy. So from a metrics perspective please track your metrics. The biggest problem we see was in our own organisation is we were scaling Stormpath and now that we’re at Okta, and the biggest problem I’ve seen with all of our peer companies that we’ve been trying to learn from them and share knowledge is not just that people aren’t tracking metrics, it’s that people are tracking all the wrong metrics. At the end of the day, developer relations is a go-to-market strategy, developer experience is a product function and it should be measured as a business unit, it should be measured based on return on investment. So common mistakes we see often are measuring developer relations by Facebook likes, Twitter retweets, GitHub stars, poll requests, that’s all great. That shows a health of a community, that doesn’t show a return on investment on your developer go-to-market.
So here are some metrics that I think are important, the metrics that we track. The first one is signups, or actually the first one’s not on here, it’s visitors. Are you simply bringing in people to see your stuff? Are you getting reach? From there, are you getting signups? Are people compelled enough by your message, by your product, by your features, by everything that you’re doing that they want to sign up for your service? And then once they signup, are they actually using it, do they activate? Does a developer take the time to bring down your library, install it, open up a terminal or a command line and actually make that very first API call? Or maybe your product is different, there’s some other activation point, but are they getting there, are they putting the effort in?
And then once they put that effort in are they engaging? Are they using the features? Are they trying out feature X, Y, and Z? How deep are they getting into the product before they balance? Which is the next point, which is retention. Are they sticking around seven days later? Are they still around 14 days later? Are they around 30 days later, 180 days later, a year later? Is what you’re doing working?
NPS or referral is really important because the develop…the reason we’re all here, the reason developer relations even exist is that the only tried and true way people have figured out to market to developers is through word of mouth. That’s why we do what we do the way we do it. And so do you know if you’re generating word of mouth? NPS or some sort of referral metric is a good indication and for those of you who don’t know, NPS is net promoter score. This is where you ask somebody who signed up how likely there are to recommend your service to somebody else, you should be tracking those. Otherwise, you don’t know if what you’re doing works.
And then finally, revenue. If there’s a way to tie back to revenue, you should find a way to tie back to revenue. This is really hard, it requires a lot of systems integrations. It requires a lot of labor. At Stormpath, you know, we’ve invested heavily in this and we could ultimately tie back a dollar spent to this service to the event you found this at. Or we could tie you back to a particular blog post that you came in through. And if you touch multiple pieces of content before you purchase from us we could see that while trail and we could put dollar amounts on individual SDKs and individual framework plug-ins, so we can know where we were getting ROI, and we can invest more or less as we were coming along. Extraordinarily hard to build but worth every single penny.
So getting ROI is a big piece of it but it’s not enough. You have to advocate internally. You have to make sure that people know where your ROI story is. If you’ve got metrics, if you know how things are going, communicate them up, down, to the left, to the right, to everybody around you. If the numbers are great, scream it from the rafters. If the numbers suck, be open about it. So everybody knows that you are working on it and not hiding the ball. Educate the people on your team about your focus on ROI, make it clear to them that they’re not there just to get developers to like them on Twitter or Facebook, you’re there to drive business. Educate your peers, people on the other teams, the sales organisation, the marketing organisation, the finance department, educate them on what it is you actually do. Developer relations it cannot be this dark art that only those chosen few know about, everybody has to understand where it fits and how they fit into it as well.
Share your metrics across the board, share them, if you can put a public dashboard the one like Periscope or grow.com or something like that do it and let everybody have access to it that way everybody knows what you’re doing. Then finally hold yourself accountable, right? It’s not marketing’s fault, it’s not sales’ fault, right? It’s everybody’s fault. And if your numbers aren’t working own it and try and work on it like DevRel is owned by DevRel, DevX is owned by DevX. So it’s easy to stand up here and say all these things. You know, these programs are really hard to build, I think all of us are in this room today because we’re hoping to learn from each other because this is hard and it’s a dark art. There’s no books written on this and we can’t just copy Twilio.
Join our new journey
So, you know, we’re in an interesting situation where at Stormpath we built a lot of the stuff out and now we’re part of a larger organisation and we have this interesting opportunity, we have to go rebuild it from scratch. We have to go rebuild developer relations and developer experience at a brand new organisation from scratch. And so one of the things that’s interesting as for those of you who care is you can follow us, you can go check us out. We’re gonna be talking openly about our experiences that we’re having to build out developer relations and developer experience for our developer platform. And, you know, we’re making a really, really big investment. We’ve talked about this publicly, we’ve talked about it, you know, in a number of press releases. And we’re just getting started. In fact, our Twitter handle literally just went live today.
You know, and so follow along, follow us on Twitter, follow us on you know, either, you can email us as well with questions, go to our website. Our developer experience today is not great, you know, we did a tremendous amount of work and we’re really, really effective with the very limited resources that were there before this big investment of Stormpath but we’re gonna be making major investments.
So, you know, don’t judge us based on where we are today, give us feedback. Send it for the service. Go through the flows. Tell us where developer experiences can improve. Follow us on Twitter, tell us where developer relations is not working, where you think we’re off the mark, where you think we should be focusing on. And then follow our blog because we’ll be talking about the mistakes we’re making along the way, the lessons we’re learning because one company is not the same as the other and so there are mistakes we’re already making and we’ll be very open about it. Yeah, and keep us in mind.