Made by
Made by
Elmer Thomas
In October 2017, SendGrid’s goal was to receive 80 PRs at Hacktoberfest. They received more than 1,000. In this talk from DevRelCon London 2018, developer engineer experience Elmer Thomas explains how they handled the overwhelming response.
How many here participated in Hacktoberfest this year? Awesome. How many people know what Hacktoberfest is? Very cool. Very cool.
So this talk is designed to help you in a couple of ways. One, to learn how to use Hacktoberfest as a tool to grow your open-source communities, but also a lesson in systems exploding and how you can try to avoid that for unexpected success.
So, the Hackening. I’ve learned to call Hacktoberfest because it’s truly spooky and scary for us because we actually start planning now since February after this first happened to us because last year we were expecting 80 PRs and we had over 1,000 and we didn’t know what to do.
I curled up in the foetal position for about a week, and then eventually, I figured out, “Okay, we have to do something.” But what you’ll learn as we go through this experience, it doesn’t have to be spooky and scary. So a summary of the talk is I’m going to give you a brief history of Hacktoberfest in general and specific to SendGrid. And then I’m going to go through some of the points about planning for Hacktoberfest so that you can make it a successful one for your open-source projects.
And then also I’m going to talk about how we execute and how we learn from that first year to this year. There was a dramatic difference in how we executed the first year to the second year, mainly because of the learnings and the failures that we had in the first year. So I’m hoping that by me teaching you these things, you can skip the whole massive failure part in the middle.
All right. So who am I? I’ve been with SendGrid since near the beginning. I joined about eight years ago. I started off in product and then QA. And then I was a DevRel for a couple years, travelled a lot. And then I settled into a developer experience role, which means that I manage the open source repos for SendGrid.The main ones we have, they covers the seven most popular programming languages, and they provide an easy way for people to interact with the SendGrid API.
If SendGrid was still a start-up, this would be my title. And back in the days, everyone in the start-up would make their own title. Mine was Hacker in Residence, but then we became official and so I had to become the developer experience engineer. Very super boring. This is my preferred title.
All right. So this is a summary of the history. So I’m going to go through the origination with DigitalOcean, GitHub, and then SendGrid, and then how Twilio came in this year and joined in in the fun. So 2014, DigitalOcean started this thing called Hacktoberfest. They had a pretty interesting model. Back then, we had to do 50 or more commits.
As you may know, now it’s about doing pull request. But back then you had to do 50 commits, and they had 768 participants. You know, pretty nice for an opening try there. Then they joined up with GitHub. And as you can see, the signups dramatically increased, you know, talking almost 15,000, that’s pretty amazing.
And look at that number. 5,700 people made four more PRs, that’s tremendous. That’s a lot of contributions. That’s a lot of developer love there. I think of PRs as an expression of love from a developer because they’re giving you their most precious thing, their time, to contribute to your project. So I think that’s super special.
And the fact that they had so many, that second year, is amazing. And then, so one day, I met at the office and Sean says, “Hey, have you heard of this thing called Hacktoberfest? But no I didn’t. He says, ” Well, you really should because it’s pretty awesome.” So he introduced me to it. And so that first year, we labeled some stuff, said, “All you have to do is just put the Hacktoberfest tag on some issues and then see what happens.”
So that’s what we did. And we had 43 contributors. And back then, that was amazing for us, because maybe we’d get 5 to 10 contributors a month. So getting 43 contributors we’re like, “Wow, this is magical. Maybe next year, we should actually, you know, make this an official thing.” And so in 2017, we had a goal of getting 80 PRs and that was the first year where things just exploded, and we had over 1,000 PRs that year, 600 contributors.
And that was this shirt, our amazing… I have another slide that gives credit to the creator of that shirt. But that year was when we really saw the potential of Hacktoberfest and what it could do. And then finally, just to kind of compare with DigitalOcean/GitHub that year, look at what they did.
That’s a lot. I don’t even know how they even begin to deal with that scope of participation. But it’s a testament to how many people are interested in contributing to the open source community. And you should definitely take advantage of this. So this year, the official numbers haven’t crystallized yet, or maybe they have and I haven’t seen them yet.
But, you know, we just finished 2018. I can say that talking to Daniel at DigitalOcean, they already eclipsed their numbers like two thirds of the way in. And similarly, we also eclipsed our numbers from the previous year. So I know this year is just another amazing year, Twilio jumped in and added some support. It was really another just phenomenal year.
But this year, we had a much more sane experience. I didn’t feel as crazy this year, and so the rest of the presentation is going to explain why. So the first thing we’re going to do is talk about planning for what we call the Hackening. And I’m going to go over the swag, the internal and external marketing, and then the most important part, in my opinion, is the actual issues and all of the stuff that goes around the issues that you put out there for your project.
So the first thing is with swag. So we use a service called Kotis. Kotis is amazing because they have an API, swag is an API. How cool is that? You can actually just hit their endpoint then magically, your swag will appear at its destination. That is super awesome. And they also provide the swag for the official Hacktoberfest competition.
So we figured if they can deal with tens of thousands of swag, they can certainly deal with our thousand. So I want to give credit to Virginia and that’s her dribble there. She created this awesome, amazing design. And if you look at most awesome designs on our site, she is responsible for that. Super cool. So, qualification. So you saw in the very beginning what GitHub did as they said, you know, 50 commits and then they switched to the PR model.
And last year when we first got those thousand PRs, we only required people to do one PR and then they would win some swag from SendGrid. This year we decided to switch it up because we wanted to focus more on quality. And we said, “Okay, we’re going to require five easy PRs, two mediums, or one hard one.” And our goal was to see if we can increase the quality.
In fact, we did. With like half the contributors, we receive double the amount of story points and gave away, I think, this year, a little bit under 200 items of swag. So we give away a shirt and we also have a sticker with this logo on it.
So the one of the big problems that we had is fulfillment. That first year, we were using a spreadsheet to fulfill the swag. And you can imagine if all of a sudden you have to send a hundreds of pieces of swag. That became a very annoying thing, very tedious, very time consuming. So of course, we wanted to figure out how we can automate this. Fortunately, our partners at Kotis, they gave us an easy way to be able to do this where now we could just send out the codes to people and then they can go to the Kotis website and then directly order their things.
Because we saw last year, we had a lot of strange addresses. We had some addresses where we would say things like, “Go to this gas station and make a left here and then drop it off there.” Like, I don’t know how to do that. So now, we don’t deal with that anymore. We just outsource that over to the fine folks at Kotis and that saves a tremendous amount of time. So whether or not you use this company or whatever company you do, make sure you think through your swag strategy and kind of the budgeting is also key.
And also make a statement as to what your limits are like. So like you can only give away 100 shirts. Be sure to be very clear about that because one of the biggest feedback we get is when people don’t get their shirts or they don’t get their shirts in time, you’re going to have a lot of people complaining about that. So make sure you spend the time to get the swag piece right.
While it’s about contributing to the open source community and giving back, it’s also about the shirt. People want to have the shirt and they will email you every day until they get their shirt.
And so next thing pieces internal and external marketing. So one of the things that we found is that when you have that many PRs, code reviews becomes a problem because there’s not all quality stuff.
You don’t just want to go willy-nilly merging everything just because they made a PR. And just to have a couple people doing code reviews, that’s non sustainable. What we did this year is we used some internal mechanisms. Of course, being an email company we would be remiss of not using email. But we also use HipChat, we just recently went to Slack, so it was kind of a mixture there.
And then we had an internal hackathon. And so all of these things and then combined with some additional swag for internal gridders. So what we had is the steins, beautiful steins, with the logo on it. And what we did is we said the top 20 people that contributed would be able to get this stuff. And so one of the things we’re able to do this year that we couldn’t do the year before, is actually programmatically determine how to give points to people for doing code reviews.
Because if you use an, I don’t know, v3 of GitHub API does this. But v4, when you do your call there, you’ll get back a list of all of the reviewers. And so you could use that to programmatically assign points. And that’s what we did to help configure our leaderboard at the end there. But it’s key to get buy-in from the engineering organization, because if you’re going to ask them for help, you know, normally, October is a very busy month for developers.
And if you’re asking them to take time from their projects, that’s why we did a hackathon. We had one day in each of our offices where we basically had like an office hours, and we said, “Look, if you can just come in, spend 15 to 30 minutes of your time, we’ll bribe you.” And in our office we bribe them with Chick-fil-A. And so look, you can have some chicken and cookies and just do a couple code reviews. And it turned out to be amazing.
I think this year we had over 20 internal volunteers. And without their help, it would have been really difficult. One other note there is with labeling. Actually, I’ll get to that a little bit later. So externally, what we did is we, we have a podcast and we promoted it there. We used Twitter, we used our blog, we wrote a couple blog posts, we used Facebook, and then a steady stream of issues.
So one of the things you’ll notice, if you go to the official Hacktoberfest website, there’s a link there that links you to the page on GitHub that has all the issues that have the Hacktoberfest label. And by default, it shows the most recently submitted things. So every time you submit issues, it pops up there on that front page. So we figured out this year is that instead of just putting all of our stuff out there right at the beginning, we trickled it out day by day.
And we just used a basic spreadsheet for this and we just did I think about 20 to 30 a day. And then what we saw is like sometimes you put stuff out there and literally within seconds people will pick up your issue. But you have to trickle it out because otherwise, it gets lost in those, you know, tens of thousands of PRs that… I mean, tens of thousands of issues that other people have put out there. So with the issues, there are some tips I’d like to give you for how to make good issues for Hacktoberfest.
Remember that when somebody is going to contribute, they’re going to see a list of all these different issues. And so it’s very important to have a catchy title. You know, you might want to talk to your marketing department, get some catchy titles, whatever you need to do, but make it very clear in your title, what you’re asking them to do. And just think about, like, “If I were going to do an issue,” and keep in mind you have to do five, and I’m trying to pick ones that I can easily do, I want to be able to quickly and clearly look at the title and figure out what your issue is about.”
And then that nail it further in the brief description. Keep it very brief, no more than a couple sentences and then finally labels. And so next slide, I’m going to talk a little bit more about our labeling strategy. But the labels are super important for a couple of things. One, for helping discoverability. But also later, when you have to do reporting and when you have to do automations, it’s very useful to have a labeling strategy for that.
And then finally, acceptance criteria. This is where we got bit the first time a lot because if you’re not very clear with what constitutes acceptable PR, then you’ll get a whole bunch of random things, you know, because a lot of people are just quickly going through just trying to get their five PRs and be done. So you should really take the time to make sure you specify what the acceptance criteria is.
And again, the steady stream of issues, you want to constantly be putting new issues out there. And then we put together a dedicated categorized page. So this page allows people to… Let’s see if I can tab into there. So what this page does is you can easily click it and go to what language you’re comfortable with, whether you want an easy, medium, hard, or very hard.
And then, so this is an auto-generated page that gets refreshed every hour. So people would always come to me and say, “Look, I want to do something easy in Python.” So before I would have to like, go look and find them. So now they could just go here and get their things. And just while I’m in here, this is what our leaderboard looks like.
So then with the execution, I’m going to go through real quickly our labeling strategy, the dashboard, and reporting.
So one of the things that.. And I forget I’ll have to put the credit in the slide notes. But I read one article about how to organize your tags in on GitHub and I really fell in love with it. Basically, it has that you have three different types of labels that you should apply to every issue. First is difficulty, and we use this to kind of gauge the points, but then you also want to have status.
So is it work in progress? Does it need review? The status that helped us the most this year was a status called “Hacktoberfest approved” because what happened last year is that we mistakenly thought that you had to actually merge the PRs for them to count and it caused the big ol’ mess towards the end there. We were like scrambling trying to make sure everybody got their credit. But this year, using the internal help, we said, “Okay, if it just looks like a legit PR, just market status “Hacktoberfest approved”and then at the end of the competition, we’ll just run our script against all those labels. And that helped us out a lot.
The type of issue like, is this a bug? is this a feature request? that kind of thing. And then GitHub allows for these beginner labels like up for grabs, or good first issue, or good first PR, you want to use those to help people out. And then for the dashboard, I showed you the internal and external leaderboard.
Code reviews. So we have separately generated page that helped our internal people go in, say, “Okay, I’m good at Python. I can review Python so they can just click there and then they’ll see all the issues that needed a code review in there.” So for reporting, so you’re obviously going to have to report out your results here. So one of things… Again, that label, the status “Hacktoberfest approved” really helped a lot because then I can easily determine what were the valid open PRs.
You can mark PRs as invalid that will cause them not to get credit with the official Hacktoberfest. And that’s good for spam. number of contributors. Number of story points for PR, including the code reviews. So we have a script that does all that stuff and we’ve open-sourced it at the end here on the slides to have a link to it.
It’s called the DX-Automator, and it uses the GitHub v4 API. It’s all dockerized, very easy to install, and you can actually run these reports yourself. And then you receive swag recipients, in our case, they needed five points or more. And then the leaderboards. You’d be amazed at how people they… the leaderboard is really effective. I’ll just leave it at that.
So little story here. You know, there’s nothing to fear with the proper planning. My daughter here, she used to be scared of these things, so I would use them to make her not go up the stairs. But then eventually, she decided, “You know what, these things are not scary.” And then the other day I actually had five of these things. And she had all five in a group hug and she was doing some sort of dance with them. So, yeah, nothing to fear here.
And, yeah, thank you for your time and attention. I really appreciate it and I hope that you have a successful Hackening next year. My takeaway for you for this would be start your planning today because it’s actually it needs that much the time just goes by like that.
But wait, there’s more. So the DX Automator I talked about, you’d be able to click in here and go and it’s all open source, is written in Python. There’s a D-Zone article that I wrote called SDKs and World Class Developer Experiments. That will help you create your repo and make it prepared and it has all that labeling strategy stuff in there.
And, of course, the obligatory, we’re hiring. My team in specific is looking for someone to help us manage and grow all this open source goodness. So thank you very much and hope you enjoy the rest of the conference.