How Indeed more than doubled Hacktoberfest participation year over year

Indeed’s open source programs office have encouraged their colleagues to contribute to open source during the past two Hacktoberfests.

In 2019, they saw a 200% growth in the number of people contributing to open source through Hacktoberfest, with 10% of their global engineering organisation taking part.

In this talk, Indeed’s Alison Yu talks through the techniques they employed to increase participations and what they learned as a result.

Transcript

Hi, everyone. Thanks so much for joining me here at DevRelCon Earth. Today, I’m going to go over how Indeed doubled our Hacktoberfest contributions from 2018 to 2019. So my name is Alison Yu, I’m the open source community manager at Indeed.

I’ve been in this role for roughly about two years, a little over. So if you have any questions about my presentation, feel free to either wait until the Q&A, which is directly after this session, or you can tweet me @alisonjudy. My Twitter handle is on the bottom left corner of the majority of the slides moving forward. So for those who don’t know about Indeed, we’re the world’s number one job site, with more than half of our users outside the U.S.

We’re in over 60 countries, and we operate in 28 different languages and have over 250 million unique visitors each month, and those visitors have access to about 25 million jobs at any given point in time. So if you just think about that, we’re actually a really large company, and a lot of our stack is built on open source, and that’s why we’re so invested in making sure that not only that we consume open source but we’re actively contributing back to the open source ecosystem.

Over the past few years, we’ve really leaned on Hacktoberfest to help connect Indeed employees to the external community and to land excitement and gamifying upstream contributions. So we’re going to start off with an overview of Indeed and Hacktoberfest. And then we’re going to talk about how our Hacktoberfest approach… how we approached Hacktoberfest in 2018.

We’re going to talk about how we engaged with our engineering organization at large. We’re going to go over the results and metrics that we collected. And then we’re going to talk about a few of our lessons learned. Directly after that, we’ll talk about what changed in 2019 and our engagement.

So, we’ll talk about how we learn from those lessons, go over strategy and a few of the tactical approaches that we implemented to receive our results. And then, of course, we’ll talk not only about our 2019 results but we’ll do a direct comparison between 2018 and 2019. So let’s just hop into our overview of Indeed and Hacktoberfest. So, Indeed’s Open Source Program Office was established in November of 2017, with the hiring of Duane O’Brien, who is our head of open source, and we’ve been really, really, really lucky to be at a company that values open source, and we’ve had very strong executive sponsorship throughout the years and, in fact, even before the Program Office was established.

So one of the pillars for Open Source Program Office, or OSPO, is around open source sustainability, and that means everything from event sponsorship dollars to supporting and sponsoring open source organizations, like the Apache Software Foundation, Open Source Initiative, Linux Foundation, etc., and, of course, increase contributions into the ecosystem.

In this talk, we’ll be focusing on increasing open source contributions by Indeed employees through the form of pull requests, but we wanted to make sure that everyone was reminded, and this is very important at Indeed, that not all contributions have to be code related in open source.

So if you think about it, a contribution could be anything from a logo design, to event support, to documentation, to opening an issue about a bug that you found and so on and so forth. And we really want to stress at Indeed that open source is really open to everybody.

We want people to think about open source projects as mini companies. So you really need all functions of a company, so whether that’s marketing, design, documentation, legal, etc., running at full speed in order to be a successful product or project. You can’t just have a great engineering project, people need to know about it, and that’s why we really want everyone to know that everybody can contribute back to the open source community.

So this is our Open Source Program Office and staffing as it stands today. So we have been and are open to staffing up our team across all Indeed locations and even remotely. It’s just a coincidence that we’re largely based off of the West Coast at the moment. So one of the virtual events that we take part in every year is Hacktoberfest, which is organized by companies like Digital Ocean, Dev, GitHub, Twilio, and more.

It’s a global virtual event that’s open to everyone and takes part in October every single year. So in 2019, to complete the challenge to get a t-shirt, people needed to make four quality pull requests on GitHub. So last year, over 20,000 issues were tagged, which gave participants a wide range of issues to choose from, so everything from good first-time issues to more complicated requests.

And this is pretty typical for Hacktoberfest, which is important because it means that no matter what your interests are or how experienced you are, you’re now able to find something that suits you so you have the ability to contribute to open source during the month of October. So in 2018, we had a pretty casual approach to rallying Indeed employees with our wider global community.

So let’s just jump into review. So in 2018 in October, by October 1st, we had four people in our Open Source Program Office. So that was Duane, our head of open source, a program manager, a community manager, and an engineering intern. Mind you, we were all based off of the West Coast, specifically in our San Francisco office at the time, and the team was mostly new.

So out of the four people of the team, three people had been at the company for six months or less. So we had a very casual approach to rallying Indeed employees to participate during Hacktoberfest in 2018. What you see here is our single email that we sent to encourage more open source contributions during Hacktoberfest.

Basically, we did a quick recap and promised some extra swag at the end of it. But as you can see, there are really quite a few areas for improvement beyond the fact that we only sent one email. One, we sent the email out after Hacktoberfest started. And two, the main hero image or really the only image was broken.

So let’s see how we did from just this one point of engagement. So I’m going to give a little bit of backstory on why we collect GitHub IDs and why this is important to us. So we started collecting GitHub IDs in September of 2018, so only one month prior.

And this allows us to collect contribution metrics via the GitHub API. So by collecting more GitHub IDs, we’re now more able to effectively measure our employees’ contribution to open source. So if an Indeed employee contributes to open source but we don’t have their GitHub ID, we’re not able to track their contributions and participation.

And this is really big for us as an ability to track our health as an OSPO and see how engaging we are with our community because if we’re not able to track that, we’re not able to see if our trends are trending upwards or downwards, and if we’re being a healthy partner to the open source ecosystem.

So as you can imagine, we do put quite a bit of weight on this metric, and it’s how we’re able to collect data to see how we’re doing. So, prior to Hacktoberfest, we only had 29 GitHub IDs. So, during the month of October in 2018, we’re able to more than double our numbers, which was great for us.

By identifying the 68 GitHub IDs overall from September and October, we were able to see that 35 people participated in open source with Hacktoberfest, and on top of that, 26 of those participants had contributed to open source on at least 2 days out of the month. And that was big for us because we honestly thought that we might have quite a few drive-by contributors who only did one or two contributions on one day, but we were seeing that people are actively coming back day after day to make contributions.

So that means that people are coming back and wanting to contribute to open source. So, that was exciting for us to see. Of those 35 participants, we had almost 700 contributions and over 150 pull requests. And our internal wiki, which we only had one blog post and a wrap-up blog post, no less, we got a little over 100 unique views.

However, seeing that we only sent out one slightly, very, very slightly imperfect email, we were ecstatic with these results. It showed that there was an untapped thirst for open source at Indeed, and we had only begun to scrape the surface.

But with 2018 behind us, we started looking towards what we could do in 2019. So let’s jump in and go over it. With our excitement around our results from 2018, we knew that if we leaned into Hacktoberfest, there was a good chance we’d see great engagement across the board at Indeed.

So let me walk you through some of the strategies that we pulled together to hit our goals. But first and foremost, just to set the stage, you’ll notice that our team grew, in fact, almost doubled year over year. So, by October 2019, we were a much more formidable, well-rounded team. We had obviously, Duane, who is our head of open source, a program manager, community manager, engineering manager, two software engineers, and an engineering intern.

And this meant that we had more resources than ever to dedicate to Hacktoberfest. But before we did anything on a programmatic front, we wanted to analyze our results from 2018 and set goals. So we went back to the metrics, and we decided from here that we would set what we thought was a stretch goal, which would be to double every single metric across the board.

But we knew we needed help to do that. Enter stage right, our ambassador beta program, or as I like to call it, the Open Source Ambassador Light Program. So we started formulating and recruiting for an open source ambassador program roughly in August of 2018, with recruitment wrapping up in September of 2018. As you can see, this helped us immensely with global reach, not only now covering more time zones in the U.S., but for the first time ever, we had people in Asia who were able to answer questions, open source questions, during the work day.

And this was super key for us. So we set up a series of meetings to align with our ambassadors and to communicate our one clear and concise goal, to double Hacktoberfest metrics across the board. But don’t worry, we didn’t just throw them into the deep end. So we created a wiki page, an internal wiki page, which was the single source of truth for our event and updating this page as needed throughout the event and throughout the month of October.

We also created a now open source project called Mariner, and that helped us pull a list of open tickets from Indeed’s open source dependencies. This is key because anytime someone would say, “I can’t contribute to open source because XYZ, my job,” now, we can actually show them issues that would help them make their lives easier during their jobs.

And this list allowed us to create a weekly list of issues that were then distributed to the wider internal team at Indeed. And we used that list during what we called our regional study halls. So study halls were a term we came up with to describe weekly events that took place across five different offices.

Luckily enough, October spans across five weeks, so if you didn’t have perfect attendance, there was still an opportunity for you to complete Hacktoberfest in full if you happen to attend and take part of our study halls. Each event was one hour long, and it was a space that was dedicated and set aside to contribute to open source and for people to get their Hackoctoberfest pull request in.

To help spread the word, the Open Source Program Office supported our local ambassadors by customizing these slides to each one of their offices. So these slides were displayed on all the communal monitors, of which there are a bountiful amount in each office, for the five business days leading up to each study hall. At the study halls, we also provided a checklist to the ambassadors on how to run each event.

And this meant there was a uniformity across all regions. That means regardless if you were taking part of a study hall in Tokyo, Seattle, or Austin, or even San Francisco, you get pretty much the same experience. We found this checklist to be really useful because we had a ton of people who varied on how new or comfortable they were with committing to open source. So as an example, in one study hall, we had an office manager who was completely new to coding and a maintainer on an open source project in the same room for a study hall at one of our locations.

So this checklist helped people pair off and lent more of a hand to mentorship. It also helped identify if someone was stuck on an issue and how others could rally around them and help them get unstuck and cross the finish line and submit their pull requests. And because I ran the ones in San Francisco, there was my clock. It was fun.

We also created a dedicated Slack channel for Hacktoberfest. And this was our real-time communications channel that allowed the OSPO ambassadors, participants, and more just to chat. So we used this channel to alert people when study halls were starting, posting the open issues from Mariner, celebrating people’s first-ever pull request, which there were plenty of, helping people find good first issues, celebrating people completing the challenge, and more.

So as you can see from a few of these screenshots, even if people were not very chatty or even if they were silent observers, seeing everyone else post their party parrot, which is something that Hacktoberfest awarded you with when you completed the challenge in 2019, it really spurred people to complete their very last pull requests.

And of course, at the end of October, there was a flurry of activity with people trying to go ahead and finish their challenge. So there was tons of people at the end trying to find open issues that suited them. And with everyone on this channel, it was really a great way for people to help source open pull requests for those individuals.

So now that everyone knows a few of our secrets, let’s hop to the results. So in our Hacktoberfest channel, we had more than 100 people join. We collected an additional 75 GitHub IDs and saw over 100 Indeed employees participate, with almost 70 of those people being recurring contributors, which means that they had participated and contributed to open source on two or more separate days in October of 2019.

That’s great. Once again, we’re trying to find recurring participants. From those 106 Indeed employees, we saw over 2000 contributions and a whopping 329 pull requests. And last but not least, we had more than 1000 unique views on our internal wiki pages.

So let’s see how that compared to 2018. As you can see, it was a success. We more than doubled participation across the board. So when comparing Slack channels, it’s a little unfair because we didn’t have a Hacktoberfest channel in 2018, but you can see that we almost doubled the number of people on our general LSS channel.

But from a more focused viewpoint, we had 115. So that’s a huge improvement over zero. We almost doubled the number of GitHub IDs collected from October 2018 to October 2019. We more than tripled the number of Indeed employees who participated in Hacktoberfest and doubled the number of recurring participants.

So I would say this was a smashing success. We were also very close to tripling the number of contributions year over year and more than doubled the number of pull requests. And last but not least, we increased unique views to our Hacktoberfest specific wiki page more than tenfold. So we’re really seeing that people are actively engaging with our community and really digging into Hacktoberfest.

So this was really exciting to see that some of the strategies and some of the things that we had done really helped Indeed employees actually contribute to open source. So you might be wondering what’s in store for 2020. Well, we are definitely looking forward to this year in October and Hacktoberfest, but we’re very much still in the ideation phase of our program.

We’re really excited to take part again this year, but with all the changes to the global landscape and with Indeed committing to being remote until July of 2021, we’re very much feeling out our “new environment.” But we’re aready on super-solid footing with an expanded ambassador program, and we’re reaching more offices than ever with more ambassadors and more representatives.

This means that access to people is just going to be so much easier across the board. So just to wrap up, one of the things that you really want to think about is globalizing and localizing. Find people in major geos and time zones to mentor others, especially if you want to make Hacktoberfest more accessible to those who are new to coding or open source.

You can really help by building local pockets of open source contributors. This will only help you down the line. We’re in the mindset at Indeed that it only helps to have allies when thinking about working in the open. If you can engage with people in a consistent manner, they might become open source advocates who then help their teams become more involved in open source and have them actually committing and contributing upstream down the line.

So this is a really key thing that we really want people to walk away with from Hacktoberfest. We really want to change people’s behaviors from working in a closed environment to really working in the open and defaulting to the open. Secondly, make it as simple as possible for people to contribute.

Ease the path to contribution as much as possible. It can vary from finding people issues that are related to their job to finding an open issue that matches people’s skillsets. Facilitate an easy way for people to find mentors or mentees.

This is a huge part of our engineering organization’s culture. So we really put a huge weight on mentorship and especially on mentoring people outside our immediate teams. We are committed at Indeed to helping people grow with the company and to grow their skillsets over time.

And, of course, help people find open issues. Really, that sometimes the biggest hurdle is finding an open issue that hasn’t been claimed yet that people are interested in. So for example, if someone’s new, it could be a fun open issue such as… I think, last year, there was one called emoji bop, where you described a song in six or less emojis.

So it’s just a fun way for people to get engaged and, like, really start thinking about the process in which to submit a pull request. Sometimes, that’s honestly the biggest hurdle. And remember, community is key. Help people build bridges to one another. It helps them find reasons to cross the finish line and to join the Hacktoberfest parrot party.