Dogfooding developer products: gathering insights from internal hackathons

Josh Brown shares Spotify’s experience of using internal hackathons to learn about the APIs they’ve developed, in this talk from DevRelCon London 2019.

Transcript

Josh: My name’s Josh and recently I started a new job in Stockholm at Spotify, where my role is to help support Spotify’s community of third-party developers and to help grow our developer products.

Many of you may not be familiar with Spotify’s platform, so here’s a really quick overview. We have three popular developer products. Our Web API, which helps developers access metadata from the Spotify catalog, manage playlists, and follow users and artists. Our Web Playback SDK, which allows developers to stream music in the browser. And Spotify’s Embedded SDK, a binary that developers can include in hardware and other devices to help power audio playback.

Many of you may have worked on internal hackathons before, hack events within a company where employees are working on hacks over a period of time. There was one such event during my second week on the job at Spotify called Spotify Hack Week. Hack Week 2019 included more than 100 teams across Spotify’s offices worldwide and each team took one week to build a product or feature of their choosing and then, at the end of the Hack Week, present it to their peers.

Many well-known Spotify features originated in Hack Week. For example, Discover Weekly. The event is a big deal and developers look forward to it. I had two goals during Hack Week, this being the second week of my new job. One was to stay out of the way and the second goal was to learn as much as possible from the developer experience of hack teams that were building on our API and SDK products during Hack Week, of which there were many.

Studying DX from employed developers can be tough. They are very different from external developers who use your SDK or API and they often bring background knowledge about a product that an external developer would not have access to. Based on what happened at this and previous internal hackathons that I’ve worked on, I put together three tips to make it easier to get useful DX insights from an internal hack event.

The first tip is to avoid collecting feedback on your entire platform. At a Hackathon, I’m often tempted to collect and document every piece of feedback that I hear. Instead, during Spotify Hackweek, I decided to pick two really specific topics that I wanted to study and to do a deep dive on those. This could be a new tutorial that you created for external developers, a new sample app, or maybe part of your developer sign up flow. Focusing on a small area really helped me put the feedback into context and make it actionable for the product team.

Another tip that I think is really important is to be super aware of which resources your internal developers use when they build their hacks. Employees at a company tend to have extra access to resources like log files, source code, and other developers who may have worked on the developer tools that they’re going to use. When hack teams’ internal hack teams stray away from using your public documentation, that decision can be really insightful and could indicate a gap or problem with what’s publicly available on your website for developers.

And my final and probably most important tip is to take some time at the end of your internal hackathon to interview developers one-on-one about their experience with your chosen process or product area. It helps to bring pre-planned questions and to be really specific in these interviews. For example, instead of asking, “What do you think about our developer sign up flow?”, you might instead ask, “What was the biggest pain point involved with obtaining an API token?” As part of the interview, it’s also really important to take time to understand their day job, the team that they worked with on their hack, and also the project goals of the hack because this context really helps place the feedback, makes it actionable, and gives you context that can help turn that into something you could bring back to your product team.

So those were my tips. Thank you for coming to my talk and if you’re interested in learning more about this, stop by the Spotify booth. We’re also hiring so feel free to stop by and hear more about our openings too.