Made by
Made by
October 13, 2017
Usability testing in progress
Jenny Wanger
Building lasting relationships with developers doesn’t end at the sign-up. Events and content get a lot of the attention, but they’re a waste of time if a product’s developer experience is lacking. Jenny Wanger, who heads the DevEx Rockstars at Arity, now looks to apply her user experience skills to developer experience. This will be the subject of her upcoming DevRelCon London talk, ‘The UX of DX: User Testing in the Invisible World of APIs and SDKs’.
APIs and SDKs can languish at the bottom of the usability heap; if they make it to the pile at all. Often, rapid development is prioritised instead. As developer tools, not many people think of them in terms of usability. However, for Jenny, who has a strong background in design, this is shortsighted.
Whilst an API might be all code, with nothing shiny for the marketing team to wave around, the design of an API is just as important as an intuitive user flow through a web application. Jenny’s team have found that the exact same usability studies traditionally applied to end-user interfaces are just as valuable in evaluating APIs and SDKs.
Fundamentally, Jenny notes, “if your endpoints are hard to use, if your documentation is hard to use, developers are going to just go find your competitor. They don’t want to have to work to try and decipher what you meant. They just want to use your stuff, and get on with their lives. So, if good design was a blocker to them moving as quickly as possible, then they’re going to get upset with you.”
This is especially true for people working in strongly competitive API spaces, where it’s a given that developers will be shopping around to look at all the offerings available. And, “if your competitors are easier to use, you’ve lost already”.
She also makes a strong agile argument for DevEx testing on APIs. “If you can mock up your APIs with a prototype before you start building it, then you can test if it’s usable. You’re not wasting time later on changing things once you’ve figured out that people are getting stuck or frustrated.”
There’s also the argument that, by actively baking considered API design into the build process, and giving developers a smoother experience down the line, ultimately end users will have a better time too. Developers will be able to work more efficiently, and spend their time building things that work better for users, rather than needlessly troubleshooting.
The DevEx Rockstars team do the bulk of their testing with a pool made up exclusively of developers, all of whom have been recruited over the course of the past year. Over that time, the company has partnered with other companies to continue to expand and diversify their user group.
Just as with a UI study, a sample of developers will be given one thing to test drive, be that a wireframe, or a hypothetical API. They’ll be asked questions like; “Imagine you’re trying to call this API. What does it do? How do you do it?”, with extra coaching given as required. Although the developers won’t be asked to write code in the exercise, they will be encouraged to interact with documentation as if they were.
As with any other aspect of product development, the specifics of testing will differ depending on what level of fidelity the researchers are looking for. For the early stages of a product, for example, Jenny will recommend simple mockups. In the past, she’s made API wireframes by simply printing out diagrams on a sheet of paper, and asking developers to imagine making a call to specific endpoints. As a project matures, the level of fidelity will rise in tandem, right up until there’s a fully-fledged back-end built out and ready for in-depth testing.
There’s one final golden rule that the DevEx Rockstars have for their testing; no two developers are the same. Even in a big enterprise tech pool, Jenny comments, you’ll never find two people “who have the same experience,” or who do everything exactly the same.
Much like the field of developer relations as a whole, developer experience is an emerging concept. It’s through interaction with the wider community of experts that industry best practices are emerging and coalescing. Jenny is hopeful that in time, as more companies develop API programs and API strategies, it too will come to seen to be seen as an “absolutely critical” component to the design process.
Come hear Jenny speak at DevRelCon London on December 6th at The Barbican.
Photo by Aaron Fulkerson. CC-BY-SA.