Made by
Made by
Tao Dong
Tao Dong, user experience researcher at Google, looks at the user experience principles and methods that can be adapted to improve API usability. He shares five lessons from his work on Google’s Flutter API in this talk from DevXCon San Francisco 2018.
Thanks for the very nice introduction by Tamao and a little bit more about my background. I’m currently a UX researcher at Google and my focus is on developer experience and API usability. So, before that, I was in academia doing Human Computer Interaction research. I did not do my PhD on developer experience, but I found a lot of ideas and methods I learned from doing general HCI research is applicable to DevX. So, hopefully, some of the ideas I will introduce today will also be generally applicable to your products and developer experiences as well.
As Tamao mentioned, I am currently working on this product called Flutter which is still fairly new. It is a UI toolkit for crafting beautiful native experiences for both iOS and Android. And it is an open source project hosted on GitHub. You can check out its source code and look at what kind of interesting interactions who have been with our contributors.
So, Flutter has three important product pillars. And the very first one is high-velocity development. So, this is where my work mostly comes in because we really wanted our developers to spend their time delivering experiences for their users instead of learning tools or learning APIs and finding errors.
So, what really attracted me to this conference was this question in the CFP, “What can DX learn from UX?” And this is a question I have been thinking about a lot since working on Flutter, and I hope I can provide my take to that question with this talk and, specifically, I hope you can get at least two things out of this talk. The first thing is that you will feel inspired by some of the ideas from UX and apply some of the research methods to better understand the experience of using your API.
The second goal I had in mind is to inform you about what a UX research can do for a developer product so, you will feel more comfortable talking with a UX person and perhaps ask for help. I know in many organizations, you probably have a UX function for your consumer side or business, but not necessarily have UX help for the developer side of business. What I want to demonstrate today is that a lot of methods and ideas are actually similar between the consumer side of things and as a developer side of things.
So, I will cover these three topics in the rest of this talk. I will start with this very basic question. What is API usability? So, let’s start by look at API in the larger framework. So, this API column is a very commonly used API for laying out your user interface. It’s going to put all the child widgets into a column vertically. So, you have the user class name, arguments for the constructor, and types for each of the arguments. But what I want to argue is that the experience of this API is much larger than its shape and its functionality. What really matters is the total experience of using that API in a realistic context.
So, with that argument, I wanted to further point out two factors we must consider when we assess the usability of API. So, in addition to API design, which is the signature and the types and argument names I just showed, we also need to consider the tooling and documentation for the API. Why is that? Because nobody is really expected to write code on a whiteboard except in a coding interview. So, your coding experience is always supported by a lot of features in modern IDE and as well as a large body of examples, thanks to sites like Stack Overflow. So, we should not assess the usability of API just in itself.
And why is API usability important? I hope I don’t need to convince this room of people since you are already in this space. But in general, we don’t want the developers to write buggy code because they cannot understand your API correctly. We also don’t want them to feel frustrated and confused when they use your API to accomplish their tasks and solve their business problems. So, what can we do to make APIs more usable? And here I would like to share some lessons I learned through conducting API usability research on Flutter and hopefully some of the lessons will be useful for your product as well.
So, the first lesson is to reduce context switching. And I have a specific lesson here which is about avoiding switching to other applications to look for information while the programmer is coding. So, we observed this behaviour very often in our studies and just anecdotally, that programmers use the “go to” decoration and “go to” source code feature in IDE very often to look for information about API in the source code of that class. Sometimes they prefer to look for documentation this way over our online, nicely formatted, web documentation. Why is that? Because it’s usually easier to get to this type of documentation in the code within IDE without getting distracted by other things. And the lesson we take is that having a lot of documentation is great, but we must consider how and where the developer is going to access and consume different kinds of documentations. I personally, am a strong believer in high-quality API documentation in the source code. And this can allow the developer to get to this type of information without going out of their current context of solving problem and without having to run the risk of getting distracted by a million of other tabs you happen to have in your web browser.
And then the second lesson is to help developers build mental models. This is especially important for complex APIs. And before I get into a specific example from Flutter, let me briefly explain this term “mental model” which you might have heard about from different places. So, a mental model is a user’s understanding of how a system works; which can be wrong sometimes. In contrast, you might also have heard about this term “conceptual model” which is the model of how the system actually works from the designer’s point of view. So, when these two models do not match, the user can experience breakdown when they carry out tasks. So, a classic example of such mismatch between these two models is actually the thermostat we use almost every day. Here’s a simple question. Will setting the thermostat to 90 degrees warm up the room sooner than setting it to 80? So, if your answer is yes, then we need to talk after this presentation.
Okay. So, back to the context of API. So, here we have learned that some developers had difficulty understanding how different building blocks in custom animation in Flutter should be wired up together. Flutter has a very, very flexible and powerful animation library, but it does introduce quite a few concepts upfront to a beginner user of this library. A lot of things are not presented in the most clear way about how they should fit together. So, one way we have thought about addressing this problem is to visualize this conceptual model as envisioned by the library designer, and then map different parts of this diagram to relate to the methods and classes. So the developers have a frame of reference when they need to reason about the behavior of the animation code they write.
And the third lesson I wanna mention here is to speak your users’ language when you design your API. This is probably a pretty obvious point but it has a huge amount of impact on the way the user can discover and use your API. Again, one thing we learned from our own mistake, we used to have this API called Block. Anyone can guess what this do in the context of the UI programming? Any clue about what this API called Block might do? Yeah. Actually, it does a little bit more than that. So, it’s okay if you don’t know what exactly it does because not a lot of people know. Okay. But how about this name? ListView. Does it sound a little bit more concrete or familiar? Yeah.
So, it’s a list view which holds a number of items in a vertical manner and it also scrolls the view to reveal more items if it’s more than what it can display. So, what you probably didn’t know is that Block was the old name of ListView until early 2017. So, we repetitively, from that our study participants, could not find a container to hold items and make the container scroll because when they were looking for this functionality, Block was never the name that came across their mind. So they could not guess that name and put the name, potential keywords in the search box or pay attention to anything that may match the expectation when they browse through our docs.
And then next lesson is enable programming by examples. A number of developers have told me that examples have become the de facto way of learning programming and using APIs. I’m not sure if you agree, but to some extent, I think there is some truth to this, thanks to the success of online Q&A sites like Stack Overflow and also open source code in GitHub. But the lesson we took is one step beyond that. We found that merely having a ton of examples available is actually not enough. So, we have this collaborative feature early on our site which allows a developer to search through our code repository to find all examples of a class.
So, this screenshot shows a developer looking at the search results for an API called Card which draws a card on the screen, but the developer cannot decide which one is actually useful. So, he said that, “This is nice linking to actual code, but it’s very difficult to choose which one you’d like to use unless you see the output.” So, we switched to a different approach. We saw quantity is not really helpful in this case. So, we started to provide really curated code samples in our API doc, and in the same time we also provide screenshots and diagrams to illustrate the output of that snippet, so a developer can quickly understand whether this example is actually relevant or useful to the task they have in hand.
And the last lesson I would like to share is to promote recognition rather than recall when you think about your API and tooling design because the human brain, recognizing the right option out of several different choices, is relatively cheaper than coming up with a correct answer from scratch. An example of a recognition-based UI versus recall-based UI is the difference between retrieving a password from your memory versus picking the most likely option from a manual.
So, back to the API usability work we did on Flutter. We used to have an API called Colors. We actually still have this Colors API. But the way Colors API used to work is to store all the shades of color in a map in order to access a specific material design color palette. So, the problem with that is the developer has to remember the key to access a specific color and it does not show up when they try to use autocomplete in IDE. It’s a kind of a recall-based UI and nobody knows how to proceed unless they have already used this API before. So, we redesigned the API to surface all the shades of color as properties. So in this way, this API works very nicely with autocomplete. When you need to look up the shades that’s available for a particular color, you can bring up this dialogue and see all the options and we even show a preview of this color on the margin of this IDE, if you notice that blue square on the margin. So, this is an example where redesigning the API can enable tooling and it can really take advantage of what your IDE can do for the developer.
So, a quick recap. These are the five lessons I have learned myself from doing research on Flutter’s API. Hopefully, some of those will also be useful for your APIs and the developer experience. You might ask, so how do we identify these issues in the first place? I will quickly introduce one of the methods I often use to understand issues and frictions in using our APIs. This method is called API usability testing. There’s nothing too complicated about this method. In a nutshell, it’s really about observing a developer solving realistic programming tasks using your API. And you can follow four steps to plan and conduct API usability study. And I will briefly walk through these steps.
And the first step is to figure out what API is in the scope of this study. Create a list of APIs you wanna test and this will help you specify the task you wanted to use in this study. And this will be a task that you ask the developer to try to solve with the API you want to test. And then you will need to moderate that session to observe how developers actually solve this task. You typically need two rooms. One study room and the other room will see the live stream and you can invite your team members to observe the study. And during the session, you need to do two very important things. One is to remind the participant, it’s a test of the product, the API. It’s not a test of how well they code. It’s very important to make sure they would act normally as much as possible. And then you need to remind the participant that they need to think aloud when they program so you can understand what’s going on in their head. And here is a quick demo of how it works. So, I will do the voiceover. So, I will do something first. Okay.
The last step is, of course, evaluate the data to look at the results across different sessions and decide if any change to the API tooling and documentation is needed. So, it’s a time to let the data speak. But before you do that, you also need to be careful about what data you want to take into consideration and how much weight you give them. You might collect a bunch of information from these sessions. My advice is that you should pay your most attention to behavioural data which is about what the developer actually did in solving this programming task. You can also use the verbal reports from Think Aloud protocol, but you should take a grain of salt into statements about their future behaviour like, “I will do something in the next month.” Humans are not very good at predicting their future behaviours, and you also need to be careful about interpreting their retrospective accounts. Sometimes people try to rationalize their actions by inferring what they were thinking instead of giving you accurate account of what happened in reality.
So, I’m running out of time here. So, in summary, I introduced some lessons from my own API usability research work on Flutter and, hopefully, some of the lessons will be useful for you as well. If you have any feedback or questions feel free to talk to me and ask Neli about Flutter after this session. Thank you very much. And I can be reached via email or Twitter.