How user research can improve the developer experience


Amelia Abreu

Amelia Abreu

The developer experience is rarely “one-size fits all”. Though many Dev Tools are created through a “roll-your-own” process, end users will often differ wildly in their background, approach, and mental models than the core engineering team that conceived the product. How do we bridge the gap, and provide a better developer experience for all developers?

Learn from Amelia Abreu in this talk recorded at DevXcon 2017.

Transcript of Amelia’s talk

So today, I’m gonna talk about a cluster of ideas I’ve been working on over the past year, Developer Tools Learning and Care. So I’d like to start, I’ll tell you a little bit about what we’re gonna talk about today. I’ll tell you who I am and why I’m talking to you about this. I’ll talk more about this idea of care and challenges for advocacy and the developer experience. It’s so nice to be in this group where, you know, the notion of advocacy is very widespread. And then I’ll talk a little bit about the work that I’ve done with teams and some anti patterns or, you know, I’ve seen the past two presentations about some best practices, but I feel like I’ve witnessed some not so best practices. Lastly, I’ll share some design principles that we developed on a recent project.

So who am I and why am I talking to you about this? I’m a design researcher and over the past few years I’ve worked a lot with developer facing tools and worked with teams building developer tools. And I’ve been in and around the tech industry for about 10 years, but I started out as a human computer interaction researcher. And lastly, I’ve also been the primary caregiver to my daughter for the past five years and this experience has been so enlightening to me and also changed the way I’ve seen the way that we do work in so many ways, but it’s been very hard for me to reconcile this with the way that we tend to work in the every day.

So, I’m gonna make a confession, I hate the term “user.” It’s such…as Brenda Laurel calls it a demeaning little word. And yesterday I was chatting with the brilliant Sarah Sharp and we were talking about the term “contributor.” Like instead of we make this thing that somebody else is just gonna use, like how are the folks who are interacting with your product or interacting with your framework down the line, how are they contributing to what you’ve already built? But coming, you know, the field, it’s often, you know, people say, “Well, what do you actually do Amelia?” And the work that I do in terms of understanding how people use products, understanding common errors, spending time doing as new graphic interviewing, this all comes from human computer interaction which has a long background.

Ancient…we have evidence of ergonomic alterations from ancient Greece from the 5th century BCE. So folks have been trying to optimize tools in order to work better and to work more productively for millennia, but the roots of contemporary HCI are in the post-war era. And I like to say that I work from the assumption that we are all about Homer Simpson capable. So, a lot of the literate that we draw on, does come from this era of pilots and nuclear power operators, working on analog computers. And, you know, this is, you know, we are a very blind to material culture, especially when we’re working on code centric products, but these things are super useful to think about. So where is this relationship between experience and advocacy and care? I’m gonna do a brief summation of what I’ve been working on and where I see these things connecting.

I come from back… you know, I’ve drawn on heavily in my own work in the field of feminist ethics and the notion that all work, you know, all contributions are equally important. And we often work, you know, we often work from a standpoint like what we are doing is the most important. But, you know, what enables us to go to work to be productive, it’s often a whole chain of folks, who are making life possible for us. And secondly, you know, what do we owe our users? Like how can we care for the users of our products? This is something that I think that the tech industry has to reckon with.

Can we see a security breach as a lapse of care? Can we see bad documentation or lack of documentation as a lapse in care? Especially since what the word developer means now is way different than what it meant 10 years ago or 20 years ago and we have an increasing global diversity in our ranks. It’s not enough now to assume that just because this works for me, I built this for me, it will work for everyone else. And so, how do we provide that? Over the past year, I’ve been looking into how different frames of thought frame care and if you look up at just the term care in various academic journals, you’ll find analyses of care in nursing and bioethics and medicine, but you also find it in, you know, all sorts of applied disciplines.

And I really like this definition from sociologists that say caring takes on three forms: Instrumental, which is the like washing, feeding, getting you places, the sort of thing that you do for a small child or a very old person, or…but then there’s… but then there are two other dimensions, which I think are very prominent in developer advocacy work, emotional and informational. Emotional care includes listening, troubleshooting, walking folks through things. Informational care is applying best practices in a way that makes sense to the person who encounters them. So, developing a tutorial is a form of informational care. Developing documentation is a form of informational care. And I think it’s important to understand on a global economic level, just how important care is.

In the industry, we think of ourselves as economic drivers, but the truth is that 81% of U.S. jobs are in the service sector and this number is expected to go up. Professional caregiver is gonna be the largest occupation in the U.S. by the year 2020. And this accounts for 67% of U.S. economic output. Moreover, care has a global supply chain. We have a president, who’s a hotel man and who works in hotels, immigrants, who works taking care of children, often immigrants. And economists and sociologists have done in-depth work on migratory patterns that revolve around care. But in our current way of thinking, we often think of things in terms of the more we can do to obscure care, the better…the smoother it’ll seem for the end user. This is from Adaptive Path, which is a San Fransisco based Cisco based design agency and they say, “Oh let’s, you know, obscure the processes and systems to make it seem like magic,” which this is a little infantilizing. I don’t wanna, you know, I am a systems, I am systems person, I wanna see how things work, I don’t want to be told that it’s magic. And I also think that this is where things hit the fan when we are building backend products. Is that…we know this isn’t magic, we know that things are messy, we know that the development process is fraught with errors and bugs and misunderstandings and things that don’t work, and yet we still say, “Oh, we’re going to build things where the messiness is concealed.”

And we also like to a prioritize the work that we do over any other sort of work. You know, if you’ve ever worked on a team that did lock down or death marches, like you can’t even walk your dog if you’re working 24 hours a day. And this limits who we are, you know, who is creating technology and who we’re creating it for. There’s also the question and this has come up a bit, I’m not trying to slag on Facebook, I promise, but, you know, this has come up a lot in the recent weeks and months about content moderation. There’s over hundred thousand folks in the Philippines working in this industry of content moderation. This is something we’ve offshored, we’ve obscured. We don’t really talk about when we have…when we talk about building community and moderating content. We often frame it in terms of AI and in terms of machine learning, but it is human effort and often very hard human effort on the other end of this.

And then there’s a question of who is doing the care work and, you know, if we’re gonna talk about the problem of diversity in the tech industry, maybe we should be talking about the problem of care, because it’s black and brown women doing the care work in the tech industry, even though we’re often very underrepresented. And this to me on a personal level is a quandary. I grew up in a Spanish speaking household and if I’m…if I hear someone speaking Spanish in a client office or on a tech campus, it’s usually the cleaning staff or the kitchen staff. And what does it mean when you walk into a professional situation and you’re statistically a lot more like the support stuff. And is there a way that we can engage with care that helps us think through this better.

And so, over the past year, I’ve been doing a lot of soul searching and a lot of thinking about, you know, how is it, how can we effect this change? And I reached the conclusion that by working…my continuing to do my work not, you know, quitting to run a recreational marijuana farm which is, you know, legal in Oregon. But, you know, by working it to improve developer experiences and to bring care to developer experiences, this is a way to affect cultural. And what happens…what kinds of cultural shifts have to happen in order for us to design for care, you know, what happens, you know, how do we…how do we change our mindset in order to, you know, go from as John Schiffer said so hilariously, “I’ve never written a bug,” to understanding vulnerability, to understanding making mistakes, to understanding that what we build is for ourselves and for people like us often requires a different set of accommodations for other folks. And so I’ll share some anti patterns that I’ve seen in my own work.

Over the past few years, I’ve worked with teams at Nike and Intel, Mozilla as well as smaller startups and open source projects on building tools that helped on board newer and different developers. And we did this work by doing ethnographic interviews. I’ve probably interviewed about 50 developers over the past two years in terms of usability as well as worked with teams of many dozen more. And my goal in this is always to understand workflows, to understand mental models and to understand what’s missing that could improve the developer experience across organizations and across teams. And I’ve noticed a few things that keep coming up in different organizations and in different ways. And I’ll talk about these a little bit more in detail. The first is the idea of rolling your own solution and then assuming that everyone else will grok them. The second is a culture that does not dedicate resources for documentation and I have a feeling that a lot of the folks in this room do not come from cultures like that. But it means that, you know, when there is no…When there are fewer resources for documentation, it means that documentation is everybody’s job and no one’s job. And thus, it doesn’t happen.

Third is this culture that discourages stupid questions. As a design researcher, I loved stupid questions. “Oh, what is an STK?” And not to…not because I’m ignorant, but because I want to hear how you define it. I wanted to hear how folks, how these parts fit together in other people’s minds, because there’s a lot that we can learn from asking stupid questions. And the last factor that I’ve noticed is a very uneven answer to the question who is this for. A lack of strategy and planning in terms of who we think is going to use this product and in what way.

So first, the roll your own. This hits a lot of bumps right away, like, what about users who are different from you and your team. And, you know, one of the clearest examples of this is keyboard shortcuts. If you are an emacs user, you have a different set of keywords short cuts than a Vim user. And this is something that comes up not only in developer tools, but also in accessibility. And the tool that I put together…the tool that I’ve made to counteract this is a coloring sheet and I made some copies that you all can come and get from me. And to what, you know, I guarantee you’ll see things differently if you sit down and with, you know, some colored pencils and color and the keyboard shortcuts that you’re assuming folks are gonna take. And this is something that, you know, when I interview developers they say, you know, “I just want to see like some best practices and a story. I want to see the story of why I would use this.”

The second aspect and this is something, I’m probably preaching to the choir here but, you know, this lack of resources and urgency around support, community, and documentation. Working with a hardware company recently, I was interviewing developers about how they use the internal APIs. And they would say, you know, I heard this over and over again, “Well, you know, I know the documentation is not great, so I’m just gonna go find the guy who made it.” And it’s like, this is not…this does not scale at all. How do we, you know, if we rely on relationships, this will not scale outside of your building. And there’s also just a lack of scaffolding around documentation, so if somebody does write documentation for a tool that they’ve built, oftentimes, it’s not organized or organized in a standard way and even simple navigation elements, you know, plain old HTML are often very useful in this regard.

And next on this…this little blurb is from the blog coding horror, it’s this question of, if I ask for help or I ask for clarification or even if I ask for more documentation, I had met vulnerability and I risk, you know, looking, you know, looking foolish. And this is something that I don’t have the answer to, and I don’t think any of us individually have the answer to but how do we build a culture that allows folks to point out the holes that they see, allows us to point out weakness in terms of I don’t understand that, that doesn’t make sense to me, rather than, “Oh that looks good, yeah I can totally do that.”

And the last, I’ve noticed is this question of who is this for? And over and over, I will see, you know, I’ll…I’ll work with a team, they have a product that they’re very excited to roll out, you know, they think it’s great, they’ve been using it internally, but when a new set of eyes…when we bring in a new set of eyes, they say, “WelI, don’t really know what this is for or how I would use it in my job like,” or if it’s a, you know, if it’s an API written by C developers, who then when you put it in front of Javascript developers, they say, “I don’t… these calls don’t make sense to me, I don’t understand what these functions are.” You know, how can we both, you know, deliver abstraction and automation, but also deliver, you know, a cohesive set of stories and documentation around it?

So what I’d like to do is to urge us all and this is something that for this crowd is probably not news, but to work rather from this vague notion of empathy that’s become very common recently, because empathy without action is pretty meaningless. How can we work not only as advocates but as allies, as accomplices? I love the scene that came out of Indigenous Action, accomplices not allies. But the idea that we forge connections not to act for other folks, but to act with them. And what can we do with brand new developers in other parts of the world? What can we do with folks who are completely different? How can we partner with them in order to build things that we would never build to begin with? I think there’s a lot of potential here. And there’s certainly a lot of opportunity to build new viewpoints based on this.

So lastly, how much time do I have? Okay cool. You know, one exercise that’s proved very useful in working with teams is developing design principles. And this is from a recent project, but we built our design principles in three areas. Tell me why I should use this? Tell me what it does? Tell me how it works? You know, put all of these out in the open, because we can’t work off of assumptions. Then next, show me how I can use it? Give me visuals, give me clear terms, give me a generalist introduction that does not…that is not insulting. And this can happen in terms of contacts, it can happen in terms of industry specific things. And one developer I interviewed he said, “You know, context is more important than content,” as he scrolled straight to the sample code, because he wanted to see it how it worked in the wild.

And then next is this notion of building community over time. It’s not enough to, you know, I often come in and people say, “Well, we know we need to put in new documentation.” But investment in this over time, so that folks can become acquainted with it at the beginning, they can see how to use it and to bring in that, you know, nurture those relationships with…those relationships with experts, so that we can have examples of how to use it, so that users can decide that the tool is for them. So what do we do next? I’ve given you an earful and how do you apply this? I…you know, I would advocate for you to, you know, work on developing user research and design research in the developer tools process, generate user flows and understand requirements, understand contacts through spending time with the folks who will use your product, and in the environments that they’re going to use your product.

I’m a big fan of rapid iterative testing and one of my favorite things to do is to work with teams to get them ramped up to test features as they come down the pipe instead of putting off usability testing till the very last minute or I will do that on the next release that you never will. But if you integrate it into your process, it’s a lot easier. I’m also a big fan of the participatory approach. So, having folks come to you, having them, you know, iterate on the features they want to see or the approach that they want to see and use that for your docs. And I also, you know, advocate again for organizational efforts to value and understand care work, whether it’s in documentation or if it’s in driving the shuttle bus or cleaning the office, you know, how can we see this work as, you know, as crucial as it needs to be?

And lastly, let’s work together. I started a learning program last year called the UX Night School and I also love to work with teams individually. So come find me later, let’s chat more about this and thank you very much.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.