Better Bug Reports -> Faster Fixes

Eric: Okay. Thank you so much. Yes, such a pleasure to be here. Thanks for everyone for taking the time to listen to my talk. As mentioned, I am the lead at DevRel, at Grammarly, but I'm also the only person doing DevRel at Grammarly, but I took the title anyway. You don't say no to that. So today my talk is about bugs, right? And let's try something. Raise your hand if you love bugs. Like almost nobody. That's the right answer, right? We don't get into this industry to deal with bugs. They're not part of our OKRs, they're not on the funnels. It's not what we get excited about. But I'm going to make the case that bugs are important and that if you write a better bug report, they'll get fixed faster, which is going to be better for your community. And that's my name of my talk.

So why do bugs matter? First of all, I think they can be a major source of frustration for the developers we serve. You do all this work to get their attention, to activate them, to train them, and the one bad bug come and sweep out their legs at the wrong moment, kind of undoing all that other work you did. So having those bugs not be a risk into onboarding and other sorts of adoption flows is really good for everything else you do. And I think even if someone's in your ecosystem, long-term bugs that remain unfixed for a while, they kind of erode trust that the company really cares about its developers, right? You're there to constantly prove that developers value that this ecosystem matters. And if a bug lingers for a long time, it undoes that too. And if you make a better bug report, what it really is doing is it's greasing the skids.

It's allowing the engineering team, the product team to fix that bug faster. And they only have so much time and attention. I'm constantly dealing with teams that are focused way more on the future of what's coming and less on the past of what currently is. And so the more I can kind of make it easier for them to fix a bug, the more likely they're going to actually do it. So the first step of filing a good bug report is to not file it at all, but to just search. So many times I've spent a long time crafting the perfect bug report to have it closed as duplicate within about five minutes, which really hurts the soul. So I really think you should take a lot of time to search first different keywords, different combinations. If there's a bug out there, you want to pile onto it, you want to make that one bug hold all the information and all of the impact, all of the value so that it is its strongest version of itself.

And so you may have learned something new. Maybe a developer came to you and said, Hey, I ran into this issue. There could be an existing bug that didn't recognise this certain case or didn't have this piece of information. Take the time to make that existing bug report better if you find one. But if there is no bug report, the next thing to do is really just confirm it's a bug trust. But verify your developers emulate their environment. See, you can get the same error. Unfortunately, not all bugs are reproducible, right? Some only trigger in Japan. On Tuesday, you may not be able to get every bug repro, but maybe for those cases you just kind of trust a little bit extra. Another big thing here is check that what you're seeing is not the expected behaviour. Obviously, a lot of times working as intended can be something you're slapped with on a bug report.

So do the time, look at the docs, talk to the engineers, figure out is this the actual behaviour we're expecting for this case or is it really a bug? But even if they say it's the expected behaviour, that doesn't mean it's good. That could be still crappy behaviour. So this could be a chance to pivot with what you're doing into filing a feature request to actually make the system better. Alright, but now you've confirmed it is a bug, you got the same error code. The next thing I really like to do is to narrow down the scope. Kind of like some people say that carving a statue is removing all of the rock that is not the statue. I think sometimes I get an issue from a developer and it's full of all this extraneous detail, but I don't know what's extraneous. And not until, all right, let's remove this factor.

Let's take out that factor until eventually I find the smallest thing that causes the bug to actually repo repro. And if I remove anything else, then everything works fine. So I think doing that really hones in on where is the issue. But if you go too narrow, sometimes you could end up missing what can be a broader impact? Yeah, maybe it fails in this one exact case, but then test the other kind of sideways cases around that because maybe the kind of blast radius of this bug is much bigger and therefore it's much more important that it gets fixed. So narrowing and then a little bit of broadening.

So now you're going to actually file your bug. So the first thing I recommend is a really good title. I think you want to include things like the error message keywords that someone would be searching for the name of the method or endpoint. You want to have good SEO on this bug so that not only you can find it later, but people can quickly realise what this bug is when they're looking for it. Next up is the repro instructions. So I think you've done all this work to reproduce it. You want to give a crystal clear, super explicit set of steps that say if you do A, B and C, you get the error message. I think the more ambiguous you're like, well, in this kind of method, sometimes they're getting this error. No, you want to create a recipe for the engineer who's going to fix it to trigger that bug as quickly as possible, being super explicit.

And I really love this pattern of repro expected actual, I use it in all my bug reports. The repro is the set of steps. Expected is what you thought should happen and actual is what did happen. And it's another great case to call out maybe a difference in understanding because you might still be corrected by an engineer. It's like, well, what you expected here isn't quite right. Or at the very least it helps clarify what you were hoping to test. So the engineer can figure out, oh, they were going to this area of the code.

The next thing I like to do is include a little notes section where I kind of riff on some factors. You did all this work to kind of carve out all the things that weren't related. You have that in your head. But it's useful, I think, to pass that along to the engineers that are going to fix it. Because what might be something simple for me like, oh, even when I move the query parameter, it still worked well. It could be that query parameter is triggering an entirely different flow of the code and you listing a non-factor rules out whole systems that they don't have to debug. So including some of those things that you realised don't matter can actually be useful in helping you, the engineer narrow down on what does matter more quickly.

And then this one's a little dangerous, but if you are familiar with the code base, I think it can be useful to pose a theory from time to time. Now, obviously an engineer who's deep in this code might out school you every day of the week on exactly what's going wrong. But I think you have a couple advantages. And one of those is you have a broad vision of your product. An engineer in a certain area may know it really deeply, but doesn't know the entire ecosystem of the API. You might be aware that last week there was a change in the auth system that seems related to what's going on here. And those little clues that you can pull from your larger vantage point can also help when it comes to getting it fixed more quickly. But be careful. You could obviously lead people down a rabbit hole if your theory ends up being wrong.

So I think do this judiciously, and this is one that I often forget to do, but I think is so important, which is to talk about the impact. We are on the front lines. We see what happens, what the effect of this bug is. Is it ruining some launch? Is it causing some failure? That's very important to our product, that impact, but I think it's useful to communicate it out to the other stakeholders. So talk about who's affected. Is it everyone? It's a huge percentage of developers, or maybe it's actually your most important developer, your most important customer. Does it break an entire application or is it just like a little hiccup? Are there workarounds available? But are those workarounds super painful? And then maybe you're trying to tell a story or trying to avoid a certain story about your product, and this could play into that.

So maybe the optics of it are bad. I think all of this information can kind of help raise that bug's priority, what it eventually gets looked at. But just filing the bug isn't quite enough. I think the last step is to really socialise it. So I often just dump it in the system and expect that it'll find its way to the right people. That's not always the case. So two tricks that I like to use. One is sometimes I'll know what engineer works on that part of the code and I'll just slack them the bug directly because personal pride is a pretty strong thing. And an engineer who realises that their code doesn't work, they might actually put down their other work to kind of quickly jump in and fix their mistakes. So sometimes getting in front of the right eyes can get it fixed even sooner that if it flows through some prioritisation system.

And the last thing is just if you're in a venue, like a team meeting or something else with your engineers or your product team, share some of those personal stories. I mean, everyone that you're talking to, they are people. And when they hear about there's this developer who's doing this amazing thing, but then it totally fell apart on the day of their launch. Those stories, we know them and we feel them. And that's why we're doing all this work. And being able to pass those stories along to others can also help get more attention onto that bug. All right, so just to recap here, spend a lot of time searching. Confirm it really is a bug. Narrow it down, but not too much. And then spend the time to file it with a great title, repro some non-factor, your theories if you're confident and what the impact is. And then lastly, just get in front of those right eyes and then tell the stories about it. That's it. Thank you so much.

Audience member: I have a question for Eric, so I really appreciate your talk by the way. I thought it was really, really helpful and you condensed information very beautifully. My question for you is, I appreciated the caveat you put the asterisk on posing potential series because engineers typically know the code base better, but sometimes especially at large companies or the multiple teams, they might tunnel vision on specific features. So you have that context. I'm asking my question is, how do you balance on that knife edge of trying to provide context without also coming off as arrogant or I know the code better than you Because I think engineers, they take pride in their work. And so sometimes when you're showing bugs, I've encountered in my experience, sometimes I've made it harder on myself because the way I presented it, it came off I was attacking them or something like that. So do you have any advice on basically how to do that cordially and delicately?

Eric: Yeah, that's a great question. I definitely have worked with some engineers that can be a little prickly and you don't want to come off the wrong way. Otherwise they could not exactly be the most receptive. In my case, I think I try to be very neutral with it. That's why I really like the actual and expected framework. I'm not talking about there's no emotion behind that. This is what I think it should do, this is what actually happened. And I think with theories as well, I always tend to caveat it. Even when I do code reviews, I'm always like optional colon, like personal opinion colon. I really like to get across that. There are times when I feel strongly and times when I don't. And when I think you write that kind of notes in your bug about what you think the cause could be, do some caveating. I'm not really an expert here. And that kind of just sets the right tone that no one's going to think that you're coming in and telling them what to do.