How to understand user feedback even when all a user says is 'not good'

Emily: Welcome everyone to understanding user feedback even when all user says is not good. My name is Emily and I'm a developer. Sorry, this always happens, developer relations specialist. And I've had various years of using curating, understanding user feedback as a customer success engineer in DevRel support engineering. And then I've done various types of documentation and content creation, both with video and writing. So I have dealt with a slew of user feedback over the years. So this presentation is going to go through some of the stuff that I've seen and kind of how it can help you and your own products. So for our agenda, we're going to be covering tips and tricks like understanding vague data, helping to better curate data from your users. So it is less vague. I'm going to go over some real world examples that me and my team have experienced.

And then we're going to talk about some prioritisation tactics you can utilise. And in the second half of this portion, we're going to actually do some group discussions. I've got some worksheets for you guys and we're going to go over some of the tactics that we've learned and how to apply them. And then lastly, we can round out the session with some general q and a. All right, so a little bit more about me. What I currently do, I work in a place called Mapbox. We do all sorts of maps technologies and actually if any of you guys have used Instacart, shout out to Instacart over there. We partner with them for some of their tech, but yeah, so we make all sorts of things. We have SDKs, APIs, editors, we have about over 30 products. We do various language supports and we get about 300 feedback tickets a week.

So we're dealing with a lot of different things on our day-to-day, which will kind of help curate this presentation. So if you only one product in your team are a hundred, these tips will help you out. Alright, so enough about me, let's learn about you guys. So by a show of hands, who here has done written documentation in their day-to-day? Okay, cool. Loving this. Any video content creators here? Cool. Okay. Does anyone travel a lot? Go meeting up customers in person conventions. Nice. Anyone do actual prs like submitting bugged fixes, feature changes into code directly? Cool. Okay, so we have a good array. Any shout outs for anyone I missed? Okay, we covered everybody. Let's go. Alright, so as I said in the presentation, the description, this will be focused around documentation, but I will let you know that a lot of these concepts are universal.

I do game dev for fun on the side and I actually use a lot of these concepts when play testing my own game. So we'll apply generally if you don't do a lot of documentation writing. Alright, so let's get into tips and tricks. So obviously we've seen the classic thumbs up, thumbs down and we want everyone to say everything is great and helpful, but you're more likely getting this kind of stuff, a vague keyboard smash. Oh, I would like to preface every single piece of feedback that is in this presentation is a real thing we have received at Mapbox. None of these are fake, so we're going to go over some strategies. We'll start with proactive ones, which will help you curate better data and then switch to reactive, which will help you deal with the slew of vague stuff that comes in. Alright, so for proactive tips, let's go to tip one.

Find a balance between informative and simple submissions. So again, we've all seen thumbs up, thumbs down, and this is great for getting clicks, but the problem is it's really vague, it's just quantitative. If you get 300 thumbs downs on an article, what does that mean? Why is it bad? And so your gut reaction might be to do something very like in depth input fields, a lot of multiple choice options, but you're going to overwhelm your users. You got to find that kind of middle spot that's really going to help you get good data without overwhelming your users. So something that me and my team use, we have a multiple choice field where you can kind of pick what's wrong with it and allows you to add additional information, but you don't have to. This is three clicks, which is a lot faster than a more in-depth field, but we're still getting more data than just thumbs up, thumbs down.

So again, try to find that balance. Alright, tip number two, group edits where you can to kind of describe what this means. I'm going to go with a real example. So we got this feedback on an article here and they were talking about how some of the information was out of date, one of the screenshots was wrong and so when you're having one issue in your article, there are bound to be other problems. If one screenshot's out of date, likely other screenshots are going to be out of date. So I went in and checked, we have a tool that we use called Sentry which tracks all of our user feedback data and I kind of checked like, okay, this screenshot's out of date, what else is going on with this article? And I found 71 reports. Now this is really important. Your first instinct might be to be, okay, I'm just going to fix that one screenshot, but what if there are other tickets that reported the same problem?

And what if there are other related wins that you can tackle? If you need to update screenshot for step four, you might have to redo steps one through three. So you might as well do that while it's in there. And by doing this method I was able to close out 60 tickets with a little additional work as opposed to just the one I was originally going to tackle. Again, this is very scope dependent. If you have a typo and it's going to be a 20 minute fix, it's not worth it. But I highly suggest when you're updating out of date information or bug fixes to double check your other tickets. Alright, so now that we've talked about Proactiv, let's move into reactive tips. So again, you can do your best to curate good data, but you're still going to get either default or keyboard smash info and it's going to be a little hard to figure out what's going on, but we can kind of tackle that with these steps here.

So tip number one, you are not just an expert, you are also a user. And to illustrate what I mean by this is I'm going to do it with another example. We had some feedback come in about this article that was hard to understand. There was some out of date information and so I was like, okay, I'm going to go through audit this guide, follow all the steps and see what's going on. As I was doing that, I think I had gone through the audit five or six times and I ran into these two code snippets here. So what you're supposed to do is paste a code snippet into the right one, but I accidentally posted it into the left and you can kind of see there's some similar things. Again, I was tired. It was like the fifth or sixth time I had gone through this guide and I got a bunch of errors and so I spent 20 minutes trying to figure out what on earth did I do wrong.

And so as soon as I finally figured out, I was like, okay, I goofed, it's fine. Everyone makes mistakes. Of course I'm the only person who's going to do that. And I took a step back and I was like, you know what? Let's just throw a note in the guide just for the heck of it maybe in case someone else does experience the same issue. And it's actually really funny. One of my teammates was reviewing this guide and he goes, oh my god, thank you for doing that because I ran into the same problem. Troubleshooting tips are extremely helpful. You're going to run into problems and make sure you document those as well. This guide we were able to get from a 66% positive feedback ratio to 88% and to illustrate what that means, it went from 66% of users reporting on it positively up to 88.

So again, with other edits too, but troubleshooting tips are really going to save you. So make sure if you run into a problem, document it because a user might as well. Now with all this in mind, it's good to empathise with your developers, but don't forget you are still an expert and this can influence your decisions and how you view content. So let's move into tip number two, turn off your brain during edits. To illustrate what this means, I'm going to kind of do an example first or a little audience participation. Who likes riddles here? Okay, thank you.

This will be fun then. So what computes faster than a quantum computer is deadly to consume and lives for all of time. Would anyone like to take a guess? Oh, say again. That's a good guess. I like that one. Not the answer I intended, but I like it. Anyone else? Okay. That's right. So kind of like a stumper, right? It's like, oh man, it's got to be some weird obscure thing like time or my brain, right? No, the answer is nothing. Nothing is faster than a quantum computer. If you eat nothing, you'll die. Nothing lives forever, right? Well except your opinion. So in hindsight like, oh my god, it's so obvious. I've totally seen those kind of riddles before. But here's the thing, we are all really smart people. You are all programmed to overanalyze everything you do. You've edited tutorials time and time again.

You're going to know every single thing that's in that tutorial. You've might've used a tool a hundred times every button, click every knob to turn. You could be a subject matter expert in your field. We are so used to looking for those tiny minute details. We're going to miss the bigger picture a lot of the times because we are influenced by our experience. And so it's really important when you're doing these types of edits to stop, take a step back and try to remove your expertise when doing these reviews. Now I'll preface, I'm also very guilty of this. I was literally writing this presentation and I missed a button click in one of my tutorials because I've clicked that button so many times it's muscle memory at this point I didn't even think about it and thankfully my coworker caught it, but our experience influences us.

And so you might be asking, okay, well what do I do? How do I exactly prevent this? There's a few methods I'll recommend. First, I love doing edits when my brain is tired. Let's say you spent three hours trying to debug a code sample snippet, can't figure out why it's broken. You finally fixed after hour four of banging your head against it. That's a great time because you're so tired and you've done all this thinking, you're going to follow that guide exactly. And when that button click isn't included, you're not going to click it and be like, why is this tutorial not working another good time? I always recommend stepping away from articles or edits for a little bit just to give your brain a reset. Sleep on it if you can, but at least give yourself an hour before going back after an edit.

Lastly, I love reading tutorials out loud to myself by activating another sense, you're able to find things you've missed. I'll do voiceover scripts for video tutorials and I'll be like, wait, is that the way it's written in the actual guide? Because what I just said to myself does not make any sense. And then if you don't like reading out loud speech tech tools like speechify are awesome to use. But yeah, so this will help you follow every step of a guide to the letter, which will help you catch those mistakes. Another way to illustrate this mindset, I'll actually pose with a question. Who here has kids or nieces, nephews. I'll even take a dog that you're trying to train, right? Okay, so there you go. Everyone's had some experience. When you tell a kid to do something, they will do it to the letter. May it be malicious compliance or because they have a lack of critical thinking, you tell them like, Hey, don't lie, lying is wrong.

Okay, well at Thanksgiving they're going to tell grandma that her meatloaf is the worst thing they've ever had. So think about your articles and edit them, how your kids would read those articles, follow those things to the letter like your kids would. Alright, so now let's move into tip number three. You won't be able to fix everything and that's okay. This is really important to remember because there are going to be things that are outside of your team's scope, things that are too low prioritised or also let's say you get a guide to a 93% positive feedback ratio. That's awesome. 93% of people liked your article, phenomenal. You're never going to get to a hundred percent though you're going to get overwhelmed. Imposter syndrome is huge in the text industry. Do not let it overwhelm you. And then also too, sometimes there's this I would like to reiterate, these are all real things we've received at Mapbox Orlando, of course. So there's only so much you can do with feedback sometimes. Would anyone like to guess what the difference? We get some metadata on our feedback. Would anyone like to guess what the difference between these two are spaces ago? Say it again. I heard another one over here.

A good guess, but no, sorry, as of honestly the top one was unhelpful and the bottom one was helpful guys, so obvious. No, no, but so obviously the internet is' insane. Same. There's only going to be so much you do try your best not to let it overwhelm you. Alright, let's move to tip number four. So understand your arsenal. What I mean by this is everyone has a slew of issues that their company is dealing with and everyone's problems are different. So you're going to need to figure out where are your team's bottlenecks and how can you optimise the usage of the tools that you have? To give an example, so some of the issues that me and my team face, we get vague feedback from users. Sometimes things are written in other languages and then some things are out of scope of our team either doesn't handle that product or it's just something we don't have time to do.

And so you might think, okay, well obviously I would just do a guide audit. For the ones that are vague, I would use translation tools. And then for the last one, well, I mean it's not my team, it's not my problem, but the thing is there are going to be lots of questions that are going to come up that you really should know the answer to. Otherwise there might be negative effects on you and your team. I'm going to skip prioritisation for now we're going to talk about later, but I'm going to cover the bottom two a little more in depth. So for translations, how do I save time on a repetitive task? I have tickets that come in that I use that sent tool to view and try to understand, but sometimes they're in other languages and at first I was literally copying and pasting it into Google Translate, which only takes 15 to 20 seconds, but if I have 50 tickets in a language I don't speak, that takes up time.

So it's like, okay, how do I speed this up? Fun thing if you use Chrome, if you highlight and copy or right click, you actually have an option to translate directly on the page and I'm assuming other browsers have this option as well as well as extensions, but less about hyping up this thing, but more find solutions for repetitive issues because it's going to cause you headache if you just keep brute forcing it and it's going to save you time overall the little things do add up. Alright, next we'll cover out of scope. So again, well that's not my team, not my problem, but what is its influence on your analytics? You're going to see a dip in analytics if say for instance, people are reporting negative on like, Hey, do we offer this product? And they're like, oh no we don't at this time.

Okay, well down vote, I really want the product, but you can't control that. You aren't the engineering team, you aren't building out these products, you're not product, right? So even though it's not your team, it is still your problem at the end of the day. Some solutions on how to handle things that are out of your control. I always recommend creating separate charts. You can have your original chart to showcase people, but take another chart and remove all of the product requests, all of the things that ui, things that you can't change and it'll help you better tackle what are the actual issues with your documentation that you do have control over. We actually see this a lot where we had one of our sections dipping down really low and when we removed all the product stuff, it bumped up. I forget it was like 15 or 20% or something because it really does a dip on your analytics.

Next, I always recommend tracking issues for product. If you go to them and you're like, Hey, people are really complaining about that bug, it's like, okay, what do you mean really complaining? Oh hey, 300 people reported an issue in the last week. That is a number that might catch your product team's attention. So try to track analytics where you can. And then lastly, offer other avenues for reporting. So feature request pages are super helpful because users can thumbs up, thumbs down them, and again, you can bring that data to product or if it's a bug, for example in the actual product and not your code, you can offer like, oh hey, redirect them to a bug report page that will go directly to engineering and then that will take out that influence on your analytics. Alright, now that we've talked about your arsenal, let's move to tip five.

Sometimes the problem is not the problem. To illustrate what I mean by this, I'm going to talk about a real world example. We have a quick, simple, easy integration for Mapbox if you want to just test out the product, but we were getting a 59% feedback ratio on that page, which again, it's quick, it's easy, it's simple. Why is it 59% right? This should be a copy and paste and we were seeing crazy things like it's not working a Firefox, it's not working in fiddle. When I remove the comments, it magically works. I'm like, okay, what on earth is going on? So I went in, I copied the code, pasted it into an HTML file and it worked and I was so confused because again, we were getting all of these negative reports and so finally I was like, okay, let me pretend I'm a brand new user.

I've never even heard of Mapbox. What would I do? All right, I'm going to pretend I'm going to make a webpage that adds a map. Okay, I'm going to Google. How do I add a map to my webpage? Oh, I see this simple map example, I'm going to click on that and then I'm going to go skip the code because no one likes to read and immediately copy and paste and put it in my ID and that's when it hit me. Our documentation automatically adds access tokens to your code snippets for ease of use, which is great if you're logged in, but if you're not, you get an error and the error is not super clear. So if I'm a user who's never used Mapbox, I'm going to go, well, it's broken. And so yeah, sure, you can see there's a couple of code lines there that say, oh hey, change this for me please.

But again, if I'm just copying and pasting it, I'm not going to read it. Yeah, sure. We put a little note at the bottom of the paragraph, just like little sentence that says to change it, but again, quick copy paste. I'm not going to think to read it. So I discussed it with my team and we were like, okay, we're going to put a bright yellow warning at the bottom of the snippet to be like, Hey, please copy this and adjust this section immediately it worked. We immediately saw a 10% increase in feedback after the next month or so, and all the new negative feedbacks that were coming in were nothing like the other ones were. We had smother other issues we had to address but immediately fix the issue with just a simple little note. And then after fixing these, we were able to get this up to an 83% feedback ratio.

So it's really important to understand when a user reports a problem, try to understand the real problem they're trying to solve. Alright, now that we've covered all these tips and tricks, let's talk a little bit about prioritisation and some of how ways you can deal with all the stuff that's coming into your avenues for feedback. So as I said, many types of avenues of feedback. You have thumbs up, thumbs down. You have your multiple choice, you've got internal resources, external resources, you've got Google Analytics. There's so much stuff, but that's extremely overwhelming. It's hard to figure out where to start and I can't tell you what the right method for you and your team is. It is something you're going to have to discover on your own, but I can lead by example and tell you some of the methods that we use and why we use those methods.

So what we typically do when handling user feedback, we'll do three methods. The first method we do is something called largest sweep. So every couple of months we audit the top 25 pages of our documentation, see what have the lowest feedback ratios, and then edit them accordingly. We'll then kind of check back in every couple of months to see how they're doing and then repeat the as necessary. This is great. We get a tonne of tickets closed and we're able to be like, Hey, look at us, great numbers, but the downside is this only includes our largest products because those are the pages that get the most views. So then we also supplement with two other methods. Method number two we have is called First Touch. So first touch is great. We go through and currently we are standardising all of our instal guides to follow the same regiment, make sure all of our codes s are up to date and we're creating how to videos to incorporate other types of learning into all those avenues.

This is great cross product work, but it takes a lot of time. We've only audited I think four of our guides. We have a fifth one on the way and we still have six to go. There's a tonne of different avenues we did to cover. The last method that we do is something called daily monitoring. So we do daily monitoring checks for internal and external feedback. Again through that app or that product I was telling you about called Sentry as well as Slack. And so this is great because it's cross product, we're fixing live issues as it comes in. If I've got a technical account manager being like, Hey, this team member or this customer was like, Hey, this thing is bad in the docs, looks great. I can talk about cross collaboration internally, but the downside is you're going to close out a ticket that three people reported you are going to miss issues.

You can't cover every issue that comes in. And so it's important to do all of these kinds of concept piecemeal because they'll cover the cons of the other ones. Method three doesn't cover a lot, but Method one does. Method one tackles a lot of tickets. Method one doesn't do cross product, but method three does. So they help kind of compliment each other, but regardless of what method you find works best for you, make sure you are being consistent over a timeframe, maybe like two months, six months, a year, you'll need to look back at your analytics, see what was working, what wasn't working, and adjust accordingly. Remember that you need to set attainable goals. My team does about 80 to 85%. That's a goal that we've typically can hit for most of our documentation, track trends and not numbers. When one of our sections doesn't hit 80%, that's okay because two months ago it was 50, now it's 72, you're seeing progress.

And then three, learn from your mistakes. You're going to try methods, they're not going to work. You're going to try auditing guides, realise you have to do them over again, but just make sure you're learning from your mistakes and growing. Alright, now that I've covered some tips that you can use in the field, I have a fun group activity that we can do. My two teammates, Andrew and Chris, thank you guys again are going to pass out some worksheets for you guys to use. And I've got pencils and erasers that you guys can have as well. But you're going to take what we've learned and apply it to these examples I'm going to give you and we'll break out, you can discuss with the people around you, but we will analyse the feedback, we'll create a solution to fix it and then you guys can present what you and your friends have discussed.

But one problem you're going to run into, for those who haven't seen the things yet, you're going to be like, Hey, well I'm a JavaScript developer, I'm an iOS developer, I don't even do code. What's a universal thing that we can all tackle, right? Food. We've all read a recipe that doesn't make sense, right? So I gave you guys very simple recipes that have some things that could be improved about them. And so you can go through, analyse them and we'll start with example one, how to make a peanut butter and jelly sandwich. I'm going to give you five minutes to discuss with your peers or do it. So if you'd prefer and then we'll discuss as a group. Thanks guys. Oh, sorry. Thank you for the clarification question. The feedback at the bottom is what your users are saying is wrong with it, so edit according and if you find other things too, I definitely did not do a good job at writing these, so find anything that's wrong with it, but thank you for asking. Alright. All right. If you guys want to start wrapping up on example one, we can bring it back to a group discussion. If you guys need a couple of seconds, feel free to Yeah, I'll give you guys another 30 seconds.

All right. Are we good to go back to group discussion? Yes. All right, sweet. Alright, so feel free to raise your hand if anyone had specific solutions, but so for the peanut butter and jelly sandwich, would anyone like to suggest a change you would like to make to this article and why? If you can raise your hand, I can call on you guys if you want. You would like to start. I like that. Thank you. Anyone else? Bad choice in tone is what I'm hearing. Phenomenal suggestion. I love it. Anyone else? I love that. Fantastic. Actually that was Andrew's suggestion for that feedback and I was dying laughing, but yes, no, there's no reference to a knife anywhere in this, so fantastic call out. Any other suggestions? I love that. Yeah, having a structure in steps is always great. I think it's just like two paragraphs blob together, right? Yeah. So it's like having steps would be fantastic. Anyone else?

Speaker 2: [inaudible]

Emily: So funny? I will never know. I like that suggestion. I love this mixed messaging. Okay, this is pretty good. I'm loving this. Oh sorry, in the back. Ooh, I didn't think of that one. Yeah, that should be specifically clear. That was a very intentional one on my part of course. Chris, I think you had your hand up. Kind of the same comment

Audience member: [inaudible]

Emily: Ooh, next steps. I like that. Okay. Yeah. Next A really good suggestion. Anyone else before we move to the next one? Feedback number two, what was feedback number two again? Yes? Did anyone catch why the peanut butter was bad? Good catch. That was my trick one. Alright, anyone else? Alright, let's move on to example number two, how to make a bowl of cereal. I'll give you guys five minutes or four and a half minutes and I'll let you know when there's 30 seconds closed out. I'm going to give you guys a 32nd warning or 30 seconds to another minute and then we'll wrap up. Alright, if I can have everyone's attention again, we'll discuss as a group. Is everyone good to wrap up? Alright, perfect. Alright. Would anyone by a show of hands like to suggest how they would change this tutorial

Audience member: [inaudible]

Emily: This is a fantastic point. Again, very intentional. Thank you. Anyone else in the back? Oh sorry.

Audience member: [inaudible[

[inaudible]

Emily: I like that. Okay, I love that you guys are suggesting things I didn't even think of. Okay, did you have one over here? Yes. Yes. That's a hundred percent what happened? It's really funny. I ran this example by my boyfriend and he also said the same thing I thought just a ladle. And he's like, yeah, but what if it's a normal spoon with a really tiny bowl? And I was like, oh my. I didn't even think about that. So that's a really good suggestion. Anyone else? Oh wait, back there. See these are things I'm not thinking about. I love that. Do you have over here? Oh interesting. So you're saying an unnecessary recommendation and people should be able to eat at their own pace. I like that. Alright, anyone else? Same with the note. I forget what is it? Oh, that's right. I was trying to see if I can make anyone the audience angry. I know some people do milk first and it's very taboo. I like that. That's a really good point. Your opinion is always correct and forever lasting. Just for the record and I rest my case. Anyone else again? Yeah, there's no size recommendation so you really don't know. I like this really good. Anyone else over here?

Okay, you don't read recipes as prerequisites? It's only ones I read. No, I'm kidding. Fantastic tone. Really important tone. Again, we always think of branding or up here in the front.

Audience member: [inaudible[

Emily: Great point. Got a link to other guides back there. Good catch. Hundred percent. You guys are really catching everything I put in there. That would've been such a good feedback to put in there. Thank you. That's a new feedback. It's in there now. Anyone else? Yeah,

Audience member: [inaudible]

Emily: Love that it's always good to, because yeah, you have a diagram where you're talking about, that'd be really helpful. Yeah, no, what's it called? My friend had joked because I ran this by some people and they were like, oh yeah, you mentioned about the captain crunch. Always cutting the roof of your mouth. So yeah, got to add a hardness ratio diagram. Alright, sweet. Anything else before we move to Oh yeah, you going over there?

Audience member: [inaudible]

Emily: I didn't even think of that. That next steps. There's no next steps. That'd be a good next step. Put the cereal away. These are phenomenal. Alright, we got time for the last one. If you guys want to move on to number three, which is how to make toast. And I will give you five minutes again. Good of you guys. A one minute warning. Alright, if you guys want to wrap up, we'll go back to group discussion. Lemme know if anyone else sees a couple seconds but think we're good. All right guys. Alright, show of hands. Any suggested solutions on how to enhance this tutorial back there? I totally didn't forget to update the template. Anyone else I

Audience member: [inaudible]

Emily: This very good suggestion. Good catch on the lever too because that was one of the ones. Some people have buttons and then toast your oven. I didn't even think that's a good one. So have tonnes of options. Anyone else? These are things I did not think about and I'm so happy you're putting again, you guys have been catching things I didn't even consider. Thank you. That is fantastic. Anyone else back?

Audience member: [inaudible]

Emily: Yep. Got to fix that crumbly butter. Am I right? I got a hand over here. I'm sorry you've never had uncosted toast before. No, that's really good catch. That's phenomenal. Yeah, thinking for, because I know for Mapbox we have certain words we don't use because they're not easily translatable. I guess toast should be put on that list. Great. Thank you for calling that out. I think I saw another hand somewhere. Oh yeah.

Audience member: [inaudible]

Emily: Yeah, because what if you're using a tub versus a, because I know the stick butters have it, but tubs don't. So fantastic point. Anyone else? Oh yeah, in the front.

Audience member: [inaudible]

Emily: Next step. What should enjoy with the glass of milk should always be the next step. But yes, next steps. Really good. Yep. Fantastic. What gave you that inclination? Yeah, the last one. Beautiful. I didn't know if people would catch that one or not. I was like, that's the class I pick of the problems. Not always the problem, it's like throw out the whole toaster, toasted it not how I wanted the front

Audience member: [inaudible[

[inaudible]

Emily: So some brilliant person, whoever wrote these I forgot, put rating next to those stars. I mean that was again an Ooh, that's a good one. That'd be a good debate of which is the hardest. Oh yeah. Over here

Audience member: [inaudible]

Emily: Fantastic catch. Be like, hey it's fine if you broke your entire computer, we'll just tell you to go buy a new one. Put your warnings in early. Good catch. Anyone else? Alright, sweet. These have all been fantastic solutions. Again, you guys caught a lot I wasn't even thinking of and I think you caught every single one I intentionally put in there. So thank you all so much. We have a couple minutes left for some q and a or if you guys have stories of your own you'd like to share. But seriously, thank you all for being an amazing audience. Thank you to the staff in the back and running this organisation like everyone and my team for helping pass stuff out. You've all been amazing. Thank you. Does anyone have any questions or stories they'd like to share of crazy feedback they've received?

Audience member: [inaudible]

Oh, interesting to reiterate the question to confirm my understanding. So you're saying you got so frustrated with a product and having to say it was broken so often you stopped wanting to use it worse. Okay. Oh, I'm following now. Yes. That's a really good point. So for us, we try to combat it by only putting it at the end of tutorials. So you'll only ever see it once you've completed one thing and usually excluding our getting started guides, most tutorials take several minutes. Also, we don't do any, we make sure everything is optional. There's never forced feedback requests. So if a user isn't interested in giving feedback, they can skip over it. And then actually Andrew created a cool news system of how we showcase some of our help tutorials and you don't even get the prompt until the very last slide. So I think if that answers your question, but that's how we deal with it, of course. Anyone else? Any other questions or stories? Oh yeah. Yes. Oh, I can do you. One worse. So I am a born and bred New Englander. Yes. There's no R at the end. We have something called a fluff and nutter, which is peanut butter and marshmallow fluff. So yeah, we love to put our sweets on things

[inaudible]

Emily: Ooh no. What's that?

Audience member: [inaudible]

[inaudible]

[inaudible]

[inaudible]

Emily: That's really cool. I hadn't heard of that, but that's a really helpful concept. Yeah, I know. Yeah, I would love to learn more. It looks like we just ran out of time. But thank you again everyone. Seriously, so much. It was so nice meeting all of you.