AA166 - The Perils of "Just Build It": Why User Validation Matters

welcome to the arguing the Agile podcast where enterprise business agility coach Om Patel and me product manager Brian Orlando debate the real world challenges Agile professionals will face. We are not here to sell you anything. We are here to argue about Agile so that you don't have to. Om, someonee said something to me the other day and uh, oh boy, it really upset me it was something along the lines of Hey, Brian, if it's, if it's quick, why don't we just implement it? We don't need to test it with users or do prototyping or test it in any way, shape or form. We can just implement it quick because it's quick because it's great. That's the second thing that somebody could tell me that would upset me so much. First would be, Oh, you owe me money. Um, listen, we've seen this. The first would be, I don't need coaching. That would be. Yeah, we'll start this array at zero. I'm the expert. I don't know. That's right. I'm the expert. Um, this is not that uncommon, honestly, because people think about this and say, Why do we go to the trouble of doing all of the due diligence when we can just do it right? And it is one of those things that on the if you take it at face value, and if you take it up to leadership, they're going to say, Yeah, we're going to How long is it going to take? If you want to prove something, how long is it going to take? Oh, it's not long. Okay. Well, just do it then. Right. You know, the other thing I've heard is the time it takes us to talk about what we want to do. I should have already developed it. Yeah. That looks, that means you haven't thought about it. So, yeah, yeah. So you're right. So you're right. So let's, let's get into this, right? If you're just going to do it, you don't even know what it is that you're going to do. You have an idea. And you have basically just a, a very, very nuanced view of what you're going to do. Which is to say, you really don't know beyond your idea. You haven't thought about it laterally. You haven't thought about it, you know, widely. What about the repercussions? Does it align with your current product? Let, let's, let's go up in the podcast. Let, let me start in the, I don't know why I'm on this side. I was upset by this when it came up and I felt like I, I, I felt that I didn't know why I was reacting negatively. When I was reacting negatively to this, I couldn't, like I had no, I only had a feeling of unease and I was like, I don't want, this is not good, but in the moment, I couldn't tell why. Okay. But the, the, the, there are two things that I clearly can understand as the benefit as someone seeing a benefit. Okay. If someone's going to see a benefit for anything that we're about to talk about, I can clearly see two things. Number one, I can build it. Right away. We don't need to spend a bunch of time talking about it. We don't need to prototype, but we don't need to spend a bunch of effort. So two things. Number one is I can, I can build it into the application faster than we could prototype and go back and forth with users on it. So speed of delivery. Okay. I can deliver faster than our feedback cycle works, which first of all, let's, let's, let's revisit that here in a second. Okay. Let, I know, I know you want to dig into that because I, even when I say it, I'm, it sounds ridiculous. The other side of this is. If I don't have to engage a designer, and I don't have to engage a product manager, and I don't have to get customers on the phone, and I don't have to delay building the feature, or whatever. There, there is a perception of a reduced upfront cost. I don't have to pay all those costs to do all that stuff. I'm just going to throw it in the application, and throw it out there. And also, I feel there is a legitimate claim in between these two things. I can just do it faster, and I don't need to prototype it. And to vet that prototype out there, there is the potential of a fake door feature. I can just throw a button in the app, throw it out to the, to the, to the users for a select period of time, and then toggle it off, right? Assuming we have good development practices and then toggle it off after whatever, 24 hours, 48 hours, however long you We're doing the test for, we're, we're running the test and I can turn it off and now I've, I've basically introduced that feature into code direct, you know, instead of quote, testing it via prototyping and user groups and whatever else. Those are the two things that I feel will be used as, uh, to fly the flag of the banner of, we should do it this way. We should just test ideas by just implementing the ideas. Meaning either implementing the, the full blown idea or the, the, the, the very MVP of the idea and just throwing it in the application or implementing a fake door version of the idea and just throwing it in the application. And then we don't need, you know, clickable wire frames. We don't need to hire a UI UX designer. We don't need to go through, you know, having user testing groups. We don't need this, these feedback cycles and all this kind of stuff. We don't need any of that. We'll test it all live in production. Yeah. I think there are several ways to kind of reduce your risk there. Right. So in that, uh, in that scenario, we said fake door, some people can use it. Maybe, uh, you can maybe just, just feature toggle the thing away and say, okay, only a select group of users. Can have the, uh, you know, the key to unlock the toggle, so to speak, right? You can do all of those kinds of things. The thing I would like to caution here is this, how small of a feature is it? Right? Uh, so if you have a UX component to it and, and, and, and you're going to say, just build it, are you using the same UX standards that you've used so far? Are you going to use something quick and dirty that is It visibly looks different where it could be jarring as an experience if you're not careful. But if it's that small that people aren't going to see it and it falls into the same UX themes that you've used before, you could try it. So how I would like to proceed with something like this given the choices, not just do it and put it into production, just do it in production. Do it quickly. You don't have to bother with all of those things that you talked about, you know, that cost money and time. But don't print production yet. Put it in a elevated environment. Let the customer see it. And if they like it, you can promote it up to production. So that's kind of again, reducing your risk a little bit. if that sliver is not so thin, right, then, and this is very kind of, I guess, contextual. you got to think about that and say, are we introducing risk? Can we break something right? That's already working. First of all. Secondly, is this something that aligns with the rest of the product and where we're trying to go with it. Are we trying to solve the problem for one customer or is it? Part and parcel of the product itself, right? So it's going to serve every customer or is it select personas, etc. You got to figure that out, right? I think there's still some time that needs to be spent considering all of these things before you say, yes, it's just go build it because it's just going to be two hours worth of work, let's say., so those are things that I, that I immediately think about, , when given this scenario, right? And then, and then the other thing I think about is. Okay. What if something goes wrong here? Right? How do we retract that out? Now, if you're feature toggling, that's easy but if you're just promoting something into production, you've now got to retract that and say, we're going to take that back down and go back to version n minus one. All of that is already costing you money You would have been better spent doing the due diligence in the first place, right? So there's there's those things to consider as well. And lastly, if it's taking more than just a couple of hours, more than a day, what's the opportunity cost? What should you have been doing instead? So those are considerations that I think about right away. Uh, the opportunity cost is a great one. Like what, what else? Well, because we're doing this, what else are we not doing right? You know? Yeah. Yeah. We can do fake door features. Yeah. We can do whatever. Yeah. And that gets a, like, if you're, especially if you're B to C and you really don't have direct access, like you, maybe you have so many customers that you, that direct access to any one particular persona or group of personas is not Feasible. Right. Even when I say it, it's not even really fair because you, you still probably going to have a select group of beta users or power users or, you know, early adopters. Yeah. Early adopters. Yeah. Example users. You're going to know your early users who are, who are really, you know, You know, they're on top of it. They know when new features come out, they're playing with them all. You know, they, they're giving you feedback, that kind of stuff. What I expected to become an issue here is the lack of user validation. Of saying hey, why don't we just create this new feature and throw it out there? it's gonna take less time to To develop it and throw it out there than it is for us to talk about and agree on what we want to do With different screen designs and internal voting and whatever like I'll just create something throw it out there Yeah, but you haven't like then there's no there's no consideration of your your Yeah, this is why I say put it into a stage or a UAT environment, let them, you know, get their hands on it, get some feedback. You may decide to say, okay, this is fine. We're going to go right away into production. That's fine. That's the ideal case for you, right? It's quick. And that's the straight line path. But you may, Just hear something from the users that are beating it up in the UAT environment that says, Oh, that's not quite right. So at least you haven't risked it playing production yet. Well, also it could be the wrong problem that you solved. Exactly. Right. Like you, how are you going to, without directly talking to users, like sitting on a call, when users are walking through it, like how else are you going to get feedback like that? Basically you've solved the wrong problem, or you solved the problem at the wrong stage, or the wrong step, or whatever. Like I think about mobile apps specifically, Like, you solved, like, yeah, maybe you've reduced the pain, but what you did was reducing pain in the wrong spot. Maybe the very painful spot is still in a different spot, and you, like, because you threw something out there. Yeah, I mean, okay, maybe you did resolve some pain, but the more valuable fix. Was in a different area. Yeah. You may have solved it for the wrong persona, even right. It can happen. That's right. So depending on the product. So, yeah, I think it's easy to do a snap decision and go, this is, it's faster to do it than just talk about it. Right. But oftentimes there's bigger repercussions. Are you building up technical debt as a result of this decision? I almost said hasty decision. Cause I want to say that it is a hasty decision. At the end of the day, you're trying to do something quickly, right? And so go back all the way to the beginning of that. Why are you trying to do something so quickly? Are you trying to leapfrog the competition on this new thing that you think is going to be so valuable for your customers? Right. Why, why not go through all of the due diligence? I mean, I'm glad you brought that up because I've been in a lot of companies. Where the development teams are not in the mode of talking to customers, where they get excited about a solution, or the sales mainly leads the development efforts, like prioritization and whatnot, and they get excited about something, and that's why they jump on a particular solution. Yeah, oftentimes I reduce that down to the developer saying, Hey, I got a solution. Do you have a problem for that? Right. So, yeah, I mean, that can, that can spell a lot of trouble for you from a product lens that you're going to have a product that is not only, , not fit for purpose in the end, like a hundred percent, let's say, or 95 percent if you're happy with that. It's gonna go deviate, you know, it's gonna deviate away from that because developers are already solutioning things that they think are needed versus validating the need first. Another thing to bring up, , is the different types of companies. Like, I wanted to do a, I wanted to do a whole different podcast on the different types of companies that are out there. They, they, there are visionary led companies like the, the, everyone always thinks of, , Apple under Steve Jobs. Like it's, everyone's basically following his lead to what to build next, the visionary led company. You have one particular founder who's, who's a, who's a real visionary who has a idea of the future. And everyone's kind of, basically that's the source where the ideas are coming from. from versus a company where the ideas are coming from sales. Like what, what they're saying customers need to sign deals and whatnot. So you're following sales, you're a sales led company, you know, versus a technology led company where the technology wing of the company is saying, Hey, we discovered this new way of doing whatever we're going to implement it. Isn't it cool. Now they're trying to engage sales, marketing, whoever to say. Go drum up some customers for this cool new thing Yeah. That we have available. The idea of the technology led. Like, we're gonna, we're gonna build something cool and hope that we can find some customers for it. That has its own set of troubles. Yeah. It's, I think we're gonna delve deeper into it in that podcast, I find. I think that's gonna be good, because I like that topic. Yeah. Yeah. It's, it's, it's also an interesting one because, uh, the more I dig into the Steve Jobs visionary myth,, the more, , I realize that, , it's not true at all. Yeah. C Jobs had to be convinced to build the iPhone. He, like, he was, he was resistant to the idea of building the iPhone in the first place. So was I found., really, really, really interesting blog about that a while back and I wanted to have a podcast on it. We never have yet. Yeah, there's visionary like companies that people think about like Apple on the jobs, but also now these days, , Elon Musk as well, right, as a visionary., and, and, and to a lesser degree, perhaps Richard Branson is when he came out with, you know, originally, I guess it was the record company, right? Uh, I'm sure he didn't think about, uh, owning an airline at that point, right? But anyway, so yeah, that, that we're going to save that for later podcast. Yeah, yeah. But the, the idea is. Like the, the kind of through line that we were talking about is, , that it would save you costs to do this. But the reality is it ends up increasing development costs to go up, uh, by building everything that you're trying to prototype, building everything in the final application basically, and, and, and sending it out because you end up, you end up getting, feedback, which leads to rework. Again, assuming you don't have any architectural challenges or technical debt or anything, assuming none of that, , you end up every time that you get feedback, you need to change something. You end up having to go back into your application and change things that like the development team thinks is minor or, you know, maybe not minor or whatever, maybe because the way it's implemented, it's major to change, you know, small, tiny things. But now that you've implemented it one way, You have to come back and refactor every time you want to change it for customer feedback. Yeah, I agree. And all of that feeds into that other topic, like of, you know, opportunity costs. You now incurred all this cost because you wanted to cut corners in the first place, right? Um, I think there's probably somewhere, almost a middle ground, but maybe not even middle ground. Um, engage your customer first before, You put this thing in anywhere, right? And just say, well think about doing this. Mm-Hmm., does that have value for you? Well, you, you've just kicked us all the way back to the user validation step. Like if you're the old business analysis, well, at least, at least the, the need validation upfront. The need validation, and then on the functional validation later, that's, that's the old business analysis, like the, the, the dirty tricks of the ba, like the hey. If we did X, would it solve your issue? Would it, would it solve the problem that you're, that you're seeking to solve? And then if we did Y on top of that, would it, would it further make you happy with the solution? If then, then if we did Zed, Hey, if we did X and Y and Zed all together in the application, would that be something that you would be willing to pay for? And then like they, they led the customer along all that stuff happens. When zero, uh, software has been written at all, but if you're saying, well, I mean, I can just code something and throw it out to the customer, you know, um, I mean, I guess if you have a whole development staff, just sitting around, not doing anything that you have the money to just throw out active prototypes, right? I mean, I guess that's great. I just don't know a lot of companies that have that access. Yeah. Um, if you, if you have that luxury to do user validation in live code. User validation in live code. It's risky at best, right? I mean, so if you had a lot of developers sitting around doing nothing, why not have them create three different scenarios, right? And, and put all of that in production and have the customer test on all of them. You're basically doing customer validation or user validation in production. Which is a risky proposition. Right, assume you're pre product market fit. You're not sure that you know what these customers problems really are. So you're taking shots in the dark. What would resolve their problems? So rather than mock it up, you're just going to build real life software to, to, to deal with their problems. Like, I mean, will, will users actually know that real life software is different than a mock up with fake data? Will they really know that? I like, I don't know, but what I'm saying is you're validating ideas. The, the, the four Marty Kagan,, categories., is it valuable? Meaning what do our customers find value? Will they use it? Is it usable? Uh, can our, can our customers figure out how to use it? Is it feasible? Can, can we actually develop it with our technology and developers that we have in house? And, and is it viable? Does it actually work for our business model and how we're going to sell it and that kind of stuff? If you can zip through those four things. And validate every single one with your customers. Great. But like, how are you going to just build something that validates all four of those things right away? I mean, you're taking on a lot of responsibility through the development team. I mean, those four things are like barely related. That's right. You said responsibility. I was going to use a different R word. You're taking a lot of risk at that point. You're basically saying, I'm going to put all this, my, my stack of chips on double zero right here on the roulette table. And, uh, yeah, maybe, maybe once in a while, you know, Maybe that, that little ball falls in the double zero, but hey, that is a small chance. Right. Even in the Marty Kagan categories, he splits those up. I mean, the, the, the, the usability goes to the UI UX designer. The, the value goes to the product person. The feasibility goes to the tech leader, the devs. He splits those up to the whole team. So you're basically saying, I don't need to split those up. I'm gonna just code them straight out in the application. Uh, like, I guess it's, if our company was like, uh, the Brian and Ohm Software Development Company 3, because remember we sold our first and second companies. Yeah, I sold the yachts afterwards. Yeah, the yachts also got sold because apparently we have problems, financial problems. Anyway, I don't know what there may or may not have been an edit here, a hard edit here. Uh, we were discussing, uh, country music, uh, circa five years ago. But the, the lack of user validation, that's what we're on. Lack of user validation. Like, I mean, I guess, I guess if you're going back to our podcast that we did with Alex talking about, um, your intuition versus evidence. Like when you're building something brand new. You need intuition to guide you. Sure. But then As you build things, you need evidence. You need to be evidence based. You need validation. Need validation. Yes. Yeah. So like skip this step, I guess what I'm saying is skip this step at your own risk. Definitely, definitely skip it at your peril because you don't know how it's going to land, especially if your product is such where you have. People that use it in different ways. It's not like just the one thing, right? If you're going to circumvent the validation piece, you're essentially just gambling. Yeah, so so skip those steps at your peril because nothing much good can happen out of that. What about the scenario where,, maybe you work with your largest customer or representation thereof., bring them in and,, bring them into your organization and co develop something with them. Pretty quickly, and still I'm falling short of this idea of putting into production, even if it's for them, like let them kick the tires in a sandbox, maybe like a U A T type of environment. When you hear good signals, you can then promote that up to, you know, up up to production. Otherwise you're taking a huge risk. I feel I want to pivot , missing opportunities for course correction. Because I feel that when you're, when you're saying, well, I don't need to do a prototyping or I don't need to do discovery. Why do you discovery work? Like I'll just build it in live in production. Okay. Maybe, but also like what happens when you're very close to the mark, but not quite at the mark, right? If you're doing wireframes, you're doing clickable prototypes, for example, and you're sharing those, a customer tells you something like you can, like, if you have a UI UX designer, that's really on point on right there on the call, they can make some changes, roll it out while you're kind of discussing the back and forth. And fire away, and they can say, yes, that's exactly what I'm thinking about. You know, they can tell you right there on the spot. Yeah, and that's the best way and the most efficient way of reducing the delta, right? Yeah. So, I agree with that. I think if you're not really doing that due diligence, you're missing that opportunity. To, um, you know, like it says, course correct, basically. Um, you're assuming, A direction is the right direction, right? And you're then not only putting in production, you're almost inflicting on the user, you know, that that specific direction, right? And when you hear back, inevitably, I find that that's not quite right. You may be close, but it's not quite right. Now you got to. Come back and do it, but it's already in production now. Assuming you can immediately come back to it, many, many, many different programs I've worked on. We support many different types of personas and say, Hey, we're trying to do something in this sprint for this particular user. And we did something and that we demo it to them and say, Hey, And the user says, well, this is close to what we need, but if you really did this one other thing, it would really make us happy. But we have commitments in the next couple of weeks to say, well, we, we won't be able to get back to this feature for, you know, a sprint or two or, or likely like three, right? Like six weeks out. Right., because we have so much of this stuff stacked up for other users that we've committed to or promised to, or maybe we're doing, you know, maybe we were doing a juggling act cause it's sales renewal season or something like that. Yes, we built something and we've, and we got it out to them quickly, but the risk is if we don't exactly hit the mark and they want small tweaks, like the, the, the, the, the, the lighter feature in the Kano model of like this very small tweak, we could have given them very quickly. We can't do that now. Cause we got all this other stuff stacked up for all these other personas. That's every development team I've ever been on. We know we can't get to the, to all the work. So we just have to prioritize what we're trying to get to. And that's like where the product management function makes all its money because there's so much work we want to do. I can't even imagine the, an environment where you can, you can get to everything where you don't need to do this. It's rare. I I don't, I've seen none of that in my past. Right. So to your point, so that delayed delighter is no delighter because now No way the customer experience is already out there. Yeah. And it's not ideal. Right. Right. You didn't nail it. Right. You might be in the same zip code, but you didn't nail it. And consequently, you basically all of this comes down to kind of like the reputation, right? You're not really making me happy by putting this in product. By the way, you didn't get any feedback from me to begin with. I could have told you. It's customer's voice, right? So yeah, you're right. It's several days, weeks later that you can actually make the correction. Think about that, right? Because a lot of this comes back to you after the fact. It's in production now and you're running. It's in production, but you've you now introduce an additional layer of user frustration or extra steps that user has to go through. The other way this manifests that I have seen as a product manager, it, it manifests in low adoption of your new features. So you'll say, Oh, we, we, we did this thing that everyone said they wanted. But no one's using it. You should have known before you launched it, pre coded it, that no one was going to use it. You should have gone down that road. You should have. I guess what I'm saying is your leadership is going to perceive it as. Why did you launch a feature that no one is using? How did you not know that before you launched it? And that question, it sounds very challenging, but if we peel it back to, to, to trying to be a hopeful question it is the right question to ask. Yeah, it usually comes back that way. You're right. So if you rush this thing into production, you likely did not put in telemetry in place to see how the user is using it. Are they using it? Are they struggling? Are they happy? Are they content? Are they delighted? You don't know any of that because you didn't put it in. You just rushed it, right? So yeah, nothing good can come out of that. I don't think unless again, you happen to be so lucky that you know that little white ball landed in the double zeros, Right, right, right. It's very rare. I mean, do you want to take the chance? That's what it boils down to. I mean, broken clock is right. Twice a day. Like that's this kind of scenario. Like I I've been on with a lot of people that said we need this feature and they were right. But also I've been with a lot more people that said we need this feature and we built a feature and they were completely wrong. And once we built the feature and it was completely the wrong feature because we took, we took it on face value that that stakeholder, usually it's executive sales, something like that, that they did their research and they were correct on the right thing to build. I would say it's starkly in the camp of the, they did not do their research and they are wrong. And the adoption. Fails to meet expectations, and now that comes back on the product manager and development team. It does not come back on the stakeholders who suggested or heavily implied or demanded or whatever the feature. Those stakeholders will turn right around and say you didn't do it right. That's in the, that's actually, that's in the Marty Cagan transform book actually he even goes out of his way to say, It doesn't come back on any of those people that demand the feature. That's right. It comes back on the product manager. People that, yeah, people that develop it and implement it. Exactly. I think this is kind of broadly in the category of, build it and they'll come. It's like, you don't know what they need, right? So maybe you happen to build the right mousetrap for the right person, you know, at the right time, for the right size mouse. Uh, you gotta get really lucky for that., don't take your chances. Don't don't take your chances. Oh, well, I mean, if you're willing to take chances, take a chance on this podcast. Like, like, subscribe. Definitely. Take a chance on this podcast. We didn't talk about,, stakeholder alignment. Yes. We didn't talk about that one. Aligning always says so like a, if you're doing prototyping as opposed to just throwing stuff into production, , you have the opportunity to align your stakeholders and, and align and, and actually, and not just the stakeholders, it's also the features you're building. You have an opportunity to. To really think about, Hey, a lot of stakeholders think this would be a great feature, but does it really line up to the, to, to, to my product vision? Like my meaning, I I'm the product manager now, does it, does it really line up to the product of vision that I'm trying to promote, and if you're just kind of haphazardly building things, throwing them out there, are you purposely moving forward? Is what I'm asking. You're likely building a Franken monster, right? If you keep doing it over and over again. So that's what I meant earlier. I think I mentioned that a little bit. It doesn't align with your overall product strategy and direction, right. But in, you know, in regard to the stakeholders, so you put something out there in production, do you know if that can be supported by your support? People have, they've been apprised of what's going on here, right? Or is, are they going to be surprised? I mean, you have to go through that, right? Enable your ops people, right? Have you done the right change management piece that's needed and the right comms, or are people just going to suddenly discover this thing? You can at least publicize the prototyping phase. Say, Hey, we, we've got these prototypes, these screen designs, this, this, this clickable wireframe, it's publicly accessible in my company. So everybody can see the path that we're considering. No, no, regardless of what stakeholders are involved in evolving the actual clickable wireframes or prototypes or whatever, though those prototypes you should, should sit in some sort of shared environment where everyone can see them. Again, because we go back to transparency is a good thing, right? Right. To say, this is where we're going next, there should be some kind of, I mean, at least quarterly update to say, you should be constantly forecasting the findings of your, of your wire frames and your discovery activities. With your internal stakeholders. I mean, honestly, you should be doing it for everyone, but your internal stakeholders at a minimum, so they can start lining up to say, Oh, well, there's a new feature that, that, that now you can click on it and you can see, you know, X, Y, Z, all from, and, you know, from, from all the way from the beginning to the end, the application. Now that's really cool. Like, and there'll be a period of time where we can ask questions and talk about it or whatever. How do we sell it? How do we support it? How do we maintain it? How do we, you know, make it go away when it's time for it to go away or whatever. There's, there's a, there's a top to bottom kind of a life cycle that needs to be talked about internally so that everyone can line up and not, I put that on the product manager. That's that's that that is squarely the product manager's job, but also. If you're prototyping, and everyone has a say in the matter, everyone should be able to speak to that. The designer, the developers, you know what I mean? Everyone should be able to understand from top to bottom. I fully agree with that. I mean, maybe your ops people will say, Listen, we can't monitor that stuff that you're putting in there. We don't know how it's going to scale when a thousand concurrent users get on there. So yeah, you're right. I would go a little further here and say, If you're doing this. Yeah, non prod, but production like environment. Get your customers close to you into that environment and say, here's what we're doing. Does this does this look right to you? What would you change? Right? Is this a value to you? Right? That's the early feedback. Now, you're not spending a whole lot of time there, necessarily. Imagine that. Compare the two scenarios where you're just developing and shoving that into production versus developing and putting it into a lower environment, production like alternate production and getting them in and say, put your hands on this and let us know what you think. How, how much of a stretch is that really? Clickable wireframes are so advanced now you can tie them into a real database. Like, I mean, they're very, uh, compared to five years ago, they're very advanced now where they have a lot of capability. I mean, they, they can even integrate with APIs, believe it or not. Like some, some wireframes. And not every button needs to do anything. No, and they certainly don't. But the point is, you could be, when you update the wireframes, or when you get to Some intentional spot that you get to in your prototyping experience. You can send out an email blast to your, to your power users, because that's what we're talking about. Now, when you say, Oh, we're going to promote it to this lower environment. Not every single one of your users is going to have access to the lower environment, but they're probably going to be a select group of users that you value, right. To send out an email blast that says, Hey, , these new features are in this environment, if you want to go, you know, poke around and look at them. And then, , when you do, here's the link to my calendar. To schedule some time to talk about what you see whenever you want to schedule some time and I, the product manager will get on the call, you know, obviously it'll go to my calendar. I'll get on a call or maybe it'll go to a team calendar, whatever, right? Get on a call, bring my tech lead. We'll talk to what you think about the new features we started in this podcast talking about, okay, the reason that you would just Just say, Oh, if it's, if it's real easy, we'll just send it out and we'll just code it and send it out. You did that because it, the speed of delivery, you know, whoa, it's easier for me to just throw something out there rather than spend a bunch of time talking about it. And it's, it costs less. We don't have to engage the designer. We don't have to engage all this. We don't have to send whatever we can just code it and send it out. You can do the same thing with clickable wireframes. Sure. And it costs you, I would argue way less to send it out and an email blast to your beta customers or whatever, you know, alpha customers, whatever they are. Yeah. Yeah. Schedule some time with your product manager and have the people that really are interested in making a difference. Connect with you. I mean you can do outreach to you can send them screenshots or whatever, right? You can do outreach director outreach. Sure, because you know, if somebody is reaching out for you to get time on your calendar to tell you what they think they're invested in, they're really engaged at that point. I mean, you really have to value that kind of feedback because usually what usually is a product manager. What I deal with is I have to go beg people for their feedback. I have to send out 100 emails and I get back maybe five. Yeah. And out of those five, how many are willing to have a 20 minute conversation with me? Very few, right? Right. And that, that, that is a reality of asking for feedback. Which, which, which should be a whole different podcast. You know, asking for feedback. I agree. It should be. It should be. I mean, look, I, I immediately think about, you know, things like putting out something that really appeals to them in the email blast and say, This is so important, right? We need feedback by this date, and you know, the top five people will be on this little, almost like a, you'll have a, you'll have an event, so to speak, right? So the top five people will be featured in that. Basically, create a sense of urgency for them to engage with you. Otherwise, they're going to say, Oh, yeah, I'm busy right now, but I mean, you know, you know, the other thing you can do if you if you're a product manager, you have a budget and you're not in the government space, I have to point that out, you know, because in the government space, you can't do this, but in the in the B2B space. You can offer, , like a fabulous cash prize. Basically, you can offer your incentives. Compensation for their time. Hey, , 30 minutes of your time. Review and we'll send you an Amazon gift card. Yeah, an Amazon gift card. You're right. I mean, there's techniques to, you know, lure people in to do this, right? In a timely manner. That's what I'm saying. Absolutely. Cause you put this out there and you want quick feedback , the interesting thing about this is if you're not thinking this way, and this is like a revelation to you right now, like how much. Is feedback on that new feature worth your company? Is it worth 500? So you get five reviews, you give me a 100 Amazon gift card. Like is, is, is 500 worth it to you for unbiased direct user feedback? It's worth its weight in gold, in my opinion, on a feature that you launched. For sure. Look at the flip side of it, right? So say 500 bucks. Don't do that. But what if something really, you know, it hits the proverbial fan. So to speak, right? Yeah. Or you, you suffer rep damage and that is a lot, lot worse. Right? Right, right. So I definitely, yeah. 500 bucks. Sure. Do it. There's some easy stuff here. There's some lou hanging fruit here. I brought this up because , somebody that I interacted with on my normal day to day, they brought up, Hey, why don't we just build this? We don't need to do prototyping or whatever, and,, I didn't feel good about that. The hair on the back of my neck, it all stood up, and I was like, something's wrong here. My spidey sense is tingling, and I don't know why. I couldn't, I couldn't exactly describe Why I felt, , negatively about the statement in the moment, but after thinking about it afterwards, there's so many opportunities for learning and to do a better job if we just try these prototyping, uh, steps. there are very few cases where it makes sense to just build it and throw it out there. You know, I, I mean, I, I'm not completely on the fence of we have to prototype everything. I, I, I do think that if we were small enough, And we had a, controlled group of users. This tactic would work if, if our app was small enough, we were very early in product market fit. Is what I'm saying. I think you also have to tie down to specific, , small team use cases or personas, small team, then it could work. Yeah. Team, few personas, small team. Yeah. Very, very specific use cases. Yeah. I mean, very small company type of, you know, it's a calculated risk at the end of the day. I mean, that's how I look at it, right? You gotta calculate the risk factors appropriately. Yeah. But you know, having said that though, it's not like we're saying, or at least I'm not saying that you don't ever do that. You always ever do all of the due diligence. There's a middle ground here, right? And that's where I mentioned that, that , kind of like a, a preview environment where you can put this thing into quickly and get your customers in there, again, quickly. It might not be two hours, it might be a day, but that's okay. It's worth spending that time rather than rushing it out there. That's what we're saying. That does make sense for the, for the companies that, they have the money for having a middle ground environment like that. Not every company has that, cause there are certain development practices required for you to, to, to be able to have a middle ground environment like that, you know, that has all, you know, all the data of all the production data. But in this, you know, sandbox that you can't mess anything up or whatever. It's not a technical reason why they don't have it usually. no, you're right. Yeah. Cause it's easy to replicate it. It is. Right. Yeah. If you listen to this podcast, you now know that it's possible. And, , you now know that it really doesn't cost that much more. Yeah. So go, go, go do it. You have our blessings. Absolutely. And also you have our blessings to like and subscribe. Oh, please like and subscribe because you'll get more nuggets of wisdom like this that's worth every penny you didn't pay for.