AA159 - Exploring the Evolving Role of Quality Assurance (with CEO Bob Crews)

0:01

welcome to the arguing the Agile podcast where enterprise business agility coach Om Patel and me product manager Brian Orlando debate the real world challenges Agile professionals will face. We are not here to sell you anything. We are here to argue about Agile so that you don't have to. Welcome to another episode of arguing agile today on the podcast. We have Bob Cruz, who's the founder and CEO of checkpoint technologies and they do wonderful things. I'm going to let Bob talk about what they do and how they can help you. So first off, Om and Brian, thanks for having me. I am the, as Ohm mentioned founder and CEO of Checkpoint Technologies, co founder of Checkpoint Technologies. And we have been in business for over 21 years, headquartered in Florida. Have done business in 42 of the 50 states. And we started off from day one with a focus on quality assurance and software testing. So I've been in it for, Oh my gosh, I'm going to say about 36 years now, start off as a developer. And then at a certain point transitioned into quality assurance and software testing, especially test automation, absolutely Fell in love with it. And then at a certain point, just for a number of different reasons, I said, you know what? I think we've got a special type of organization we can create that would provide a lot of , value to organizations When it comes to quality assurance and software testing, or when it comes to quality, because that's something that's been lacking. Fantastic. Well, this has been a long time coming. We wanted to tackle this topic, so here we are today. Yeah, And anyone who listens to the podcast knows that qa is how I got my start and here I am now many many years later several dog years later, that's okay. Don't don't worry. Bob. You're you're fine. You're only five dog years old oh, I like that. I like that. Like anyway, here I am three dog dog ish years later, and now I'm in product management, but I feel like the special sauce that I bring to product management is the years that I spent in Q. A. Why don't we sprinkle sauce? Isn't it liquid? It should be like pouring it on or drizzling it on. Anyway, it doesn't matter. Keep going. So it's super interesting to talk about Q. A. As a business model. That's not something I would expect. Uh, companies to focus on. Well, one of the things that that motivated me was I was on an incredible project with the FBI started off there as a developer, team lead, had a great team of developers. And then over time, They asked me to transition over to the QA and the software testing side. And one of the things that I realized was that the quality assurance and software testing, I compare it to chess, relatively easy to learn, but difficult to master. And that's something that in my many years of doing this, it is still overlooked by organizations, that fact. So many executives. just believe that it is easy, that it's not difficult. They don't understand the nuances and the complexities in the process, in the tools, in the people. I think a lot of times people think about quality assurance as something you do after the work of development is done. So kind of inspect for it. At the end, as opposed to making it in throughout the process. and that's a great point because when I started in my IT career back in the eighties, it was all about waterfall and, and what you just said, Ohm, very, very true. But now with, with the popularity of, of agile, which I love, but I don't think it's right for everything, but I do love it. That has made people realize you can make it, Part of the development process. There's no real reason why you can't. Now there are some outliers there that you have to respect and, and accommodate, but that's relatively easy to do as long as you recognize that fact. Well, welcome to arguing agile. Bob. What, what type of project would be, there's probably people screaming right now in our audience. What type of project would be more suited toward waterfall? Because again, This is the nice thing about the arguing agile podcast format. Om brings a business agility and I bring product management so we can pretty much analyze it from both sides. I don't know if I would ever prescribe something being, uh, uh, optimized for waterfall. And a lot of individuals may argue that certain types of applications, those that have a high risk, where should there be a failure, it could result in loss of life that that might not be suited for agile, but I'll go to something which is much more common sense when we're talking about Pure software development and applications it, you can typically develop a module or a simple piece of functionality in a very rapid amount of time, but I also do work. We'd also do work with clients that deal with embedded systems embedded in hardware. That are going to take days, sometimes weeks to develop, put into the software, the mechanical pieces that they need to be tested. That's not something that can necessarily be done in hours or days. So that might be something where. If it doesn't remain waterfall, then I might call it almost an extended agile. So, so that's something that I would say might be the right road. The other thing is what are your competitors doing? And is there a competitive reason you need to be able to keep up with them when it comes to releasing the changes, into prod. Yeah, timing to, you know accommodate market pressures is one thing. Yeah. But there's also this idea about what is the cost of failure? Right? I mean, if you're going to inspect for it at the end. Then that has an impact on when you can finally fix it and go through the cycles again and get to market. It's obviously going to be later, but what is the, what is the real impact of that? Not just the competitors eating your lunch, but you've also got some reputation to defend at that, right? You're always late to market, for example, or your software is. Subpar suboptimal when it comes to quality. Absolutely. You know, it's organizations and I love Om that you mentioned cost of failure because a lot of organizations do not take into account cost of failure or risk avoidance. You know, what, what will happen if there's a failure? I feel we're now we're squarely in the QA versus QC realm. We are. Yeah, I wanted to move us. Well, if you have an idea that it's the difference between, again, my background is QA on the podcast, between QA and QC, it's, is the difference between being agile versus waterfall, the idea that I'm going to do All the coding and all the development upfront, and then the solution gets handed over to you and the QA team, and then you check all the boxes. And that's, that's not really good enough. Well, I don't, I don't agree with that. Well, you're on the right podcast. What, what it's all about arguing, right? Arguing agile, arguing, whatever we come up with., whenever we've met and we talk about what checkpoint technologies does, you'll hear me use the term quality assurance and software testing. Why? Because very few people, no offense, but very few people know what quality control is. Quality control is software testing. So rather than say we do quality assurance and quality control, I say we do quality assurance and. Software testing, quality assurance is all about the activities you do to prevent defects. Quality control is all about the activities you do to detect defects to find them and then correct them. So even with agile, it's interesting the arguments you can get into as to whether or not something is a quality assurance or a quality control. Initiative or activity, but my, my challenge, I tell people this, the simple way to answer the question is, will this improve your process so that next time you might prevent a defect? Yeah. That's where we enter on the podcast to see so I, I have software development teams, I have scrum teams, and we have maybe a tester on the team. How much control, or visibility, or inspection are you allowed to do on the process? When I was a product manager on contract, and when I stopped being a QA manager, I started doing product management for the first time mainly in small companies. I would start realizing that the QA people that were there were really only ever asked to test and the process wasn't really considered the bringing them out to the, to the business wasn't really considered and again, that seemed to run counter to a lot of my experience because a lot of my experience led me to believe that the process is what sabotages quality a lot, the system, the process is what, what sabotages quality a lot of the time, you know, in addition to silly things like you have to have it out by a certain date or, you know, by the end of a two week sprint or other things like we, we spend nine out of our 10 days of the sprint coding and then we hand it over to the tester on the last day of the sprint and we, send it over the fence. Yeah. Or worse yet, we'll, we'll code and we'll test next sprint. I've seen that too. But what you're, both of you are highlighting is essentially the ethos of. Inspecting for quality at the end, when I say at the end, I mean, after development is done as opposed to integrating quality throughout the process so that you're assuring quality, not inspecting for it and quote unquote, controlling, but there's no reason why your developers and your testers cannot be collaborative during the process from day one. Absolutely. One of the projects that I was on early on in my career introduced me and, and made me recognize the value and importance of a peer reviews and walkthroughs and in, in the peer reviews we would have, testers would actually be part of that from day one after the, the requirements. Were defined so that testers could start asking themselves, huh? How am I going to test that , and they could provide input as to whether or not they felt as though Eventually the requirement was redefined or interpreted correctly by the Specifications team and by the development team that is a huge benefit when testers are included from day one, even, even if you're in an organization that is still utilizing a waterfall SDLC. I was super lucky early in my career. All of the QA people at that shop were also considered business analysts. So if you think about the case where you just outlined where , what are the QA people doing the whole time developers are developing? Oh, they're not doing anything. Well, I'm not saying that's a good attitude to have, but at my shop, because they did have that attitude, they had the ability to put me in front of the business, in front of the development team, to do business analysis. Their answer was, was all the QA people will help to quote right requirements. And then because I got that early, early BA skills, I was very, very lucky. But otherwise that would have been somebody else's job. That would have been a BA's job or, or somebody else well, what you just described is better than the former description, right? Which is testers only show up after development's done, and they have things thrown at them over the wall. I can share an anecdote. So when I was working as a contractor for, Consultant for Boeing. It wasn't just software. It was embedded systems, etcetera. Avionics control systems. Testers were integral part of the team from day one. When I say day one, I don't mean when developers start work. I mean, when you start talking to people about what is it that we're trying to do and why, including all the conversations with contractors and subcontractors that that world is full of. Subcontracting things out, but we had testers in there so they could a ask the questions early and ask the right questions. Some of the times people say, we need this to work really, really well. It cannot fail. What does that even mean? It cannot fail. Things will fail. There's moving parts. Absolutely. Right. So that's where the testers would come in and say, let's define that. Right. Yeah. The grade of service is what we called at that time. Yeah. Yeah. It's, you know, testers. Right. Test, good testers should have a different mindset and, and different strengths than a developer. So and one of the things that, that I, I truly believe is that a really good tester is going to have an excellent attention to detail and, and they'll fine tune things and put them in much more precise terms. To make it more realistic to test, it kind of makes you wonder centuries ago, you know, when they built the pyramids, the Egyptians built the pyramids. Even today, when you look at those, I know you're laughing, but look, when you look at the, no, listen, when you look at the offset between one level of the stones to another, it's, it's like within millimeters. And you know, they didn't have the instrumentation we have now. Did they have testers? You wonder, I, I, I firmly believe they had testers. I like to think perhaps they had testers from the predecessor of checkpoint technologies, but nicely done. But yeah, they certainly, they certainly had the testing mentality, the point you were making that, that mindset. Yeah. In the builders that were building it as well. And they, the testers that went in at each stage, not when the whole thing was built and go in and say, Hey, level 32 is off by an inch, right? Every stage they would say, this needs to be redone or it's good enough. They, they had to, they, they had to do it at every stage. Absolutely. Computer modeling today allows us to simulate all of that accurately. And what the scientists have found when they do that is. Every level is within millimeters, not hundreds of millimeters, tens of millimeters. You got to wonder, 5, 000 plus years ago, how was that possible? It's got to come down to a mentality and that deliberate intent to build something that is of high quality, the highest grade of service. Okay, you, you have Om, you have just moved visiting the pyramids up considerably on my bucket list. I strongly recommend very nice. Yes. Yes. Yeah, 20 years, Bob. That's what is stuck in my brain., 21 years. Yeah. Yeah. 21, 21 years, 21 years. A lot of changes in 21 years. What are some of the changes in the business, like the business of QA in 20 plus years? You know, it's, it's very interesting. I've seen a slight, slight, I'm disappointed that it hasn't been more, but a slight increase. And the amount of respect, quality assurance and, and quality in general receives again, I, I started my career off as a developer. I have tremendous respect for developers. I understand the challenges but excellent testers have, have a skillset that, that people need to understand you know, to be, to be an excellent quality. Assurance individual and an excellent quality control individual. It takes education. It takes a certain amount of experience and it takes technical aptitude. You also absolutely need to understand. The business. And then there are certain mathematical things that you need to understand as well. I mean, you know, we could start throwing out different terminology, equivalent classes and things like that. But which do you use them every single day? Of course not. Just like doctors or lawyers don't use everything they learn every single day, but it, it helps you to understand it and it improves your skill. So, so that's one of the things it has improved. Not as much as I would like. The other thing is just out when it comes to. To automation, because that's another very area of expertise that I have is functional automation, especially when we started 21 years ago, mercury interactive for those of you that remember windrunner test director. Yeah. Yeah. They, they were, they were the big dog. And, and now you have a lot of excellent organizations as well. Elect excellent solutions. By Inflectra, Tricentis, these are some of our partners. Selenium is phenomenal in the right setting. As well as the Mercury Interactive Solutions are still around, now owned by OpenText. So the increase in solutions as well. And then also just the way that, that Agile. CICD and DevOps continues to grow and DevOps especially has increased the importance of automation. So those are some of the big changes that I've seen from a business perspective. There's been several there. And I mean, I've seen some of the biggest changes in the past. four years because of the pandemic and COVID. So those, those are some of the many changes that I've seen. the DevOps category specifically, I'm very active on different platforms like Reddit, for example, and , believe it or not. It's good for things other than just cat pictures or people yelling into the sky or whatever. And I don't know whatever else. And then in the QA subreddit, there was a question a while back about a QA shop who, the, the QA people at that shop were being asked to do DevOps tasks, meaning set up pipelines and deployments and whatnot. And I remember responding to that something along the lines of the, Deployment of software through development, like the release pipelines, in the strictest sense QA. Sure that's not what your main responsibility is. However, if you can add that on top of your skill set, Oh boy, like that, you, you are now like 10 X more valuable on the market and anywhere else that you want to go. So, you know, even though you may not see it directly in your, in your wheelhouse, it's a, it's a great skill set to learn well, in shops specifically that have very weak DevOps presence, if any, you're right. You're right. Right. And, and QA has that responsibility for a very specific part of the pipeline that you're talking about. I mean, obviously the developers are responsible for a certain section of that, and then it gets deployed into the next level. And QA has a responsibility for that as well. And it's. Because a lot of organizations, especially when they're transitioning to CICD or to DevOps, again, they're transitioning, they have a real challenge understanding how. All of these 1000 test cases that they now have are going to fit into their pipeline and they're going to automate all of that. And I've got to share with them that, well, yeah, you do need to automate 100 percent of what you have in your pipeline, but now you need to be more selective about what you put into your, your pipeline. And that's where. Risk analysis, which I, I speak on quite a bit and I absolutely love, but that's where risk analysis comes in as well as impact analysis. So that, you know, maybe before when you had those, and I'm just making up a number here, 1000 test cases, use cases, user stories. Now, you know what, you're going to narrow that down to 700. Those others you're still going to do, but perhaps that's going to be in a one off. Type of situation. And now you're at a point where the development of those skills for the individual, the, the quote tester, I'm doing quotes for all the people who aren't watching me, the people sitting in those seats, they don't look that But what what the process that you just described that is a function of risk management and that is a function of product management and What the, what the QA people or the testers or whatever you wanna call it, the what the QA people can help with in that case is analyzing the risk to the business. What test cases can we take and choose? Which ones can we choose not to run based on some evidence that we're seeing. Choose not to choose that or shovel off into the category of a low risk to split my thousand test cases out and say, okay, only 750 of these is core functionality or whatever the number. And then you call that critical path or whatever the non critical path, whatever that's called now. And then in that case, like product management ends up delegating a bit of their role, a bit of their leadership to the testers or QA people on the team. That's a, that, that seems like a natural evolution to me. Yes, absolutely. So, so you've to be a top notch. I'm not going to say tester, but a top notch test team. You've got to have someone who knows and understands The business, you, you've got to, so that they can understand, all right, what is the likelihood probability that this functionality that we are testing will fail once it gets into production and what's the impact to the business should it fail. So everybody has heard this before, but you've got to have somebody like that on the testing. A lot of times what I'm seeing is First of all, I want to say you're right. It is, it is the business of product management to say what's going to happen, right? It is a function. But a lot of times what I'm seeing is product managers don't really get involved in that up front. They get involved when the proverbial, you know, You know, what hits the fan, but often they don't, I don't know why they don't, maybe they don't have the expertise perhaps. So when you talk about things like the team talks about things like code coverage and whatever, yeah, that kind of goes over a lot of people's heads, right? They don't really care as much or they don't understand it. I don't know which they should, they should understand. Cause it all leads to how good is. That release at the end and what then to your point about risk impact risk risk analysis It feeds into that so they can make informed decisions, right? Yeah. I mean, this is a function of product management. The risk to the business, you know, is, is it viable? This is one of the, the Marty Cagan, the the four illities of Marty CAgan. Is it valuable, usable, feasible and is it viable for the business? And honestly, I think it just makes you, you know, if you're aware of these, I think it makes you a better advisor to the product manager, just to say, Hey, let me take care of this for you. Let me offload this from your responsibilities or from your burden. And you know, I'll give you a recommendation. You can look over my notes. I'll make everything transparent to you. And and then you can make your determination. But you know, I'll end up being your expert on this that you kind of defer to, and it makes you a more effective team member and it makes you more effective. Yeah. It makes you a more effective member of the team. Yeah. Well, the point you're making there, Brian, I, is the way I interpret it and, and it's very true. In quality assurance and quality control,, we need to help bridge the gap, the gap between IT and the business. IT is going to deliver their understanding of what the software application system, et cetera, is supposed to do. And then the business They have their needs and here's what they want it to do. And you've got to bridge the gap. I was fortunate enough. It is at New Hampshire university. Oh, I had a professor in it back in the early eighties who used to say, never give your customer what they want, give them what they need, and it's our job to help them realize what it is that they need. So that's one of the things that, that I've always tried to do in my career in a very firm. But diplomatic way with customers. And, you know, so I need to bring up one other topic that we haven't talked about is artificial intelligence and the internet of things. I mean, you do a phenomenal job at moderating the lean beers in the area. And we had a great question recently at, at the lean beer in Tampa. Where somebody asked, does, you know, is artificial intelligence providing any value? So I hope you don't mind me changing the subject. That was perfect. But that had some great discussion. And that you know, I remember the, the, there was one individual who brought up a very firm, no, it provides no value at all. I think this was Mike Miller's point. It's like when he was screaming into the void,. Oh, okay. All right. All right, Mike. And then, you know, I shared it just, it depends on what your expectations are. You've got to set realistic expectations because if you're looking for AI to do the work for you, then my answer would be a flat out no, but if you're looking for it to, I'll, I'll just use the analogy, write a draft and then you're going to go in and. Edit it. Whether it's having it create requirements, use cases, user stories, a test plan. I think, you know, if you use it wisely, it could provide significant value. It'll be interesting to see where it goes. It sure will be. And what it's doing is changing extremely rapidly. Yes. So when we say where is it going to be in three years, five years, that's too long. I think we need to say what's it going to be in a year. Yeah, six months. Right, right, right. Because things are changing so quickly. it is. And three to six months is probably the correct time to reevaluate what the solutions are that are available to you. We kind of been going in and out of skill development, but I skipped over talent acquisition. So I, I skipped over it and finding great QA talent and then bringing them on again only because I've been a QA manager before and I had to find and develop and create a good team over time, mentor and coach a good team over time. the interesting part about finding great talent, looking for, for, and looking for and finding great talent is that you have to be looking for where they're at right now. To find the best people that could go to the next stage to go to where you need them, right? So you find a good a great QA person who's working currently in a QC role or in a tester role to say, Oh, I understand like you're in a role now where they're not really asking that much of you or because of the type of the organization they're keeping you, you know, in this, in this box. Where they're not really allowing you to, to press on things and product management or DevOps or other things. but understanding the broader process, you understand where these people could be making an impact to help with quality end to end dealt with, the process and all the activities involved within. Now, I would imagine you'd be, you could be able to take them from the organization that they're in and drop them into any other organization and have them really excel I mean the, the, the, the special, the special formula is your ability to spot these people yeah, absolutely. That, that's the secret sauce right there is how do you spot that person, right? Or the other side of it is, which is you have people that know what to do. Maybe it's the same side that they're a hamstrung, right? Because their organization is more focused around QC and receiving things from developers. But this person. People, they, they really get it, you know, there's keeping abreast of the latest developments in the quality world. So yeah, that's a, that's a good topic. How do you, as a, as a company, how do you go about finding talent that you can bring to bear? To your customers business problems. Well, My Number one preferred method of finding talent is always referrals So having and I'm sure everybody understands that you know having a having a great team member Refer someone who they know well And they speak highly of I, I'm one, I respect other people's opinions. So that's one thing from, from an interviewing process, I dive deep and some of the things that I look for, nothing new. I look for passion. I want to know that, that an individual is going above and beyond to improve their knowledge on software testing and quality assurance. If during an interview they tell me, oh yeah, I attend lean beers every chance that I get, or I listened to Joe Colantonio podcast things along that line. I'm making an effort to get my ISTQB, ASTQB certification. That means something to me that shows a level of effort and a passion for software testing, quality assurance. The other thing that I do, and I've actually developed this into a topic in which I've been a keynote at a number of conferences, and I might use brain teasers, games, and puzzles during the interview process to. Help me identify any particular skills or aptitudes that individual might have. So there are certain brain teasers I might use to develop if a person is excellent or good at abstract reasoning, things like that. So those are some of the different methods that I use. We also have a few different resource partners I might utilize. Who understand what I'm looking for because culture and and when you're when you're a small organization like ours who does great and very large things. I mean, we've done business in 42 of the 50 states. Our culture is we are very close knit organization. I've actually had employees that are not comfortable In that type of a setting, they prefer the rigid robotic, in my opinion culture of, you know, the big, big corporate world. So these are some of the things that I look for in some of the ways, you know, that I try to find people. So people that are listening to this podcast, and if they've listened to others know that I'm a big fan of the journeyman model, bringing people in and then showing them. The trade right and getting them up to speed as opposed to just simply buying the talent, so to speak, which I'm not saying that's wrong, there's, there's space for each. Have you used that at all? If you have, what has been your experience? Have you got people that have come in and you've kind of really enabled them from within to grow in a place where they need to go? Absolutely. I'm going to give a shout out to one of our employees. Gentleman's name is Matt Kubal. So Matt, if, if you're listening, Matt has been with us. Oh, my gosh. He's he's coming up on 10 years. He's coming up on 10 years, and he started with us with excellent I. T. Knowledge, but no experience in I. T. He had a degree in mathematics, but more than that, he's been It was very obvious to our organization that he was a quick learner and he was eager to learn great personality, very good at explaining abstract ideas in layman terms. And Matt has grown with us we're now he is an. An account manager for us who is still very much billable as an engineer, and now he is a very senior and an expert in his own right. In test, functional test automation and, and that was all internal. We provided him training. We provided him with the tools, but he took advantage of it, he had the initiative. So yeah, kudos to Matt. That's fantastic. I'm so happy to hear that story. Cause it's very lacking these days out there in the industry in general. People are just simply using people as resources. And especially, this is definitely the podcast to show up for Bob. A lot of people just view testers as clicking around on the screen, manual testing, that kind of stuff. Right. And they're, they're really kind of diminutive about the experience. I don't know what the right word is. If you can get somebody in who really can question your quote requirements or the inspect the process, can explore test cases can be really detail oriented. Like you were talking about earlier on the podcast, talk about alternate paths maybe even someone who can do wireframing, sorry. Welcome to the podcast where Brian goes off on random tangents. Where I'm going with this is you, you, you maybe have the initial skill set that they start off with and they could go in many, many, many different directions. So me, you might have a UI UX designer in your midst. You might have a QA engineer. You might have a somebody who wants to be a developer. You never really know you potentially they might take my path where they become a product manager, Where I was asking a lot of questions, for example, along the lines of do, do customers care about this? How is the business going to take advantage of this? how are we going to sell this in the future? How are we going to scale this, for example? Those are all things that I would ask as a tester. We probably should take this, this subject offline for a future podcast. Maybe QA career paths. I think that would be really interesting and I bring it up because it's very interesting when you bring on a team of quality assurance or, software testers that you find out what they're interested in. And then you continually keep asking, okay, what is the next step for what you're interested in? And kind of, how does that line up with the market? It, one, one of the things that, that we, during an interview process and The career of our employees, we remind them they need to define success to me. They, it's, I need to understand what their definition of success is in order to help them achieve it. And I want to help them to achieve it. Sometimes that means, you know what? Telling them you're not going to find that here. I hate to say it. You're not going to find that here with checkpoint technologies, or I say you definitely will, but here's what you need to do. Are you willing to do that? So yeah, absolutely. And then when it talks about failure, I've always been, I tell people I'd love to fail because I learned from it. I learned from it. And if I don't learn from it, I, I get upset with myself. Yeah. If you're not failing, you're probably not taking risks. If you're not taking risks, yeah, you're not innovating. That's right. I mean welcome to a whole nother podcast again. Because again, if we could crack the code of why people are not okay with taking risks or delegating risks to their employees because that's where creativity happens. And by the way, if we can crack that code, Oh boy, let's bottle it up and sell it for 29. 95. Yeah. Or, or, or, yeah. Or two for 25 bucks. You're right. Why people are not okay with failure and why you can't name that in the environment and be okay. There is, there's ego. There's all kinds of other stuff but again, that's a different podcast. All right. So this has been a fantastic topic. It's been a long time coming. So thank you for all of you that have stayed with us this far. Yeah. Kubal. Kudos to you, Matt and the other two people that have stayed with us through the up until this time, let us know what else you'd like us to talk about. Down below in the comment section. And Brian. Thank you for all really enjoyed it. Really enjoyed it very much. And thank you everybody for listening. Yeah, feel free to connect with me on LinkedIn. It's Bob and Crews C R E W S. And don't forget to like, and subscribe, ring that bell down below.