Greg Schaffer: Thank you. Hi, I’m Greg Schaffer. Happy New Year and welcome to the Virtual CISO Moment’s first episode of season eight. It’s hard to believe that it’s been that many seasons so far, but happy to continue this. To start out this year, we have Corey LeBleu. He is the founder and CEO of Relix Security, a boutique offensive security consultancy based in Austin, Texas. Corey brings more than twenty years of hands-on experience in application and network penetration testing, having worked across boutique consultancies global enterprises and fortune five hundred environments and before founding relic security he held major senior offensive security and practice leadership roles and organizations including verizon wireless rather verizon enterprise solutions and set solutions where he helped build and mature penetration testing programs to find methodologies and advise executive leadership on real world risk we’re going to talk today about Among other things, we’ll probably touch on the evolution of offensive security, how penetration testing has changed in the age of the cloud and AI. I’ve got a couple of questions there myself from personal experience, and also what CISOs and virtual CISOs need to understand to get real value from the testing program. Having said all that, Corey, thank you so much for joining us today.
Corey LeBleu: Thanks a lot for having me, Greg. Go ahead. I was just going to say, like you said, I’ve been doing this for about twenty years and yeah, go ahead. Sorry. Oh, OK.
Greg: Yeah, feel free to correct me if people haven’t figured it out. The intros are often sourced through whatever’s on LinkedIn and then massaged through ChatGPT. My vast staff here puts it together to give me an intro. Vast staff of one. And that’s how the intro is. So sometimes there’s mistakes in there, and I’m happy to be corrected. But I hope I got everything all right.
Corey: Yeah, that was great. I kind of took everything I would have said and said it yourself.
Greg: Well, you kind of did because it came from your LinkedIn page. Yeah. So as we’ve just talked about, you’ve got over two decades in offensive security. And what originally pulled you into the idea of wanting to be a penetration tester and particularly penetration testing involving application security in general? And as the industry has evolved, and particularly with AI, what’s kept you there?
Corey: So I kind of started, got a degree in information really wasn’t that into computers. Took a Cisco class, kind of fell in love with that. Then my first job out of college, I was reading a Network World magazine and had some talk about penetration testing and how they were doing it from, you know, next to a pool. So I kind of got obsessed with that. Found my first pen testing job in mainly network security, much application security, physical physical security, social engineering, things of that nature. A few more jobs like that. Got a little bored with the network side, so I started playing with applications. During internal pen tests, I found and disclosed vulnerabilities in different software, including Apple and Citrix, like the Apple Web Configuration Utility. So from there, I went on to Verizon where I mainly did application security with some network security stuff on the side. Spent some time doing some source head review and application security at another company. And eventually went to a company called Set Solutions where I ran a team of about five people. And from there, just went on with Relic Security, figured I could do it all, and here I am now.
Greg: So I’m curious, as a fellow entrepreneur who made the jump to working for myself as well, first of all, congratulations doing that. That’s a huge jump. And it takes an awful lot of work and an awful lot of commitment. But just like any reward that is a lot, the risk is high, but the rewards can be high. And I know that I’ve realized that as well, too. But I’m wondering, what has been… maybe one of the most significant challenges of launching your own organization and what has been one of the major rewards of doing that?
Corey: One of the major challenges is just getting your name out there. Like it’s hard to get people to do business with you when you’re just a name or a new company. You may have twenty years of experience, but people are just more comfortable going with a Verizon type company because, you know, My dad would say, knew I ever got fired hiring one of those big companies. You know, there’s a perceived liability of going with a smaller company. One of the bigger challenges is just getting the client on the hook. But what’s rewarding is just being able to do it all, see it all, kind of have a little more direction, you know, see how the industry is going. see how your business is going and just kind of have the freedom to establish how you want to do things.
Greg: So, yeah, I think that I would agree with both of those. Certainly one of the hardest things is definitely getting clients. And I like to say that marketing and sales, they’re four-letter words, both of them. I don’t like it, and it is tough. But on the flip side, we as small organizations have – something that large organizations don’t. We’re like, instead of like, I think we say this on our website for vCISO services, instead of being a small fish in a big pond, you’re a big fish in like a small puddle when you’re with us. So you’re going to have that hands-on, really, you’re not going to be passed off from like junior member, junior member, junior member. So I think in general, one of the hardest things in our industry is to get out the word that you want to go with boutique firms because you’re going to get better service that way. That’s just a hard nut to crack. I get that. Then you mentioned about keeping up with the industry. I’m going to go to something that I know we talked about before we started the recording. Just briefly, I said I was going to jump on. That has to do with vibe coding. The reason why I asked this is I’ve gotten into doing some vibe coding using one of the platforms out there, Bubble.io. I don’t know if you’re familiar with that. But it’s provided me the way to put together an application, a web application that hopefully will help in this one project that I’m working on. But I’ve come to the conclusion that there’s potentially some security issues that can be overlooked really easily. It’s like, I kind of know what I’m doing from a security perspective because that’s my mindset. And so I’ve caught things. before it’s gone out. But what I’m worried about is those will be using this sort of tool to make applications, just trusting what the AI is going to say. Yeah, this workflow is fine. No problem. I’m wondering from your perspective as a penetration tester, do you think that there’s going to be more of a problem with these types of vibe coding, AI coding type platform generations, app generators? Or do you think that this industry is going to correct itself and self-police, so to speak?
Corey: I think one of the issues that we might run across is just how we’ve traditionally had with software development shops trying to ship it and make sure it works more than if it’s secure. So just making sure that code works versus security being integrated into it. Even with the vibe coding, just my experience of it, it’s sometimes it works, sometimes it doesn’t. So security on top of that, on top of functionality is a good thing to consider, yeah.
Greg: Well, I know one of the specific that was thinking about was I was using a tool called Zapier that talks about that can do a whole bunch of things. And in this particular case, I was taking information from one platform, moving it to another platform. And Zapier said, hey, you know, I can make you workflow real easy on this. And they did. And it worked. But what I didn’t realize until I started troubleshooting before I put it in production was that it was dumping things into an exposed S three bucket. And I’m like, you never even told me that, you know, you were going to do something like that. And, and then the Zapier AI said, well, no, no, no, no. I mean, we can make it secure if you want to. Those are the kinds of things that really, really concern me. And, and I think, I think that there’s a, there’s going to be a potential for some serious problems out there.
Corey: Yeah. We have to be very specific with what you tell it to do. And just neither related to vibe coding, just hallucinations. I have, been studying ai and i got two books and i kind of wanted to build myself a learning path and i said basically take the content these two books kind of put it together and well it started referencing things that weren’t there and then i asked it’s like i don’t even have the table of contents of this i was just assuming what it was in so it’s just arbitrarily referencing chapters that weren’t there so
Greg: Oh my gosh. And you found out about that by cross-referencing what the AI was telling you, I take it.
Corey: Right, yeah. So it had one good table of contents, the other one it didn’t even have a table of contents. It was just winging it. And then wherever I mentioned it to it, it said, oh, I didn’t have that. If you could provide it, I could provide a more detailed version.
Greg: Try to put the blame on you, right?
Corey: Yeah, exactly.
Greg: So another issue that I’ve seen sometimes with organizations and penetration testing, this is a little bit more of a traditional question issue out there. From the virtual CISO space, it’s not uncommon that we’ll come in contact with a new client One of the things we’ll ask for is show me your latest penetration test. And a lot of times, they haven’t done it, and that’s OK. They’re just learning. But what isn’t OK is when they present a penetration test, and this is one real-world experience, that was basically an OpenVAS scan that was misconfigured. And basically, the scan couldn’t find any vulnerabilities. because it was misconfigured. But yet the report that was written up with like some nice gold color, stupid graphics and this and that said, our team spent three to four days on this. And we’re happy to say that we have found no issues with your network and all of that and probably charged about ten thousand dollars or whatever for for basically about thirty seconds worth of automated work. So there’s good penetration testing out there and there’s bad penetration testing out there. Beyond that example, can you kind of give us all an idea of what a business and practitioners like myself should be looking for in a high value penetration test as opposed to one where somebody is just trying to check the box and maybe score like some business?
Corey: Well, for one thing, you want to value two different stakeholders like you want to have a good executive summary. business owners uh what the issues were some of them may not be technical and some sort of reasoning as to why it’s an issue as well as uh having you know meat potatoes technical report providing you know different details on how to what the vulnerability is how to fix it etc um a lot of my idea i don’t rely on scanners as much i do many applications and security testing so i’m usually just using One main tool, Burp Suite and manually writing reports. But you definitely want to have the executive and the technical areas. But one thing that I do also to try and bridge that gap in my reports is I write kind of a narrative, a play-by-play of how I tested everything and kind of the results of it. So it kind of takes you through what I’ve done and also shows that I haven’t just run a basic scan.
Greg: Yeah, and that kind of an approach really helps someone like in my role, in the virtual CISO role, because and I’m speaking solely for myself, but I think that this applies to a lot of virtual CISOs out there. I’m not the technical expert when it comes to penetration testing. That’s not what I do. But I do have enough technical knowledge and experience and chops to be able to understand which you just talked about, like the narrative, I can then take that and further distill that down when I’m talking with our client and say, this is actually what we’re talking about here, apples to oranges and then apples to apples and all of that. So there definitely needs to be that sort of sweet spot where you’re not completely dumbing it down Totally. And so you’re not obfuscating some of the technical knowledge as well in there as well, because it’s sometimes those little technical nuggets which can prompt change. And I think that also concerning the fact, too, you talked about audiences, you’ll have the executives that are going to be looking at the penetration test, but then also the IT suite that are going to be needing to maybe make the changes suggested. So they probably need some of that technical information as well. And I feel sometimes places go a little too technical. Like, I mean, there should be a difference between the penetration test report and a vulnerability scan report, I think. And a lot of places combine the two. Like, you’ll get a penetration test. It’ll have some context of stuff they found manually, and then it’ll have four hundred pages of scan results from this. It’s not how I do it.
Corey: Yeah, I think delivering context and actual business intelligence is the way to go.
Greg: Well, and that’s… basically translating the findings to business risk, because eventually they’re the organization is going to have to understand, OK, now we’ve found these these issues from the penetration test. What does it really mean from a business perspective? How do you work that into the report or into the presentation?
Corey: A lot of it I get by the CVS scoring metric, which actually does severity. It doesn’t really deal with. It’s kind of specific in the deals with severity, not necessarily risk. So it shows how serious an issue could be. It kind of leaves it up to the organization to decide, okay, well, what does that mean to us? So, you know, if they can take over a system, it’s a high severity issue. But if you have a mitigating control in place, it may lower that from a ten to a three. But a lot of that depends on you and what you think of it. I just present with you, like, kind of how dangerous the issue is.
Greg: And I know that there’s a prevalence of AI being used in just about anything. It seems like that you can’t do anything nowadays without something being AI-enabled. So a lot of it’s marketing. I get that. But there’s also been a lot of talk about how AI… may impact the career field, the job situation. How do you see AI impacting, in general, penetration testing?
Corey: I mean, I think it’ll have some impact. I do less of the network side than the application side. I mean, I do do some of it, but I see it impacting that more, just kind of automating things. It’s kind of like how vulnerability scans automated it. on the application side i’ve actually done some ai testing and like burp suite the main tool i use has some integrated ai which i’ve used some of that it’s interesting for kind of going down rabbit holes explaining issues but uh nothing to rely on by itself i’ve tried some third-party tools that have had uh you know less success than that they’ve been able to identify like uh portions of requests and stuff but it’s you know stuff that i can do on my just looking at it but um what also i find interesting is ai is an attack vector itself i’ve done some uh some research in like prompt injection and stuff and actually attacking the ai so that’s more integrated into applications that’s gonna be it’s on attack surface be it just your traditional network application attack stuff or like harmful content, jailbreaking, getting it to make back-end calls for you. It’s all new attack vectors.
Greg: Yeah, and that’s one of the things that whenever we have new technology, it’s like there is going to be another way that the threat actors can exploit, and we just have to expect that. But in general, I know that sometimes clients… they’ll have common misconceptions when it comes to the entire area as far as penetration testing goes. So from your perspective, what are some of the more prevalent misconceptions that you get from clients with regarding penetration testing, particularly if they’ve never gone through it before?
Corey: Sometimes they don’t understand that it’s just kind of a one-time snapshot. So they’ll try and reach out to you more and more and kind of almost reach like as if you’re a contract employer just just providing a snapshot instead of uh you know a hands-on play-by-play thing um i’ll say you know the frequency of testing some places fall into the loop of like oh we just need to test on this regular schedule every you know quarter every year and don’t take into consideration that really The most important time to be testing is when you have a major application or network change.
Greg: I know there’s the concept of at least continuous vulnerability assessment. Is there the equivalent of continuous penetration testing? And if so, is that really of any value or is it more hype than not?
Corey: It’s kind of sold. I mean, I would kind of lump it in with the continuous vulnerability assessment. The pen testing, like I said, it’s more of a snapshot time and it’s what can I do with these vulnerabilities? You know, the vulnerability assessment is to identify them. The pen testing is to show the value of them. And, you know, depending on the maturity of your organization, if you’re, you know, if you’re a fortune company, yeah, I could definitely see the, advantage of having a constant team constantly doing everything. But for a smaller organization, the constant vulnerability scanning is going to be better off with pen tests just occasionally showing you your deficiencies. Another problem I find with pen testing and stuff is just people try and play whack-a-mole with it. And I think it should be more showing you the processes or areas that you need improvement, whether it be your SDLC, or just your change management processes. So it shouldn’t be just identifying one vulnerability. It’s kind of like, what’s what’s wrong in my system? How do I fix that to cover blanket area of vulnerabilities?
Greg: So for the from the VC, so lens, What sort of advice would you have for someone like me to advise clients on their own testing strategy? I mean, I see both sides of the equation. I see one side that they’re so risk averse that they want to do it way too much. And then I got another one that says, well, we just want to do enough basically to check a box. What could we be doing differently to better communicate the correct, most efficient way for penetration testing scheduling?
Corey: I think mainly just conveying that changes to your network or application can introduce new changes and vulnerabilities and just the importance of testing as things change. Fixing one thing could open up a hole for another thing or like adding a new functionality to a program could completely expose a new API or database call. Or in my case, I could expose like an S three bucket with information without knowing what it was that I was doing.
Greg: So yeah, it’s a it’s an exciting world out there. What do you see? What do you see like in the next few years with regards to penetration testing, offensive security, how it’s going to evolve over the next few years?
Corey: It’ll be really interesting. I do think the AI will play a good part. I think it’ll be more mature. I guess that I’ve seen using in the application space, just kind of limited functionality, but I have seen kind of proof of concepts on other tools that just completely automate things. So I think we’re going to see a lot more of it. I’ll be kind of like, You know, and it’s been with the SOC where you have somebody monitoring different aspects, you know, a lot of it automated, but still have a human in it.
Greg: All this can be very stressful. I got pretty stressed out just listening to some of this. I got to tell you that it seems like that with me and with the security industry, I tend to see it looking at it through waves, ups and downs, highs and lows, peaks and troughs, whatever you want to call it. Just when I think that we’ve reached the height that things are flowing along quite better, I’m really scared about some of this, the vibe coding and the introduction of folks creating applications and opening up doors for such. And so it just from my own personal experience. And it’s not only our industry, but also being an entrepreneur is very stressful as well, too. And so we need to take time to get away and to help in a healthy aspect, deal with those stresses. I’m wondering what’s one of the things that you do to help decompress from those stresses?
Corey: I do a few things like I’m pretty big into weightlifting. I try and lift weights three, four times a week. I also have a vinyl collector. I collect records and two kids that take up most of my time.
Greg: Well, I can certainly relate. I can’t relate with the kids because I don’t have any children, but I can relate with the weightlifting. I’m currently in a… I think for the first time in my life, I’m trying to do a serious bulk phase. And basically what I’m trying to do is I’m trying to prove… or disprove a point. I’ve heard too many times that once you get past a certain age, you can’t build muscle. And I’m way past that age. And I want to be able to prove that you can actually build muscle without building fat if you do it right, even at my advanced age of fifty eight. The jury is still out on that. Now, I’ll know about it maybe in about a month or two, whether or not this has worked out well. But I definitely hear that. Well, what’s some of the things coming down the road for you, whether it be like in your professional life or with Relic Security, which they dovetail together? What’s coming down the road?
Corey: On my end, I’m trying to learn a lot about AI. We’ve discussed a lot of this. It’s here to stay in it. And I’ve been studying a lot of the theory about it just to be able to talk about more than it’s a black box. I find a lot of people talk about like it’s magic. So I’ve actually been learning a lot of the theory and the math and all that. Just trying to understand it more and be able to kind of predict where it’s going more if you can, if that’s even possible. it’s just a tool. Like anything else, it’s just a tool. So it can be unpacked. It can be understood.
Greg: I give you a lot of credit for wanting to go down that path. I just don’t know if I have the patience to try to understand at the mathematic and the logic level of what’s going on down there. I just, I get to the point where I realize, okay, it’s a tool. I like using it. I have discussions more often than not a yellow chat GPT because they’re wrong, but it’s okay. It gets me to think. And that’s really the most important thing. The way I use AI is that it gets me to think so.
Corey: Yeah. I mean, I’m just kind of, deeper in it than I probably need to be. But through studying the theory, I’ve been reading about different attacks and stuff. I’ve been seeing terminology, but I just would have no clue what they were talking about if I had not studied the theory. And probably a good piece of advice is that if you come across a piece of terminology or a statement that don’t ask AI about AI, try to research it the normal way because you never know if you’re going to get a hallucinated answer or not. Yeah, but it’s taught me a lot about linear algebra. See, you know, they told us we would never use algebra, but here we are. Now we’re using it.
Greg: So, Corey, it’s been an absolute pleasure having you on. I appreciate you taking the time and to be our first guest in twenty twenty six. Thank you so much.
Corey: Appreciate it.
Greg: And everybody stay secure.