An AI Security Maturity Model for CISOs, with Chris Cochran (SANS)

Hello everyone and welcome to SecureTalk. I'm your host, Justin Beals.
Last October, Anthropic, the UK's AI Security Institute, and the Alan Turing Institute ran the largest study of AI data poisoning we've seen. The question was how much bad data it takes to hide a backdoor in a large language model. The assumption had always been that you'd need to corrupt a percentage of the training set, so bigger models would need more poison. It turned out you needed about 250 documents, whether the model was small or more than twenty times larger.
In January, a group calling itself Poison Fountain took that idea out of the lab. They stood up a public site handing website owners links to an endless stream of poisoned data, code seeded with quiet logic errors meant to degrade any model that scrapes it. Their stated aim is to damage machine intelligence, and by some accounts a few of the people involved work inside major AI companies. It's a real attempt to weaponize a vulnerability we now know is cheap to exploit.
That works because of how these systems get built. We don't write them line by line, we grow them, and even the people who build them can see maybe three percent of what's happening inside. My guest today puts it plainly. An AI is a probabilistic system, and it doesn't care whether its factually correct, just whether you believe in it. So how do you adopt it without losing control?
Chris Cochran built a maturity model to help teams answer that. It has three pillars. Protect AI, govern AI, and one you rarely see in a security framework, utilize AI. We talk about why adoption belongs inside a security model, about giving an agent the least agency it needs instead of free rein, and about the risks hiding in cheap tokens.
Chris Cochran is the Field CISO and Vice President of AI Security at the SANS Institute, where he stands at the intersection of frontier AI innovation and real-world cybersecurity defense. A Marine Corps veteran and former leader at organizations like Netflix, Mandiant, the U.S. House of Representatives, Axonius, and NSA, Chris has spent his career translating complexity into clarity, whether guiding organizations through emerging AI threat landscapes, pioneering defensive AI workflows, or shaping the next generation of AI-security practitioners. His work blends deep operational cyber experience with cutting-edge research in AI governance, model risk, multi-agent systems, and adversarial AI, positioning him as one of the few leaders equally fluent in shaping AI strategy and securing it. Known for his storytelling, community leadership, and ability to distill fast-moving technical shifts into actionable insight, Chris is helping build the future where AI systems are both powerful and safe.
Join me today on SecureTalk for this conversation with Chris Cochran.





—-
Justin Beals: Chris, welcome to Secure Talk. We really appreciate you joining us today.

Chris Cochran: Thanks, Justin. Thanks for having me.

Justin Beals: It's gonna be a treat to chat. I think we have a lot of cross-connects. but the first one I have no background in, but I'm deeply curious about. So you spent much of your career building threat intelligence and security operations, NSA and US Cybersecurity Command, House of Representatives. I mean the resume is robust between Netflix and Mandy and Chris. and now you're fully into AI security. And so I'm a little curious about what your threat intelligence background l led either in experience or interest into AI security.

Chris Cochran: Yeah, I would say it's a mix of things. On the AI side, you know, I was interested in AI pretty early on. My senior thesis in college was on digitizing the human brain. but I would say from a threat intelligence standpoint, there are two things that really help me when it comes to AI security. One is the concept of I'm always looking for how things could go wrong, right? How could cyber criminals use a certain technology to do things against us, or even how do we use things, how do we use technology to protect ourselves? And so that's the first thing that I was really thinking about. 
But then the second thing is just the concept of storytelling, right? I love storytelling. Obviously you know, I love things like media, but threat intelligence is really storytelling, right? How do you convey a message in the shortest way possible to have the impact that you're hoping to have? 

And so whenever I'm talking to leaders about artificial intelligence, when I'm talking to them about AI security, I use storytelling just similar to how I would use it in threat intelligence.

Justin Beals: You know, you have obviously a deep background in the intelligence space. It's funny to me because as I've had conversations with folks that come from that background, they do tend to focus on this storytelling aspect, even in their work, then, because it is almost the perception of a story or thematically in a culture or community that as an intelligence officer, you're trying to read the tea leaves on and understand.

Yeah. I don't come from that background at all. I have a theatre degree, so I'm constantly curious, though, about storytelling and the power it can have. And it is very robust in the security space. So tell me a little bit about your work today at AI Sands, especially, I think, let's start with what the problem area was for some of the newest framework work, some maturity model work that you're doing today. Were you seeing a challenge and were inspired to work against it?

Chris Cochran: Yeah, yeah, exactly. So I'm in an awesome position at Sands where I spend the majority of my time working and strategizing with leaders all around the world around how do they wrap their minds around artificial intelligence, how do they get their teams to use it? How do they explain it to the board? How do they explain it to their C-suite counterparts? 
And I mean, honestly, when I was out there talking to folks every single day.

I have these dinners where we sit down with leaders, and we have conversations around AI. A lot of the same themes kept coming up over and over and over again. And I'm always one of those guys, and maybe it's my Marine Corps background, but when I see a gap where I see a hole, I tend to leap in and try to help fill it. And so I thought something like the AI security maturity model would help folks orient themselves around artificial intelligence and help take those steps confidently to improve their maturity and improve their posture at the end of the day.

Justin Beals: Yeah. You know, I'm very excited about AI, but there are a lot of threads around it that are a little that are scary as well. And I'm trying not to be too dramatic because I feel like there's enough, you know, pressurizing the system a little bit with very dramatic statements. In some ways, to me, it feels very normal what's happening with AI and kind of where we're at. It's not that innovative. 
You know, it's just a larger database with bigger model has run a more precise outcome. I mean, we're kind of throwing enough computing power at it to run the way we want it to. At the same time, it does things in very unexpected fashions. And if you don't have a background in computer science and kind of how they're built, it can be tough to understand where it's gonna go awry. Yeah.

Chris Cochran: Yeah, I would say when it comes to artificial intelligence, the thing that people have to remember is that it's a probabilistic system. And when you have a probabilistic system, that means that given an input, you can't necessarily determine the output specifically. And so when you think about the model itself, right, Dario Amodei says that you probably can see about 3% into the mind of a model, right? 

Some people think that AI is built line by line like classic software, but it's grown, right? You just have a shell of a model, and then you pump data through it, and then you start to try to retrieve the answers that you're looking for. And then once you get to a point where you're like, this seems good enough, you release it into the world. But with that system, you might have some underlying behaviors that you might not see until given a certain scenario. 

And so I mean, there are a lot of dynamics to using AI inside of an organization. And that's why you have to sort of do almost this defense in depth, even against the AI itself, not even necessarily from an attacker coming in. So I mean it's it's it's pretty wacky. It's pretty wild, but it's pretty exciting too.

Justin Beals: Yeah. You've banged my drum. You know, my prior work, you know, I've done even just simple Bayesian models, you know, for prediction, especially in the education space. And I used to tell folks when they'd say, Justin, hey, we want to build this model or this AI, or they want to do something with machine learning or data science. I'd say, Well, unlike just writing code, like if you ask me to write code like an application, I could guarantee you fairly close to when it's gonna be done and how it will work. It's kind of known. Like, we understand how to build it. But if I I have no idea when the model will be good enough, accurate enough, ready for consumption. And you're right, we'll just have to keep pumping data into it until it seems to do something beneficial. Or give up. Yeah, absolutely. So let's see, now one of the things that you know, you've talked about a little bit is the AI systems, the confusion from the leaders that you're hearing when you discuss with them, when you sit down and talk. And I think you point out something that's like very hard to know what to do about this work during the weekday, like when you wake up in the morning and show up at the office. It must be kind of frustrating to feel this wave of change, but not, you know, in some way being able to manage it. And I think that must have been part of the inspiration as well to pull together a maturity model-style framework.

Chris Cochran:  Yeah, a hundred percent. You know, it's when you think about everything in cybersecurity or really any pursuit of knowledge, it's always what do I do next? Right. If you were to take a test, the best way to assess whether someone knows what they're doing is hey, given a scenario, what do you do next? I mean, you could do that in education, you could do that in cybersecurity.

But when it comes to something like artificial intelligence, what we're asking leaders to do is not only maintain their day-to-day, right? Because leaders, practitioners, everybody, they're you know fighting fires just about every day. I don't know too many practitioners or leaders that are just sitting around tooling their thumbs. But I would say that then we said, hey, now here's this brand new technology you need to learn. And then on top of that, what are the security implications behind that technology? 

So It's really a whole new field that's really opened up. And then we're expecting everybody to just kind of, through osmosis, learn everything. And so, you know, it's hard enough trying to, you know, step out of your day-to-day role and say, okay, now let me orient myself around this new technology. Let me figure out all the security stuff. And so having something like a maturity model is, you know, it's I wouldn't say it's a silver bullet that solves everything, but it helps folks.

have a lexicon. It giv it gives a framework for folks to start to orient themselves around it, start to organize it in their own mind and in their own teams. And then they can start to take those steps forward towards maturity.

Justin Beals (09:16.853)
It you know, we're always trying to iterate forward, more accuracy, more reliability like a test is built on accuracy and reliability from software quality assurance assurance to high stakes education testing. we you must have looked around at some of the other maturity model certifications that are happening in the marketplace. Obviously, we're seeing CMMC grow quite a bit. Was there anything that you wanted to bake in with this iteration compared to those others?

Chris Cochran (09:44.809)
Yeah, I would say, you know, obviously maturity models, they have different use cases and and they're you know really useful for different organizations. I said with my particular maturity model, I wanted it to be as usable as possible, right? I didn't want it to just be an infographic where people are like just kind of Kentucky windage trying to figure out where they're at. And so, you know, being evidence based, it was a decision very early on, saying, hey.

I can actually point to the documentation that sets up this process or highlights this aspect of AI security. And then also, you know, it brings in some of the different concepts that I think are really important when it comes to things like agentic AI, concepts like lease agency, right? We have lease privilege, you know, it's been around for a long time, but lease agency is really trying to constrain a model or an agent around.

Exactly what you want it to do without giving it too much free will to operate in your environment and do things that you might not want it to do. And so when I thought about this, I didn't just want this to be an ego piece where it's like, yes, I made a maturity model. No, I really want this to be usable. And so it's really a living document. So as things come in, as people give me feedback about the model, I want to continue to iterate and make it more usable for folks out there, and I'm not gonna stop until I retire, probably.

Justin Beals: That's wonderful. Three major pillars to that usability in the model. Protect AI, Utilize AI, and govern AI. And I think what's interesting is that you included Utilize AI. What inspired that to be added? Not a common part of the security framework. Yeah.

Chris Cochran: No, it's not. It really isn't. And the biggest thing that had me include that piece was the fact that, you know, I do these dinners, these Jeffersonian dinners. I've been doing them for years at this point. I started doing them just in for cybersecurity in general. And then obviously for the last few years, they've been focused on artificial intelligence. And I would say still, even with the dinners that I have today, you know, be you know, between twelve and twenty-four, so leaders usually half the room are self-professed AI-sceptics. And it's really getting to the point where we see autonomous attacks starting to occur, starting to mature. 

Obviously, the adversary is using artificial intelligence. And one day I just said, Hey, look, half of you are self-professed AI sceptics. I said, From here on out, you are no longer an AI sceptic. You can't afford to be, because the adversary is using the technology. The technology doesn't care if you believe in it or not.

I said, so in order to fight fire with fire, you really have to understand this technology and start to leverage it. The analogy that I give is that I'm imagine if you're a fantastic mathematician and your entire dream is to work at NASA. And you go, and you're in the interview, and at the end of the interview, you say, Hey, you know, this has been great, but I just want to let you know that I don't use a calculator or a computer. How much of a disadvantage are you at against the folks that may not be near as good as you at math, but they use those tools? So just to say that, hey, I don't want to change my process, I don't like change, I don't like new technologies, I don't think it cuts the mustard when it comes to the utilization of AI.

Justin Beals: Yeah. Is there any area in the security space specifically where we see some leaps ahead or good advances from using AI from a defense perspective or a monitoring perspective in the security practice?

Chris Cochran: Yeah, I would say pretty early on, from a vulnerability management, pen testing side of things, it you know, we saw a lot of early wins. you know, companies like Expo, different organizations out there. And then, of course, we have the Mythos, Project Glasswing stuff, that you know has been in the news quite a bit lately. And so being able to find vulnerabilities has been one of those things that has exponentially grown.

But it's sort of like this double-edged sword because it, you know, it's great that we now have visibility into all these different vulnerabilities. The problem is now our to-do list is 10 times as long. So we need to start to look at artificial intelligence on how do we number one prioritize those vulnerabilities, but then number two, how do we start to close those vulnerabilities? You know, and start to improve the posture, right? Because it's great to find all the holes, but If you can't plug those holes, then the boat's gonna sink.

Justin Beals: Yeah. Chris, I have a question I like to throw at you a little bit when we talk about frameworks, because I think there's a common misunderstanding sometimes. How much of this particular thing in this maturity model that you're developing do you think has to do with how the business operates? And how much of it is a cybersecurity tool that gets put in place?

Chris Cochran: I would say it's more about the business process. And I would even say it's more than a business process, it's more of like an organizational framework. Because even though this is really geared towards security practitioners, the security program, the foundation of the model is governance at the end of the day. Because I feel like you really need to get the entire organization on the same page.

I would say that AI permeates just about every aspect of business. It's going to continue to become more and more ingrained, more integrated, and the more you have people on the same page, the more you're having communications around use cases like, hey, I want to be able to do X, Y, and Z. Okay, well, then what are the governance? What are the privacy, the compliance, the security concerns that we need to work through in order to make this a reality?

And so the more we bring folks together, the more we have these conversations, the better we're going to be able to mature the organization. But if you have IT or you have security just making decisions in a vacuum, that's when you start to run into issues. That's when you run into things like shadow AI. It's just better if everybody's on the same sheet of music.

Justin Beals:  It's I feel like it's a fairly common I call it a mistake, that's that's my personal perception, that when I work with companies or we're talking with them about meeting a security framework or outcome or assessment, that they start with the the IT things that they need to buy and not, you know, kind of that overarching operational characteristics that drives the decisions of what you might buy as infrastructure. Yeah.

Chris Cochran:  100%. And in fact, one of the biggest realizations that I've had is that when you think about AI in general, I think a lot of folks had top-down pressure to use AI. And so they started just implementing it everywhere. And then they say, well, we'll just figure out security after the fact. But the folks that are the most mature right now, the folks that are using AI to the maximum capacity that I've seen, are usually the organizations that truly understand the processes inside of their organization, whether you're talking about IR, you're talking about triage, you're talking about audits, you all all the things that you can think about. If you've been able to take it and dissect it into these nice steps, these nice digestible pieces that all fit together, and then you start to think, okay, where could AI be plugged into this system?

That those are the folks that do really, really well. But if you don't really have an understanding of like how should we be doing something like incident response, and then you just start throwing AI at it, it doesn't make things better. And in fact, it probably makes things a little bit more complicated.

Justin Beals:  Yeah. Let's I I'm curious a little bit about the scoring methodology that you decided to use for the maturity model. I think I'm very excited about this because I think when I think back to like a SAS 70, that's very past fail, or you know, we caveat in these words like exception into numerical scoring. That's actually a big leap forward, a lot of times in an assessment value. 

So, a couple of nuances. I think that it's evidence-based to begin with. Is that correct? Yeah. And then you move through to a ranking on the pillar or across the pillar. So you can start to get some synthesis as you move higher up the stack. Is that right? Yeah. How do you think about describing, approaching, you know, a methodology around rolling this particular framework out to an organization?

Chris Cochran: Right. That's correct.

Yeah, I would say think of the maturity model as a starting off point, right? I help organizations all the time work through the maturity model, and sometimes they feel like it's prescriptive, but it really isn't because there might be questions, there might be statements that don't really apply to a certain organization, right? Given an industry or a certain size. You really have to just look at it and try to understand what it's trying to get at. 

Like what is each pillar really focused on and then at each stage what does that really say about the maturity of the way you do AI security and so when I think about it I look at a model if I was an organization I would look at the model and I would look at each section and I would say okay I'll read through it I'd say where about would we say we are from this standpoint, right? What documentation do we have? What processes have we already gotten sign-off on, and then I would say, okay, so we're probably right around here. And then there's even a little chart where, and even the chart itself is just a starting off point from the standpoint of how do you index or weight each of the sections. because I wanted the there's nuance to different industries, and so even if you wanted to change the scoring of that, you could right, you could be creative, you make it make it your own. I'm a huge, huge philosophy fan and one of the best philosophers. I mean, you might even laugh, but Bruce Lee, I think, is an incredible philosopher. And one of the things when he would talk about martial arts, he would talk about taking things from different styles and then throwing away everything else that doesn't work. I would say similar to that, you know, look at the model. You can look at this model, you can look at any model, take all the things that are useful, things that you think apply to your organization things that are gonna help move the needle or help explain to somebody exactly why we need to take a certain course of action and then everything else throw it away. But I would say just start wrap your mind around it, see what works, and then start to move forward.

Justin Beals: I think this is part of getting that C suite together. You know, it and I like to call it like putting in an ounce of energy up front has compounding results way down the road. And if we can get you to sit down and do a baseline assessment, look through what's applicable, what's not, and just have the discussion with your colleagues, the end result will be a much more efficient and effective security program to operate with. Yeah.

Chris Cochran:
Yeah. It does. A hundred thousand percent. Couldn't agree more.

Justin Beals: You know, and also I think it builds that story and that culture 'cause people start to believe in the work that they've built.
I have used the joke way too often, probably, that you know, a framework's gonna, it's not gonna tell you that you have to encrypt data, it's gonna ask you how you keep data private. Yeah. this currently the the maturity model that you guys have developed, it's mostly being used as a self-assessment. Have you thought about a third-party assessment regime much? Is  that something on the horizon that you consider?

Chris Cochran: Right. Yeah.

Justin Beals: Or do you like more releasing the information and letting people decide how they'll use it?

Chris Cochran: So right now, with some of our customers, you know, I have been sort of like coaching folks along, like they're still taking it inside their organization and I might, you know, point them in the right direction or I might brief them on exactly how I would go about leveraging it. but will it become a service inside of Sans? You know, that there's always a possibility for that. But I do know that there are some consulting firms out there that are looking at it for their own utilization for their customers. So, you know, it's, you know, free to use, you know, take it, you know, use it, operationalize it, you know, build entire products around it. I I I the more the merrier from my perspective. But yeah, I mean it in one way or another, people are using it for those purposes.

Justin Beals: Yeah.  You synthesize some laws, I think, and existing like the ISO forty two thousand one framework. are there w are there others? I think EU has been the most forward leaning on the legal front, with this work. Yeah.

  
Chris Cochran: Yeah, they've from a legislation standpoint, they've really, really leaned in. In fact, when the EU AI Act came out, the SANS and OWASP AI, had a relationship. And so I got to be one of the contributors to the security standard for the EU AI Act. But yeah, the EU is pretty far out ahead when it comes to legislation and trying to wrap their minds around it from a governance perspective at the national level. How do you think about AI? How do think about AI security? But yeah, I would say that it's not quite the wild, wild west anymore. It's getting a little bit better, but we still have a long way to go from that perspective.

Justin Beals: Yeah. Okay. I have some more questions, but I'd like to dig into some other areas like some security aspects that I think your deep expertise in. Let's start with the news if you're open to it a little bit about the fable model and anthropic going back and forth. Any what's your take on that? I mean, this is real-time happening for us right now, so early days, yeah.

Chris Cochran:  Yeah, what's so funny is as soon as Project Glasswing came out, you know, I saw and I saw how they were controlling access around it, and I was like, okay, that seems like a good move. And then I immediately thought that with all of the progress we've been making in these models, and you look at the AI race, you know, the frontier AI lab race, I said, what is to keep a basic model, a model that everyone has access to in six months, you know, six months of progress from today. What is to say that there isn't a mythos capability in that model? And then now the entire world has access to that. And so, I mean, this is coming to fruition a lot quicker than I imagined. But now you have to really understand that these models, the more powerful they become.

They can become weaponized relatively easily. It's not like we have a bunch of cyber criminals out there making their own models, even though you know they tried and they've, you know, they had some success with that. But the most success they've had is just utilizing the models that exist, whether you have to jailbreak it or get around it, you have to get around the guardrails. They said, Hey, why reinvent the wheel when we can just use this here? And so I think it's a really stark warning as to just how important it is to understand what does the world look like when you give it such a powerful capability. I'd love to hear your thoughts on the whole.

Justin Beals: Well, if I struggle with I struggle counterbalancing two areas. One is I'm in line with you with the science, right? Like the computer science side. Like it's a great iteration, larger data points. I we use anthropic models, you know, just for our internal work. And I feel like they've done good on the models, and I'm still trying to figure out which one I like from a corporate governance perspective. But It's powerful  and we do use it from for marketing work and research work, and can do some analysis very quickly. Also, I'm struggling with Silicon Valley's hype machine, like day in, day out. You know, it's like there's so much smoke behind the curtain, and so it's so ephemeral sometimes what they claim.


And they use the hype machine sometimes because there's a business valuation issue that's happening. And these guys are way out over the economic skis. yeah. And so I think that on one hand, I want them to release these in a measured way. I liked when they said we're gonna let other you know, a select group of people take a look at it. It feels almost like a editorial board before information is released. We wanna review it for its reliability. but then, you know, when they took it away, and you hear rumors about weird interviews with Amazon and the government, gosh, it just makes it hard to trust what's going on, Chris.

Chris Cochran:  Yeah, I mean there there probably is a lot of theater. And  I try to almost keep even though it's my job to really understand what's going on in the world, I almost have to take everything I hear with a grain of salt. Similar to how I'm sure people are looking at social media, right? You know, with all the videos and stuff, like how is a cat playing a guitar like that? Well, 'cause it's AI, obviously. So, it's you almost have to you have to take everything in with a grain of salt, but then also you have to understand that.

Justin Beals:  Yeah.

Chris Cochran: There might be additional forces at play that we don't quite understand. And so it's almost like you have to say, almost you have to do threat intelligence. You have to say, here, here's the information we have access to. Here is my assessment, and here's how confident I am in that assessment. You almost have to do that with everything nowadays.

Justin Beals:  Yeah. If I play it forward, what I think, and this happens I believe, throughout my computer science career, is that things were very expensive at one time, became really cheap, really quickly. And so I think that's why I'm convinced that we have to prepare for how we're gonna use these things. And I don't know that this raft of businesses is going to survive, you know, that economic situation, but at the end of the day, these types of models are here to stay. yeah.

Chris Cochran: Yeah, these models are here to stay, and to the money point, these tokens that everybody's using today are highly subsidized. And I don't know how many people realize that. And when you start to understand just how expensive things could get, like I don't know if we're gonna become more, just more efficient at being able to use tokens, or what's gonna happen in the future, or it's gonna say, hey, surprise, here's the bill.

Justin Beals: Right. Yeah.

Chris Cochran: But people need to start to understand that that is something that's going to come to pass eventually. And so I think folks are feeling like, wow, it's always gonna be at this price. And of course, to your point, everything does become more affordable over time unless you're subsidizing. But I would say I'm hopeful that we're gonna figure out a more efficient way to number one use tokens, but also use power because power is a huge, huge bottleneck for AI, AI capabilities and AI training today. We're still early on in this whole technology journey. And I'm excited for it. I think it's, you know, one of the biggest changes, one of the biggest things that could have ever happened to humanity. And I don't know. There's a lot of uncertainty around it, but I'm still excited.

Justin Beals:  Yeah, I I'm excited too. It's the powerful tools. And when we bake an intelligence layer into the software that we build, we can, you know, increase productivity, and just I find myself to be able to dive into topics like very quickly with synthesized information. In truth, navigating the internet to find an article that really explained something to me that wasn't driven by a marketing team was hard to find before. Yeah.

Chris Cochran: 
There you go. Yeah, exactly. A hundred percent.

Justin Beals:  And I you know, one of the things, coming back around to the AI management is that this is one of those risks that the C suite needs to be aware of. You know, if you base your software development roadmap on the fact that you can have productivity around these cheap tokens, you're gonna get on the other side and all of a sudden you're paying twenty times what you were, and all of a sudden you can't produce as well, and now your roadmap falls through. That's a risk, yeah.

Chris Cochran: Yeah. That's a huge risk. And so, you know, one of the big things that I I was actually a big proponent of very early on is self-hosted models, right? Open source models, being able to start to pull your stuff together. We even have it, it's focused on digital forensics and incident response, but we have a really short, like seven-hour course that's focused on how do you take an open source model, bring it on your machine, and enable yourself to do different analysis from a deeper perspective. And I would say that that's even though it's focused on deferral for the second half, that first half is a master class in being able to just start to use stuff on your own system. So you don't have to worry about the tokens in a classic sense. But I yeah, I think more organizations are gonna start to lean on SLM, small language models, and then also lean more into the self-hosted so that they're not beholden to, you know, the AI labs from a cloud perspective.

Justin Beals: Yeah. It's one of our it's a big part of our sales discussion, that we self-host all our models. Made that decision a long time ago because I I have built enough models to not trust the accuracy and reliability of a model that we didn't build. So they're just starting there, you know. so I and if the models just get more efficient, then yeah, we'll use the existing computing hardware to throw them up.

Chris Cochran: Right. Yeah. Hey, I got around ya. Yeah.

Justin Beals: There's a lot of juice to be found in the software. Another area that I wanted to ask you some questions about is data poisoning in models because I think it's an area that you can give us a lot of insight on, and I think it's one of the newer vulnerabilities. Maybe talk about the landscape of data poisoning as a threat to companies.

Chris Cochran: Yeah, so even beyond data poisoning, right? There's context poisoning. You know, I'd I tell everybody, because anthropic did this research, you know, everyone had been talking about this big bad boogeyman called data poisoning. And they were just like, What exactly what does it take for a model to get poisoned at any size? And so they set out to do this research, and they found that if you had say  250 weaponized documents or files on the open internet and they just somehow got ingested, that you could data poison a model, which is you know, scary to think about because if you think about a model, it's probably trying to train on the entirety of the internet. And if you have 250 instances of this thing, you could, you know, poison a model. And now of course, you know, the front frontier labs aren't sitting idly by and just letting all their models get poisoned.

But there's still a lot of room to grow in terms of protection around data poisoning. Then I start talking about context poisoning, which is you can still poison a model based on well, maybe not the model itself, but you, the process or the agent itself based on the tools and the information it has access to. Anytime you have like an interface that a model or an agent is dealing with.

There's the chance for some type of poisoning. And so, I mean, we really have to start to think about that when we're doing threat modelling for how do we architect agentic systems inside of our environments? Okay, it's going to have access to this. What is the worst-case scenario if this data is poisoned? Or how do we continue to control the controls around that data so that the model doesn't eventually get poisoned? And then you look at things like OpenClaw.

And then you're like, yeah, let's have Moltbook and give our agents access to basically their own version of Twitter X or whatever you want to call it. That just sounded crazy to me. And you know, it from the the IT, the technologist in me was, hey, that's pretty cool. Security person in me was like, that is terrible.

Justin Beals: Yeah. And this is not an esoteric threat. There are groups that are just anti AI that are developing, you know, systems with this kind of poison data on the network today. Yeah. I can think of one that I have just been curious about called Poison Fountain, which is essentially an open source system for sending gibberish into data, you know, a data crawler.

That's crawling across the internet. And so mm, yeah. And so I think, and honestly, this is a little bit of just a like there's a cultural backlash to some of the change that AI has brought. And I think also the way the economics have been working on some level. And I think that w we'll see this kind of like, you know, almost digital warfare a little bit in the in the cultural space of the internet over the next couple of years. Yeah.

Chris Cochran: it's called poison fountain.
I think so. I think it's gonna be a big divide. I'm hopeful that you know, because there were a lot of people that were against cars, you know, very early on, right? They're like, hey, you know, you're taking our horses' jobs, and you know, they tried to create these limits where you can only drive a car five miles per hour, right? Because it's so dangerous. But you know, technology has its way of finding its way up. That's just the nature of technology.

And so I would think that as much as you want to fight artificial intelligence, it's going to be really, really hard to do so. And so I would rather someone take that energy because there's still a lot that we need to consider from an AI perspective, whether we're talking about safety, we're talking about security. So instead of trying to stop a technology, let's use that energy to try to put it into a position where it's going to be the most beneficial.

For humanity, for e the economy, for everybody. I if we could do that, woof, I think we'd be all right.

Justin Beals:  I and this is the tale of two cities for AI. I am very excited about what it can do for good. And even in the trenches, you know, w we've been able to turn around feature sets that are really innovative with small nimble teams because we're coding alongside, you know, a machine with us now. And it would have taken us a lot longer and more cost to get that same feature out the door without these types of solutions working alongside us.

Chris Cochran:   Yeah.

Justin Beals: At the same time, you know, you can use it for sheer efficiency in economics, and there is some medium val v validity to that, or just to do not nice things in the world. The tech doesn't care, as you started out our interview with. The tech does not care at the end of the day. Yeah.

Chris Cochran:  It's all on how you use it. Yeah, when I when I think about AI, because I mean, I could almost hear the thoughts ruminating in the back of your head, because a lot of folks are concerned around AI taking jobs and stuff like that. But I I like to think of it, and I'm I'm a bit of an optimist, but I like to think of AI as like an Iron Man suit. Now imagine if you had

Justin Beals:  Yeah.

Chris Cochran: An organization and you're able to just do your capacity, like because everybody's human, everyone needs to take breaks. But now imagine if you gave that same company, everyone gets Iron Man suits, right? Imagine what you'd be able to get done. So instead of it thinking thinking of it as this completely separate entity that can just operate on its own, think about it as the ultimate tool that you can use to do the things that you've always wanted to do.

And if you could sort of wrap your mind around that concept, you could start to do some really incredible.

Justin Beals: Yeah, absolutely. Speaking of the future a little bit, what's what's next for you, SANS Institute, as you're rolling out this the AI maturity model work? How how do you see kind of the next twelve months as you're working on this? Yeah.

Chris Cochran: Yeah, I would say the next 12 months, we're doing a lot of different forms around the maturity model. We're helping some customers through the maturity model just to get number one, help folks through and actually help grow their program, but then also to receive additional feedback, right? Because a model is great when you write it, but it's really when you start to put it to use, you start to find, that, well, this didn't really work the way I intended it to, or maybe it'd be better if we add this. So

Justin Beals: Yeah.

Chris Cochran:  I want to constantly continue to iterate it and continue to mold it, make it more usable, and just see if we can get, you know, folks adopting it and just see how useful we can start to raise all the boats in the ocean.

Justin Beals:  Sounds like a real product release, Chris. what a joy to get to chat with you today. Thank you for sharing your experience and expertise with myself and our audience, Chris. And thank you for the work on helping us find ways to adopt AI in a secure manner and measure how how that has worked.

Chris Cochran: Absolutely, Justin. I appreciate it. Thanks for the opportunity to come speak to you and your folks, and yeah, happy to come on any time.

Justin Beals: Well, I am going to take you up on that offer. Hope everyone has a great day today, and we'll talk to you again in two weeks.