NIST's Victoria Yan Pillitteri: "Compliance Won't Save You". Inside NIST 800-171

Hello everyone and welcome to SecureTalk. I'm your host, Justin Beals.

I have spent the better part of eight years in security compliance work, and in that time I have said some things out loud that put me at odds with a lot of the consultants I talk to. I tell clients that a framework requirement is guidance. It is a rubric you get assessed against, and it is not your control. You have to write your own control; because the process that actually keeps you secure is yours and no author could have written it for you. I have gone further than that. I have argued that most requirements are really written for the assessor and not for the person who has to implement them. And I have come to believe that the people writing these standards are quietly innovating on the format itself, with tools like organization-defined parameters or quantitative scoring and that it is worth asking where are we going?

Those are bold claims to make in front of a room full of compliance professionals. So I will be honest with you. When I got the chance to bring them to the person who helped write NIST 800-171, I was terrified. Victoria Yan Pillitteri manages the Security Engineering and Risk Management Group at NIST and leads the Risk Management Framework team. She has spent years inside the 800 family and the Joint Task Force that develops it. If my theories were wrong, this was the conversation where I would find out.

So we tested them. I asked her what tips the balance toward one requirement over another when you build a catalog like 800-53, and how she thinks an operator should read a control statement. Some of what I believed held up better than I expected. When I said that reciting a control back to yourself does not really say anything, she agreed and took it further. Other parts she reframed in a way I had not considered, and that reframing is why this episode is worth your time.

This matters right now. On November 10th, 2025, new security requirements became mandatory in Department of Defense contract solicitations, and NIST 800-171 is the backbone of that program. We have customers running gap assessments against both Rev2 and Rev3 today. We also reach the frontier, from continuous monitoring and FedRAMP's 20X program to the AI control overlays Vicky's team is building on top of 800-53 for predictive, generative, and agentic systems, with an initial public draft she hopes to publish this fall.

If you have ever opened one of these documents and felt overwhelmed, unsure or just frustrated this is a chance to meet the inventors of security compliance requirements.

Victoria Yan Pillitteri is a supervisory computer scientist in the Computer Security Division at the National Institute of Standards and Technology (NIST). Vicky is the Manager of the Security Engineering and Risk Management Group and leads the Risk Management Framework team/Federal Information Security Modernization Act (FISMA) Implementation Project. In that role, she develops the suite of risk management guidance used for managing cybersecurity risk in the federal government and coordinates the associated stakeholder outreach and public-private sector collaboration efforts. Vicky leads the Joint Task Force working group, a partnership with the Department of Defense, the Intelligence Community, and Civilian Agencies to develop a unified security framework to protect the U.S. Government from cyber-attacks. She is also the co-chair of the Federal Cybersecurity and Privacy Professionals Forum hosted by NIST.

Ms. Pillitteri holds a B.S. in Electrical Engineering from the University of Maryland and an M.S. in Computer Science with a concentration in Information Assurance from The George Washington University. Ms. Pillitteri is a Certified Information Systems Security Professional (CISSP).

Join me today on SecureTalk for this important conversation with Victoria Yan Pillitteri.


—

Justin Beals: Vicky, thanks so much for joining us today on Secure Talk. I'm very grateful to get to chat with you.

Victoria Yan Pillitteri: Justin, thanks for having me here. Let's have fun.

Justin Beals: I know exactly. And I realize that our topic might feel a little dry to many folks, especially because we're gonna dig deeply into frameworks and standards and compliance stuff, which you are an amazing expert in. And it's funny, I'm a fan of your work. You know, I've read the 800 Family; I've helped a lot of my colleagues and customers get through compliance outcomes that were either influenced by or informed by the National Institute of Science and Technology and some of their recommendations.

I'm very curious, as one of the people who actually writes these standards, I have to imagine that you're constantly deciding about kind of what earns a place in a set of requirements and what gets left out. And when you're building a catalogue like 853, what tips the balance towards one requirement over another requirement? What gets included? 

Victoria Yan Pillitteri: Well, I always keep in mind at the end of the day, when I've pissed everyone off, that's when I know success has been achieved. So you're right, Justin. Standards, you know can be a little bit dry, but we're gonna keep this a little spicy. One of the roles of the National Institute of Standards and Technology is to really advance measurement science, promote US competitiveness. And we believe, because obviously I work at NIST, that something foundational to advance this American innovation is to be able to market it, to sell it. You need to be able to measure it. And that's the importance of standards. I think in the world of cybersecurity, something like 853- I'm sure we'll touch on 800-171- it really sets that Rosetta Stone of common cybersecurity and privacy outcomes. It's a catalogue. Not to, you know, speak specifically to a company, but I think about 853 as the Cheesecake Factory-style menu of all of the cybersecurity and privacy outcomes that you could really think about. So no matter who you are, what you're craving, or whoever in your party is craving something different, there is something at the Cheesecake Factory for you in most cases. I like to think about 853 in that same lens. What are those cybersecurity outcomes, the large perm set of permutations that people can choose from to help manage their risk?

I think we get a lot of feedback saying, 853 is so long, it's too much. Well, if you're trying to do it soup to nuts, of course it's too much. You're gonna ultimately find controls and outcomes that directly contradict each other. And that's by design, right? You need to have different options. You pick from an organization, from a system standpoint, what you need to manage your unique risk.

You and I have very dis different risk profiles. We have different; we sit in different time zones, we have different needs. It's not fair to say that, you know, we're gonna both look good in the same size shirt. Maybe, maybe not. So that's how I like to think about 853: it's that Rosetta Stone of cybersecurity privacy outcomes that anyone can use, anyone can choose from to help understand and manage those unique risks.

Justin Beals: I love where you started with measurement and at the risk of getting a little philosophical. That is such an intriguing topic to me. I think when you know measurement can be thought of as very finite or very something you can touch, like a yardstick, something that has a physical quality. But then there's a lot of times I've worked in measurement situations, especially in education, where it's very esoteric. You know, we're designing measurement around a cognitive ability that has no physical measurement. Like we're building a statistical analysis of where people place, and it's all relative. And then starting to think about cybersecurity or security in general, or risk as a measurement has got to be one of those squishier areas of measurement. It makes it challenging, doesn't it?

Victoria Yan Pillitteri: So cybersecurity measurement, I think there's some very tangible, specific things that you can almost touch and feel, right? You can measure the number of vulnerabilities. You can identify specific threats. However, it's how you quantify it to really make up what is my risk tolerance. There's no number, right? You can't be a seven. You're a seven point two. there's no common scale that works for everyone. So while we have discrete areas where we can measure the strength, like the strength of an encryption algorithm, like the number of threats, the severity of threats, there's still some inherent subjectivity, inherent context that is so essential to really making it meaningful to you. I'm gonna go back to that overall risk equation, right? I won't deny or confirm, but I might like to speed a little bit. How much over the speed limit am I willing to go? Is it five miles? Is it seven miles? It depends if there's a camera. It might be different on how much you're willing to speed over the speed limit and, you know, what time of the month it is.

Justin Beals: Yeah, absolutely. And I think risk specifically, NIST recently, and we just worked to implement it in our own software, released the risk management framework. And it's that word framework that really stands out to me when you talk about it, because these are not rules. This is like an operating system for understanding risk, more than a dictation of how to measure it exactly.

Victoria Yan Pillitteri: And that's exactly the point. I think in a lot of cases, organizations, well-meaning organizations and implementers, take NIST as a this is the checklist to end all checklist. This is my silver bullet list. If I do these seven, eight, nine, ten things, I'm good to go. And the whole point of calling many of these guidelines frameworks is it provides that foundation, that structure, that repeatable methodology.

That you know, the steps and the approach is kind of set, but there's a lot of inherent flexibility to take into consideration that unique context because one little bit of context can change everything. We all know in the world of cybersecurity threats, we see the news every day there is a new vulnerability exposed. Something is changing constantly, and that's the only thing we can promise you in cybersecurity, which  is part of the reason why NIST designs our guidelines as such, as frameworks rather than checklists. A top 10 list might be, you know, relevant only for a couple days, maybe a couple hours, maybe a couple minutes. But if we give you that approach, that methodology to understand what's going on, what's important to you, how you should protect it, we give you the process that you can continuously evolve to meet the changing needs of the kind of the larger ecosystem that we're all part of.

Justin Beals: Yeah. I talk to a lot of folks that especially customers that are consuming some of these frameworks. And it's it's initially a real challenge for them. I think a lot of them open up thinking, okay, I'm gonna get some step-by-step instructions here. I've got a, you know, one plus one equals two, multiple-choice test I've got to answer for my company, and they get really surprised to find, like the control statement or rubric. And I'm curious, you know, your recommendation. Like when an operator hits, like, a control statement out of the 800 family, how do you think they should read it and interpret it for their own organization or systems?

Victoria Yan Pillitteri:
I mean, I will be, let's be very frank. 800 853, 80171, these security and privacy controls, these security requirements for protecting controlled unclassified information, they can get weedy and technical really fast. I think it's an inherent challenge for NIST to provide that technical level of detail to really help implementers achieve that outcome to manage risks.

But also make it approachable and usable because not everyone is gonna be a cybersecurity expert. So it's a really fine line. And I think often we end up skewing more technical. But that's why we have other guidelines, other resources in our portfolio of frameworks, like the cybersecurity framework, the AI risk management framework, that does speak to a smart individual, but maybe not a cyber expert. How do we break some of these concepts down into something that is digestible and understandable? So at least you have the right direction to go. And then when you work with the right technical folks, they can dive deeper and deeper.

Justin Beals:
So, Vicky, I'm gonna ask you if I'm committing a sin in a statement I often make to folks. You can tell me how wrong I am, and I'm willing to learn and grow. I oftentimes talk with folks, especially about NIST 800-171. That's where we see a lot of activity lately. And I say, well, sure, you could take the control statement that came out of the framework, and you could just write that and say that's what you're going to do. But that's rarely the fullest definition of what you're going to do to meet that control statement. And I tell them all the time, like, you should write your own control. You can be inspired by what NIST wrote in their control framework, but you really need to provide some specificity about how you're gonna do it because they couldn't figure that out for you. It's unique to you.

Victoria Yan Pillitteri: I mean, I think you're pretty much spot on. By just reiterating the control, you're not really saying anything, right? The whole point of implementing this control or this security requirement is to say, what risk am I trying to manage? What am I trying to protect? What is my ultimate outcome? I will say using some of the terminology in the control itself is helpful if you're trying to get assessed and you know get a score to be part of the DOW ecosystem. That is obviously very important. But I think we really have to think, take a step back and say, what was the whole point of this? This is not intended to be a paperwork exercise. It's intended to to manage a very specific risk. So if we look at a control in, say, the access control family, what are we trying to accomplish here by limiting access to only the people that should have it? What are we trying to protect here? And how do I implement that for my organization? Some of the feedback that we get is NIST is both too specific and too prescriptive, and also not prescriptive enough. Like, don't tell me what to do, but tell me exactly what to do. It's very conflicting. It's like they've been speaking to my mother. Sorry, Mom. 

So one of the things that you know NIST is not able to do is tell every organization exactly what to do, what product to use, what solution to use, how to implement it, because every organization's risk risk culture, your risk tolerance, is going to be slightly different. The type of information you're trying to protect, your system architecture, your policies and procedures internally, you know, your staff is all gonna be really different. So these are small variables that you kind of have to tweak and know on your own. Otherwise we just start a really multi billion dollar consulting business, right?

Justin Beals:  Yeah, yeah. Well, and it was interesting to me coming into this particular market eight years ago as I was doing my research to realize that, you know, the amount of money globally that's spent on compliance and that really ninety per cent of it was spent on consulting, because people needed to define what it is that they were doing or interpret requirements. I like you I like your language of framework requirements. It's one I often use. Cause I'll tell folks these are the requirements coming from the framework. Think of them as a rubric that you'll be assessed against. You need to write your own control. You can use the language, but it's your process, procedure or habit that is inevitably going to keep you secure, and you're going to get tested against. Yeah.

Victoria Yan Pillitteri:  Absolutely. So here's another kind of, you know, motherhood and apple pie kind of approach to it. If you have strong risk management, if you're doing cybersecurity and privacy risk management right, you are inherently going to meet whatever compliance requirements, compliance frameworks you need to. It's no one is going you're people that you work for or people that you're providing service to, they're not gonna be any less mad if you say, Well, I was compliant with this and this and this. It's what were you actually doing to protect my services, my information? What were you doing that was above and beyond the bare minimum? Compliance is always the floor because compliance, the requirements can't change that drastically. Otherwise, it's not fair to measure.

It's actually implementing good risk management that makes you agile and adaptable to changes in technology, to changes in the threat landscape, to making sure that you actually are doing good cybersecurity.

Justin Beals: Yeah. I have another question about how the sausage is made, so to speak

Victoria Yan Pillitteri: You can't have all of my secrets. Come on, Justin. That's why my hair is so big. It's full of secrets.

Justin Beals: I know, Vicky, thanks. Yeah. Well, we'll hold back s yeah. I have to imagine that there are a lot of debates that go into putting together, you know, some of these families like Estate Hundred 171. What's a requirement that you can think of that looks simple on the page when we read it, but as you guys were working on it, carried a hard trade-off or two in the definition?

Victoria Yan Pillitteri: Man, that's hard. I want to say almost every single one. I'll tell you a little bit about how we develop the publications and how we engage with the public, academia, industry, anyone that's interested. Because ultimately, at the end of the day, a standard, a framework, or guideline is the best we have at that point in time, right? It's never gonna be infallible. The only constant is change. All of that is always true. 

And while I've had the honor of working with some fantastic, brilliant people like Ron Ross from NIST before he retired, we have a there's a lot of smart people in the industry, in academia, in other agencies and private sector here in the US and abroad. And I think one thing that we have to be very aware of is we're all interconnected in this day and age. Good cybersecurity is good for all of us. And so it's really, you know, not to be cliche, a team sport.

So part of this part of NIST development process on cybersecurity and privacy standards and guidelines is leveraging this common and shared expertise. So we work internally; we reach out to stakeholders, often we'll do a pre-call for comments to get initial feedback on what's good, bad, ugly about what's out there. We take that feedback, we do our own research, we work with other agencies to get feedback as well, we issue a draft, right?

During this draft period, we solicit, we actively solicit comments from anyone. And something that I love about this is the comment from a small GRC company such as yours, Justin, is weighed the same as a comment from potentially the Department of War. Just that tiny federal agency, right? Because we really want to look at the validity and the basis of that technical comment.

Obviously, people provide us comments and feedback that is out of scope for what NIST can and should do. So sometimes, if you see your comment not go somewhere, it might be because, well, we can't pitch your company for you in our standards and guidelines. That's not fair, right? So we really weigh each and every single comment as we adjudicate them to improve the document. How do we improve this for the scope for the the threat that we know of today? Often we'll issue a second public draft to make sure people have that chance to say, all right, I see what's changed. I see what's good, what's bad, I see I have that one final chance before NIST issues it in final. And then we eventually issue it in final. But final doesn't mean it's the end of the road. Many of our publications are on an ongoing revision cycle.

Some of our more popular, widely used publications are no longer issued as documents, but they're issued as online data sets. So we now have the ability to really update them almost like software, you know, issue minor releases, issue patches. I think it is a delicate balance of being agile and able to respond and update and creating chaos by making too many updates. So it's, you know, again, when everyone's mad at me, I know I've done a good job.

Justin Beals: That's good, I think. You're in the midst of one of those revision changes right now between NIST 800-171 Rev2 and NIST 800-171 Rev3. Certainly, we have a number of customers that are running gap assessments against both frameworks for their current activities. And I was a little curious about that particular process. Let's start off with anything you might want to add about the differences between Rev3 and Rev2? What were you starting to take into account that made you want to even build a new version of that product?

Victoria Yan Pillitteri: Well, 800-171, the security requirements for protecting controlled unclassified information, is based on the 853 security and privacy controls. It's derived from a moderate impact baseline because there is a legal requirement to protect controlled unclassified information, which is a very specific type of federal information that has specific safeguarding requirements per law, policy, government-wide, blah, blah, blah, blah, blah. There are specific rules for protecting certain types of information with a confidentiality level of at least moderate or higher, because this is important government information that we don't just want floating around ultimately. And these information types have to be protected at that level, regardless of whether it's in the possession of a federal agency, whether it's in the possession of one of our partners in the private sector, academia, industry, what have you. Those requirements remain unchanged. 

So 171 is derived from 53. We had done an update to 853 in 2000, and we've done a couple of iterations since 171, the original security requirements were put out. So as part of a refresh cycle.

Remember, the only constant is change in this industry is we had to make updates to make it up to par. We did do some more than just updates. There was a lot of feedback that we've gathered from small and medium businesses, from partners in the defense industrial base about what's good, bad, ugly about the 800-171 security requirements as they were in the original iteration, Rev 1 and Rev 2, the short version of it: the requirements largely remained unchanged. They were originally designed to be very broad and outcome-focused, which sounds great on its face. However, the challenge there was when people were doing assessments, when it was either a self-assessment, a third-party assessor, or even the DIBCAP.

The defense, the DOW's assessors coming in to do an assessment- those very broad statements led to a lot of subjectivity in the assessment. When I ask you,” Hey Justin, are you wearing a shirt today?” You're gonna say, “Yeah, I got a shirt on. I'm doing a podcast”. That's hopefully the norm. At least on this one. So depending on what assessor you have.


They might ask, is your shirt red? Does your shirt have a collar? Are you wearing an undershirt? So the consistency of those assessments was hard because there was no way to really be specific enough. That was one challenge that we heard both from organizations being assessed, doing self-assessments, and assessors. the other one that we heard was interesting, and I wanna I don't to say confusing, but a little contradictory for what one would assume. A lot of smaller, under-resourced organizations wanted additional specificity because the broad outcome statements that were beneficial to mostly larger organizations that already had existing strong cybersecurity programs. Well, if I give you a set of requirements that are very broad, like
do good access control, make sure you have physical security. They're already doing some version of that because they have that budget dedicated to security already. So it's easy to say, I've already got that done and done. For a smaller organization that doesn't have a team of a hundred, hundreds of staff focused on cybersecurity, those broad statements are, how do I even start? Like, what is good account management look like? What do I have to provision? What's an account? What do I? 

So there was also an ask for additional specificity without being too prescriptive, you know, a little bit of that Goldilocks challenge. Those are just two of the examples that really drove, in addition to the updated requirements, of how NIST wanted to look at these requirements, how we wanted to frame them out for both better consistency across our user base and to help out organizations.

Since then, we've also released some additional resources to help people because we recognize that we often tend to write to that more technical subject matter expert audience. But not everyone is a subject matter expert. We all started somewhere, right? So we've been working with our small business cybersecurity team lead at NIST to develop a series of quick start guides. Actually, one just came out this morning on protecting the CUI security requirements as well as assessing the CUI security requirements. So really how do we the requirements don't if you're if you need to meet these for some compliance or contractual reason, the requirements don't stay the same. But how do we dive in? How do we get someone started into this very complex ecosystem in a more gentle, meaningful way where people can wrap their minds around these concepts? understand how to take baby steps to apply them before they go the full the full way and not have to pay a consultant to do it.

Justin Beals: Absolutely. 

Victoria Yan Pillitteri: Sorry consultants. You're still needed. You're still needed.

Justin Beals: Well, I mean, yeah, but I think most consultants want to be valuable in their engagement. And if they feel like they're doing meaningless work or it's not helpful to their customer, then it's a losing relationship. You, it's hard to invest in it long term. I,  one thing that
really hits home for me is this full circle of measurement, right? Like, if we think about these tools as a measurement tool, whether that's internal, like I want to self-measure how I'm doing against cybersecurity, against a framework, or a third-party assessment like a C3PAO or a CPA for SOC two, one of the critical aspects of measurement is validity is the repeatability of the measurement, right?

We tested this all the time in data science. Like, would it give the same response given a similar set of inputs? Or given a change in inputs, would we see a similar outcome? So it sounds to me like one of the aspects of Rev3 is to increase the validity of measurement by kind of helping the assessor build a more repeatable measurement device.

Victoria Yan Pillitteri: Make so one of the things I only half answered your last question. One of the major changes that we made between Rev2 and Rev3 was really say what we mean, be explicit in what the requirements were, kind of break down those broader statements into more discrete, specific but not prescriptive elements. Providing the opportunity for either the federal agency issuing the contract to provide, you know, more detailed requirements. We call them organization-defined parameters. 
What's really important for them to define in terms of, like a range of things: the types of crypto that are allowed, you know, how many times a year you do something- versus letting the organization decide some of them, because some of them are just good risk management practices that, you know, an organization trying to meet these contractual requirements, they should get some say in because they're the ones running the show. So we added that additional specificity to help that consistency and understanding what was expected and what should be kind of shown and demonstrated in the programs.

Justin Beals: Yeah. It feels like you have a double audience, e even for the organizational design parameters, which I thought was very intriguing. It's not only for the person implementing it, you know, what is your design parameter? But in some ways I can see the assessor interpreting this is an organizational flexible decision. Like my assessment needs to provide bandwidth for what they want to decide to do.

Victoria Yan Pillitteri: I mean, a great example of that is, you know, I think in Rev, the original Rev Two, Rev1, we said, you know, conduct ongoing or regular assessments of X, Y, and Z. What does that mean? What does ongoing or regular? How do I, you know, is it every week? Is it every two weeks? Is it every month? And I think it became very subjective, and it's a point of tension because they're there to kind of check your homework.

So by having those ODPs, those organization- defined parameters, either the government can say, You need to do this quarterly or the organization can say, Well, we do this monthly, so you know, check that you say that I am doing what I am doing or what is expected.

Justin Beals: Yeah, absolutely. And I juxtapose sometimes the the the work that NIST does, the requirements that y'all publish, some of the rigor, and other types of frameworks. There's a huge continuum. You know, I think about the trust services criteria for SOC2 assessments, and those are very general. I oftentimes tell people that you're probably doing 80% of this already. We just need to catalogue what's going on here.

Whereas a slightly more rigorous measurement device like 800 171, especially 853, or that moderate level of handing CUI data, that is going to be a more rigorous assessment and slightly more specific. And the expectation is going to be harder to get around when you do come into an assessment.

Victoria Yan Pillitteri:Yes and no, right? Because the requirements themselves, the control statements, are more detailed, it's very clear what the expectation is. And there's no right or wrong answer on what the best framework is. I guess I'm gonna say the NIST frameworks are the best because I know who pays me. But each one of these frameworks and standards, they were designed for a very specific purpose. And we have to take a step back. Often when we get into this compliance regime and mentality. It's, I gotta meet these requirements. We have to always take a step back: why are we doing these requirements? And ultimately, we're all move trying to move to the same goal, right? We want to protect our systems, we want to protect our information, we want to provide the availability of these services that are kind of essential to all of us. And each of these frameworks tries to get to that in a slightly different way with a slightly different focus. So it's not a wrong or right. It's what is the right one for the situation you're in or what you're being told to do? Right.

Justin Beals: Yeah, yeah. Well, and I think to your point, and I  evangelize this quite a bit, we all operate in a shared garden, whether it's our country, our community, our business marketplace. And if people don't trust that at the end of the day, and we're not operating at a together at a level of rigor, I mean, fair to compete inside of your point, the floor, not the ceiling of good cybersecurity practices- then we're going to make it hard to be successful for any of us. It will be a challenge if we're not operating in fertile ground that people trust to operate with us.

Victoria Yan Pillitteri:  And trust is another one of those things that's really, really hard to measure, right?

Justin Beals: 
Yeah. Hard, but moving in the right direction. Cause I remember the days when it was the Sig Lite questionnaire and that and/or someone would say, Hey Justin, can I trust your software? And I'd be like, well, we're hosted on AWS. And they would say, okay, that sounds good. I I now trust you. I know. And I'm like, you should ask other questions.



Victoria Yan Pillitteri: But again, you know, how do we quantify those expectations, those requirements? How do we understand, all right, Justin, you're building your solution on something that on using services and platforms that are secure because they do X, Y, and Z. So I can assume that if you're using this infrastructure and you're doing these good practices, that overall your service and solution is good. This is a choice that everyone in the ecosystem makes every single day.

It doesn't matter if you're a billion-dollar company or you're a mom-and-pop shop. Just think: when you're doing your card payments, do you trust your vendor? Do you trust their infrastructure? Like, you just assume you're gonna get paid, right? So this, I know it sounds so esoteric and so in the weeds of, you know, defense industrial base, but everyone relies on
good cybersecurity, these trust networks, because this runs our economy, this runs our country ultimately.

Justin Beals: Yeah. And I think the other thing that I'm distinctly aware of: family members that work or are engaged in mission for national security, they operate in public-private partnerships with business. We are a part of the security of the nation when you contract to the agencies that are trying to deliver on those missions. Yeah.

Victoria Yan Pillitteri: 100%. I mean, I don't even work in the national security community. I'm within commerce, but we rely on the private sector. We leverage commercial off-the-shelf or slightly augmented government off-the-shelf solutions that are provided by private sector. We leverage that infrastructure where the government has turned to a cloud-first approach of how can we raise all boats?

Good cybersecurity is good cybersecurity. Private sector has a lot to bring, but government also has a lot to bring in terms of rigor and expectations. Ultimately, I like to think of this as, you know, people often say that, you know, government requirements are over the top on what private sector does, and private sector is able to innovate faster. And, you know, often they are right, but that doesn't mean government is slow or bad.

 I think it's just a different risk profile. Ultimately, if a private sector company builds fast and hard and aggressive and they fail, they fail. The government provides essential services to our citizens. Failure is truly not an option. So we don't just get to go out of business, which is why our risk profile and our risk tolerance might look a little bit different than, say, you know, a large software company in private sector.

Justin Beals: Yeah, absolutely. Okay, I want to talk about a couple of future things, Vicky. We're gonna point.

Victoria Yan Pillitteri: No secrets, Justin. No secrets.

Justin Beals: Let's start with the ATO authorized to operate and continue continuous authorization or continuous to operate monitoring. I was just recently looking through some of the FedRAMP 20X program, obviously based on a set of requirements from NIST.
And starting to consider how the OSCAL integration works, or how they want to connect with a trust center and have up-to-date information on what we're doing. Tell me how you think about moving from an ATO where it's a point in time, perhaps in a historical review of practices, gives you a license to practice that or deliver that software over a future period of time, and something more in the moment, or more iterative, continuous monitoring.

Victoria Yan Pillitteri:  So interestingly enough, I think that's actually a common misperception of ATOs. Now, obviously, this concept has existed for twenty-plus years and has been kind of a cornerstone of federal cybersecurity programs. You get your authorization to update, operate, and you're good to go for some set period of time. There's actually been a push, a long-term push over the last 15 or so years to move from that point in time, I'm good.

To an ongoing, near real-time continuous monitoring approach to authorization. Because again, it goes back to one of the foundational principles we started with: today is the only constant that we can expect in the cybersecurity world, in the IT world, is change. The threat constantly changes, minute by minute, second by second. And to think that something you did two to three years ago for your security that you got double-checked is still good today, in many cases, could be a very big fallacy. 
So NIST has actually been pushing for ongoing authorization, continuous monitoring of your ATO. Now that doesn't mean you have to check every single control every single minute. Most likely your policies and procedures are going to be more stable. They're not going to change, or they're not going to change drastically day to day. They probably won't even change that much year to year unless something, you know, crazy happens.

What do we really care about that provides those metrics on our security and privacy posture right here and right now? Those are the things that we need to be continuously monitoring, continuously assessing and making decisions whether this continues to go, the system continues to live and operate, or we need to pull it and do something different. So it's really not, it's not an all-or-nothing approach. And I think people look at it as black and white. It's really, how do we get those shades of gray?

To slowly move to that kind of ongoing live sock, I respond with all the monitors and flashing lights. So it's not, you know, it's not the sci-fi Star Trek, but it's also not like a binder full of controls, and you have this guy, you know, reading this checklist. It's that nice middle ground. We've some seen some fantastic work from OSCAL making this machine-readable schema, this consistent way to express the requirements, the assessment procedures to help with some of that, you know, the paperwork part. 

But how do we implement it? How do we take the data from our systems and translate that into a real-time monitoring, risk, ongoing risk assessment process that we understand good, bad, ugly, red, yellow, green? We need to make a decision, or we can continue to go. And this is something that NIST has been pushing for a really long time.

So I'm really excited about some of the work that FedRAMP is doing through their 20x program, whether it be offering the KSIs that show kind of that ought requiring vendors to provide that continuous monitoring information on certain controls, right? It's not all of them; it's that's not realistic, but a common baseline of things that are overall pretty important and a good good good starting point.

Now I think what we're really providing them in OSCAL format so they're machine-readable so they can be ingested into different GRC tools. These are all great things. And this is a fantastic starting point. But let's not let this starting point be the finish line because that's sufficient. But is it enough? Is it truly enough? You're getting that baseline of, all right, I know that what we're doing this is our hygiene. This is what we should be doing. But what are we doing to truly manage the unique risks to my organization? And that's when each agency is going to be asking for possibly and probably different things because each agency has a different mission. They have different systems. They have different information. There's a lot of customization that a one-size-fits-all solution might be good to get you through the door. It's a great starting point. It's actually a fantastic starting point. But I would only call it a starting point. That can't be the finish line because it's sufficient, but is it enough?

Justin Beals:  Yeah. It's been an interesting balance on the implementation side because I think there's a tendency to feel like a logging and monitoring system, which is a very specific type of like, a secure operations center or even a firewall, and you want to monitor that as opposed to something that is a little less granular.

Where we're like, okay, over the last month, was the control operated in an effective manner as you designed it? And what evidence did we have to validate that? And I think this is one of the nuances of the discussion: you're not millisecond; you're probably day, week, month, quarter type of assessment.

Victoria Yan Pillitteri: So my another thing that I think I like probably try to really emphasize is let's be reasonable here, right? Let's not, no, no, really, really. Like let's not have good enough be the let's not have perfect, be the en the enemy of good enough. Because there is no such thing as perfect. You are always going to have to accept some inherent risk. whether even if you haven't left your house yet today, you have inherently accepted some risk in order just to put one foot in front of the other. So let's not let perfection be the enemy of good enough and understanding what that risk tolerance is, what is acceptable. Because we will- we do have to continuously evolve. The threat landscape, the technology and policy landscape, continues to change. And we need to be both adaptable to change with it. But let's not forget the basics, right? The basics are all still good. All these good cybersecurity practices of good access control, good personnel security, good configuration management, even in the age of AI, is all still incredibly relevant and provide that strong foundation to build off of. Now we'll have to be faster, we'll have to be smarter, and we'll have to be able to do more to process all this data. But those foundational things still remain unchanged.

Justin Beals: And of course you mentioned the elephant in the room.

Victoria Yan Pillitteri: I don't believe we've gotten this far without saying those two dirty letters.

Justin Beals: Vicky. I know. So let's crack it open. And of course, I understand this is evolving in just the media to today alone. The discussion of AI and safety and risk is dominating at least my algorithms and feeds. NIST recently released the AI risk management framework. Thank you, by the way. Very grateful for this kind of information. But I wanted to ask certainly about securing AI, and you can answer from your perspective as a computer scientist or how you're thinking about it broadly as an organization. I just understand this is evolving quickly. Yeah.

Victoria Yan Pillitteri:  Well, we recognize that this is evolving quickly, and there is a need for better standards, guidelines, and just something. Right now, we are all collectively building the plane as we're flying it. We are seeing incredible advances in what AI can do. But there's as a security person, I'm always like, glass half full, empty. So what are the risks here? What I think in many ways there's there's an opportunity. to leverage what we have. Now it's not to say that we can just take what we have and apply it to AI systems and say, dun, dun, dun. But it does, in a lot of ways, AI systems are just very fancy software, advanced software, non-deterministic; they have different functions, slightly different functionality. But at the end of the day, they are software. So that means we can leverage the foundation of resources that NIST offers iIn the cybersecurity and privacy space for traditional systems.
 But how do we augment it to the unique risk profile of such AI systems? Some work that we specifically are working on since we talked about 853, and they're, you know, one of my favorite things in the world, and hopefully everyone that's listening today as well. We're developing a series of control overlays, tailoring and augmenting the 853 controls, and in some cases supplementing by creating new controls to help manage the unique cybersecurity and privacy risks, two different types of AI systems. This work is currently in process. We are somewhere at the nearing publication stage on initial public drafts for our development methodology, because we want to teach the community how to fish and the method behind our madness. And our first overlay on securing predictive AI systems.

We have work, ongoing work on securing generative AI systems, agentic AI systems, and we'll finally wrap up this series with a publication on best practices for AI software developers. But kind of a starting point that both highlights the unique risks and threats to each type of system because of the way that they are architected, designed, etcetera. There are different unique threats that traditional enterprise software and IT systems don't necessarily face. How do we take the existing 853 controls? How do we supplement them? How do we provide specific implementation guidance? Whether it be different organization- defined parameters, whether it be tailoring guidance, whether it be considering how you apply a specific control throughout the AI system lifecycle.

Because at the end of the day, many of us are just going to be organizations that consume AI systems. We're not going to be developing. We may or may not even be training the AI systems ourselves; we're relying on those in that supply chain. So, how do we understand what kind of controls need to happen at different points at that AI system lifecycle? Because it doesn't just show up one day. There was a whole development and training process before we got to buy it and integrate it into our organization. 

So understanding what good practices should happen at different parts of the system life cycle. So we can ask our suppliers, our vendors, what are you guys doing about this when you're developing? What are you doing about this when you're training the models? What are you doing about this during the maintenance phase? So understanding those unique risks, those threats, mapping them to controls, extending controls, creating new controls that aren't necessarily covered in 853 and maybe shouldn't be in the generic catalogue, but very specific to that type of system.

 So we're really excited about this work. It kind of ties together a lot of the work across AI and cybersecurity in the NIST portfolio and provides something that I hope will be really good, kind of a good, strong starting point for any kind of implementer, whether you're acquiring, building, or, you know, just using the different types of AI systems, just for some food for thought.

Obviously we already said this, but you know, AI is evolving at a pace that is hard, it's hard to keep pace with, which is why we really want to also lean into issuing that methodology. We give you, as I would say, even if things continue to change, some of these foundational practices recommendations from NIST are still solid. But how do we take that methodology and continue to augment? How do you continue to augment? Because even if we gave you a list of 20 controls, you would still have to tailor them. You would still have to customize them to your own organization, your own risk profile, your own the your own specific threats. So we're really excited about this work. We're really hoping to get this out for public draft in the very near future, maybe fall of 2026. We plan to issue subsequent volumes sequentially as they're done being developed and cleared. And we'll leave the entire series in draft because every time we develop more overlays, we learn more and we find opportunities for refinement because, you know that's that's how you know you're alive. And hopefully we'll finalize this entire series sometime in 2027, just as that resource. And then we'll decide is there opportunity for more work in this space? But we're really excited about this. I really think it fits; it fills an existing gap in the kind of guideline space for managing cybersecurity and privacy risk to support trustworthy AI systems.

Justin Beals: Well, some things are true about AI, and I'll use the old word data science, still to this day, after 20 years of myself building everything from Bayesian models to random forest to the type of tensor models or neural net models that we do today. You know, and some of that is foundational and and you can practice good habits, safe habits, secure habits in working in that area. And then certainly some of it is new, you know, the new harnesses, the new way of getting models to work with each other. All of that is where the emerging, I think, risks s are starting to be seen. Yeah.

Victoria Yan Pillitteri: Just providing that it's a tool in your tool belt to understand where you are, what's acceptable to you, and trying to help prioritize what do we want to secure and how should we go about getting to these good outcomes? Because good access control is still always going to be good access control. Whether you're an agent, whether you're just you, you know, the right person or process acting on behalf of a person should be only have the right access to what they're supposed to and nothing more.

Justin Beals:  Absolutely. Vicki, you probably don't get this enough, or if you don't if you do, then I'm very glad. I am a huge fan of your work and standards. We could all discuss whether we like parts of a framework or not, or the assessment methodology that was designed around it. But I am terrified of the idea of operating without a framework, because then I just have to make so many decisions. It's so inefficient on my own. And so really we key as businesses, I key off of the recommendations that come from your group and your community. And it makes my life a million times easier. So thank you. Yes.

Victoria Yan Pillitteri: Well, thank you for using it, and you know, thank you for participating in this ecosystem because real, like it's not to be super cheesy, but we are all interconnected, whether you like it or not. So good cybersecurity for you is good cybersecurity for me too.

Justin Beals: Vicky, thank you for joining us today on Secure Talk. It's been a treat.

Victoria Yan Pillitteri: 
Thanks for having me. Hope to see you again soon.