Authors - Micah Spieler, Chief Product Officer, Strike Graph, Josh Bullers, Head of Engineering & AI, Strike Graph, Mustafa Duzok, Sr. Solutions Architect, AWS, Preethi Kumaresan, Sr. GenAI Specialist Architect, AWS

Overview

This guide walks you through how Strike Graph 's engineering team deployed fine-tuned small language models (SLMs) on AWS Bedrock using Meta's Llama 3.2 3B and Alibaba's Qwen3 4B architectures that outperform commercial large language models on compliance-specific tasks.

The approach: decompose complex compliance workflows into discrete tasks, build expert-curated training data for each, apply supervised fine-tuning (SFT) followed by selective GRPO reinforcement learning, and deploy as custom model imports on Amazon Bedrock. 

By the end of this guide, you will understand how fine-tuning a small language model for Strike Graph's domain-specific tasks outperformed off-the-shelf models across various performance metrics. The solution is deployed on Amazon Bedrock and validated against a real-world compliance case study.  

Technology Stack 

• AWS account with access to Amazon Bedrock and Amazon SageMaker 

• Python 3.10+ environment with PyTorch and the Hugging Face transformers, trl, and datasets libraries 

• Meta Llama 3.2 3B and/or Alibaba Qwen3 4B weights (via Hugging Face Hub with valid license acceptance) 

• Domain expertise to curate and validate training data — this step cannot be automated or skipped 

• An evaluation dataset with known-good ground truth answers for your target task

The Problem & Key Challenges

Strike Graph's platform automates the most labor-intensive parts of compliance: mapping security controls to framework criteria, testing evidence against requirements, and surfacing gaps before auditors do. These workflows are candidates for AI automation because they involve high-volume pattern matching across large bodies of structured documentation.

General-purpose LLMs achieve moderate performance on compliance tasks — but in this domain, moderate is insufficient. In compliance, there is no partial credit: a missed control mapping means a gap in coverage that may not surface until an audit; a wrong evidence match means risk goes unmitigated; and a model that's right only 50% of the time still requires a compliance professional to review every output, which means the automation isn't saving much work at all.

Commercial APIs present a second problem as well. Every request transmits customer data to a third party. For compliance teams, that data includes sensitive security posture details, control gaps, and audit evidence. For a compliance platform whose core mission is reducing customer risk, transmitting sensitive data to third-party APIs is fundamentally incompatible with that objective.

In Strike Graph's benchmark against leading commercial LLMs, the commercial model’s recall on compliance query generation ranged from 60.6% (NIST 800-171) to 73.6% (PCI DSS) — a 13-point swing depending on the framework. On complex mapping tasks requiring four or more controls to connect to a single criterion, recall fell below 50%. At that level of accuracy, the model misses one in two critical mappings.

Running a full framework mapping exercise through a commercial API requires rich context: full framework hierarchies, control descriptions, and traversal paths. That context easily pushes the cost to hundreds of dollars per attempt, and at that price point and accuracy level, the feature is not viable for production deployment

Training models on customer data is a data privacy risk. Strike Graph is able to mitigate this with an expert-curated dataset and produce results that general-purpose models can't infer from broad pre-training. Smaller models in the 3-4B parameter range cost a fraction of frontier models' API calls to run, and because they're deployed inside Strike Graph's own environment, no customer data ever leaves the firewall.

Key Insight: Why Fine-Tuning Is Needed: Compliance reasoning is domain-specific. It requires precise understanding of framework language, control hierarchies, and auditor logic. That knowledge doesn't emerge from broad pretraining — it has to be taught.

Why Strike Graph Fine-Tuned Instead of Prompting

Strike Graph is an AI-native GRC (Governance Risk and Compliance) platform that helps organizations manage compliance programs across frameworks like SOC 2, CMMC, HIPAA, ISO 27001, and PCI DSS. The core workflows cover mapping controls to framework criteria, testing evidence against requirements, and surfacing gaps before auditors do. These tasks are high-volume, high-specificity, and high-consequence. Missed mappings may remain undetected until the audit, at which point remediation options are limited.

General-purpose LLMs can handle these tasks to a degree, but compliance reasoning is narrow and exacting. It operates on framework language, control hierarchies, and auditor logic, none of which show up consistently in general pretraining data. That kind of domain-specific knowledge doesn't emerge reliably from broad general context. In Strike Graph's testing, commercial models showed meaningful accuracy variance across frameworks and struggled most on complex mappings where multiple controls needed to connect to a single criterion.

Strike Graph also found an economics problem. Compliance mapping isn't a one-time exercise. Frameworks get updated, control environments change, and organizations add new products or business units that require fresh coverage. A feature that costs hundreds of dollars per API run can be justified as a setup cost. It can't be justified as an ongoing operational cost. At platform scale, serving hundreds of customers through continuous monitoring workflows, commercial API pricing becomes economically unsustainable

Strike Graph evaluated two paths: prompt engineering to improve performance from larger models, or fine-tuning smaller models on the specific tasks that mattered. The case of fine tuning showed following advantages:

• Accuracy: A model trained on expert-curated compliance data learns the domain reasoning that general pretraining doesn't capture reliably.

• Cost: Models with 3-4B parameters cost a fraction of 70B+ commercial API calls to run, and training them only costs a few hundred dollars total.

• Data control: Fine-tuning on Strike Graph's own expert-generated data meant no customer data ever leaves the environment, which is a non-negotiable for a compliance platform.

The outcome is not a cost-reduced variant of the same approach, but a fundamentally different class of model: purpose-built, task-specific, and deployable on Amazon Bedrock within an encrypted environment.

How To: Step-by-Step Builder Guide

Step 1: Choosing Your Base Model

Model selection at the 3–4B parameter range balances task-specific fine-tuning headroom against deployment cost. Two architectures performed well for Strike Graph's use cases:

Prereq: Both models are available via Hugging Face Hub. Accept the relevant license agreements before downloading weights.

Step 2: Building Your Training Data Pipeline

This is the step that determines how well fine-tuning works. Model architecture and training technique matter less than data quality.

2a. Curate Ground-Truth Examples

In our use case: A security control is an organizational or technical safeguard — for example, a policy requiring that terminated employee accounts are disabled within 24 hours of offboarding. A framework criterion is the requirement from a compliance standard that the control is meant to satisfy — for example, NIST 800-171 criterion 3.1.1 requires that system access be limited to authorized users. Mapping a control to a criterion means establishing that the control, if properly implemented, satisfies that requirement. This is the judgment call that currently requires a trained compliance professional to make.

For compliance, ground-truth examples cover three task types:

• Given a security control description, what are the correct framework criteria it maps to? (control mapping)

• Given a control and a target framework, what search queries surface the most relevant criteria? (query generation)

• Given an evidence item, what other evidence items in the environment are semantically related? (evidence search)

Strike Graph's compliance team worked through NIST 800-171, HIPAA, ISO 27001, PCI DSS, and GDPR to produce this ground-truth. This step requires direct domain expertise and cannot be automated or abbreviated.

Here is an example of what a labeled training record looks like for the control mapping task:

# Example ground-truth mapping record.
{
"control": {
"id": "AC-2",
"title": "Account Management",
"description": "Manage system accounts, group memberships,
privileges, workflow, notifications, deactivations,
and authorizations."
},
"framework": "NIST 800-171",
"criterion": {
"id": "3.1.1",
"text": "Limit system access to authorized users, processes
acting on behalf of authorized users, and devices
(including other systems)."
},
"mapping": true,
"reasoning": "Account Management directly controls which users
are authorized to access the system, making it a primary
control for this access restriction criterion."
}

2b. Synthetic Expansion

Once you have a seed dataset of expert-curated examples, use a capable reasoning LLM to generate variations. Strike Graph used DeepSeek and Kimi K2.5 for this step, choosing them for their strong performance on structured reasoning tasks at low cost. These models were given correct input-output pairs and prompted to generate the intermediate reasoning: why would an auditor connect this control to that criterion?

Filter synthetically generated examples against your ground-truth before including them in training. Here is an example of a synthetically generated record produced from a seed example:

#Example: synthetically generated training record.
#Produced by DeepSeek/Kimi K2.5 from a seed ground-truth pair
{
"control": {
"id": "AC-2(2)",
"title": "Account Management — Automated Temporary Account Management",
"description": "Automatically remove or disable temporary and
emergency accounts after an organizationally-defined period."
},
"framework": "NIST 800-171",
"criterion": {
"id": "3.1.1",
"text": "Limit system access to authorized users..."
},
"mapping": true,
"reasoning": "Automated removal of temporary accounts ensures
that access is continuously limited to currently-authorized
users — directly satisfying the intent of 3.1.1 by preventing
stale or unauthorized access from persisting."
}

2c. Format for Fine-Tuning

Structure each example as a prompt-completion pair matching your model's chat template format:

{
"messages": [
{
"role": "system",
"content": "You are a compliance mapping expert..."
},
{
"role": "user",
"content": "Control: [control text]\nFramework: NIST 800-171\nTask: Generate
search queries..."
},
{
"role": "assistant",
"content": "[correct query list]"
}]
}

Step 3: Fine-Tuning on Amazon SageMaker

Strike Graph performed all model fine-tuning on Amazon SageMaker using ml.g5.xlarge instances, chosen for their fit with 3-4B parameter models. The pipeline runs in two phases: supervised fine-tuning (SFT) first, followed by GRPO (Group Relative Policy Optimization) reinforcement learning on the tasks where the reward signal justifies it. Each SFT pass on 1,000-2,000 training examples took roughly 3-5 hours. GRPO runs, which generate four candidate responses per sample, took 12-16 hours.

3a. Data Preparation

Before training, expert-curated gold-standard control-to-criteria mappings are transformed into multi-turn teacher trajectories. For the search and mapping tasks, these trajectories include reasoning steps modeled on auditor logic and are formatted to ChatML. Rejection sampling on per-bucket recall filters out low-quality synthetic examples before they enter the training set.

For GRPO training specifically, Strike Graph seeded an OpenSearch cluster with real, searchable compliance data. During training, the model's generated search queries are run against the live index, and the actual retrieved results are used to score recall. This grounds the reward signal in real retrieval behavior rather than simulated outputs.

3b. Supervised Fine-Tuning (SFT)

SFT teaches the model the domain language, task structure, and reasoning patterns from the expert-curated dataset. This is the baseline - run it first and use this baseline to evaluate against hold-out test dataset. We advise only to proceed to GRPO, if there are measurable gaps in recall or precision. For Strike Graph's use case, the signal to proceed was a post-SFT recall below the commercial model baseline or less than 2-3 percentage points above it — insufficient improvement to warrant concluding training without reinforcement learning

#Using Hugging Face TRL's SFTTrainer on SageMaker
from trl import SFTTrainer, SFTConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
training_args = SFTConfig(
output_dir="./sft-compliance-model",
num_train_epochs=3,
per_device_train_batch_size=4,
learning_rate=2e-5,
max_seq_length=2048,
)
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=train_dataset, # your formatted dataset
args=training_args,
)
trainer.train()

3c. GRPO Reinforcement Learning

Compliance control-to-criteria mapping requires multi-step reasoning, not one-shot retrieval. Controls are written in operational language whereas criteria are written in framework and legal language. The model must bridge that vocabulary gap by reformulating search queries, then evaluate which candidate results are actually relevant. SFT teaches the shape of that reasoning loop by imitation. But hard cases, where many controls satisfy one criterion or one control spans many criteria, have few clean demonstrations to imitate. GRPO optimizes the search-and-select policy directly against a verifiable reward: did the trajectory actually retrieve the correct mapping? This signal is exactly what imitation learning cannot supply where demonstrations are thin.

Strike Graph applied GRPO to one of the three models, specifically the compliance query generation model used in the criteria mapping task. For the control mapping and evidence search models, SFT alone produced results that met the bar.

When to use GRPO: Strike Graph's decision rule: test GRPO against a defined recall reward unless SFT already far exceeds the baseline. After training, accept the GRPO model only if recall and precision both improve, the reasoning structure stays sound, and output length stays similar or shorter. A length increase requires at least 2-3 percentage points of additional recall to justify it.

# GRPO with TRL GRPOTrainer on SageMaker
from trl import GRPOTrainer, GRPOConfig
def compliance_reward(completions, ground_truth, **kwargs):
"""Reward = recall: fraction of correct criteria present in model output"""
rewards = []
for completion, gt in zip(completions, ground_truth):
gt_set = set(gt.lower().split(","))
pred_set = set(completion.lower().split(","))
recall = len(gt_set & pred_set) / len(gt_set) if gt_set else 0.0
rewards.append(recall)
return rewards
grpo_config = GRPOConfig(
output_dir="./grpo-compliance-model",
num_train_epochs=2,
per_device_train_batch_size=2,
learning_rate=1e-6,
)
trainer = GRPOTrainer(
model=sft_model, # start from SFT checkpoint
reward_funcs=[compliance_reward],
args=grpo_config,
train_dataset=train_dataset,
)
trainer.train()

Step 4: Deploying on Amazon Bedrock

Amazon Bedrock's custom model import feature allows you to import fine-tuned weights and serve them through the standard Bedrock API — It is a serverless service and requires no infrastructure management.

4a. Package Your Model

# Save fine-tuned model in Safetensors
formatmodel.save_pretrained("./my-compliance-model", safe_serialization=True)tokenizer.save_pretrained("./my-compliance-model")
# Upload to S3
aws s3 cp ./my-compliance-model s3://your-bucket/models/compliance-model/ --recursive

4b. Import to Bedrock

4c. Invoke via Bedrock API
import boto3, json
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
response = bedrock.invoke_model( modelId="arn:aws:bedrock:us-east-1:ACCOUNT_ID:imported-model/MODEL_ID", body=json.dumps({ "prompt": "Control: [control text]\nGenerate compliance search queries:", "max_tokens": 512, "temperature": 0.1}))
result = json.loads(response["body"].read())

Cost tip: Bedrock Custom Model Import bills in 5-minute windows and scales to zero between invocations. For Strike Graph's batch workload, this makes Bedrock more cost-effective than a continuously running SageMaker endpoint up to roughly 59 five-minute invocation windows per day (about 5 hours of active use). A SageMaker ml.g5.xlarge running 12 hours/day costs approximately $507/month; always-on 24-hour availability runs approximately $1,028/month. This is below Strike Graph's current usage threshold, and so Bedrock CMI is the more economical choice. Above it — or for real-time, high-volume workloads — a persistent SageMaker endpoint becomes more attractive.

Benchmarking Results

Designing Human-in-the-Loop Pipelines

Fine-tuned small models at 65–77% recall are not autonomous compliance agents. They are high-precision assistants that surface what a compliance professional should consider. Designing the workflow around this distinction is what makes the system useful at its current performance level.

Design Principles

• Optimize for recall, not precision. In compliance, it is better to surface a candidate that gets rejected than to miss one that matters. Design your reward signal, training data, and evaluation accordingly.

• Present model outputs as candidates, not conclusions. The UI should frame AI suggestions as items to review, not decisions already made. This keeps the compliance professional in the judgment seat.

• Make disagreement easy. A human-in-the-loop system only works if it's fast for a professional to accept, reject, or modify a suggestion. Invest in the review UX.

• Track disagreement as training signal. When a compliance professional overrides a model suggestion, that is a labeled example. Build a pipeline to capture those overrides and fold them into future fine-tuning runs.

• Evaluate recall on your own domain data. Published benchmarks won't tell you how a model performs on your specific control library and framework version. Build an evaluation set from your own ground truth from day one.

Cost and Performance Optimization Tips

Optimization Tips

• Use Amazon Bedrock's on-demand mode for variable workloads — no always-on GPU cost when models aren't being called.
• Experiment with the max concurrent calls you can get while only using 1 CMU to optimize spend. It’s often between 8-16 depending on tokens.
• Set temperature low (0.0–0.1) for deterministic recall tasks — you want consistent retrieval, not creative variation.
• Profile your tasks separately. Strike Graph found that query generation and control mapping have very different latency characteristics and optimized each independently.

Next Steps

What's Next for Strike Graph

Three features are in active development or already live based on these models:

• Control-to-Criteria Mapping Suggestions (live now): AI Security Assistant recommends which controls map to each compliance framework criterion.

• Advanced Evidence Testing (live now): Evidence tested not only against requirements but against other evidence for consistency — flagging whether a terminated employee persists in AWS admin logs after removal from an HRIS system.

• Compliance Atlas (live now): A proactive feature that analyzes a customer's compliance program and suggests improvements before auditors find gaps.

Conclusion and Call to Action

Key Takeaways

• Small models win on specific tasks. For domain-specific problems, a 3–4B model trained on expert-curated data can outperform a 70B+ commercial model — at a fraction of the cost.

• Data quality beats model size. The investment in expert-curated ground truth and careful synthetic expansion is the primary factor in achieving these results. Architecture and training technique matter less.

• Decompose the problem. Breaking workflows into discrete pipeline steps — search, then map, then compare — lets each model focus on a narrower problem and improves overall system performance.

• Human-in-the-loop is a product decision, not a limitation. Optimizing for recall and designing review workflows around model outputs is what makes the automation genuinely useful to compliance professionals.

• Amazon Bedrock + Amazon SageMaker provide the operational foundation for this approach. On-demand GPU access via SageMaker kept training costs low; Bedrock's custom model import eliminated always-on infrastructure costs for serving.

Try It Yourself

• Amazon Bedrock Custom Model Fine-Tuning 

• Strike Graph White Paper: Small Models, Big Results Full benchmark methodology, training pipeline, and technical findings.

• Hugging Face TRL : SFT & GRPO SFTTrainer and GRPOTrainer documentation for the training pipeline used in this guide.

• Meta Llama 3.2 3B on Hugging Face  Model card and weights for the base architecture used in this guide. 

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.