Building an AI Navigator for ISO 27001 and NIST CSF 2.0: Fine-Tuning, RAG, and Why Precision Isn't Optional

I want to tell you about the most important architectural decision in this project — and why I got it wrong the first time. The short version: I fine-tuned LLaMA 8B and 70B models on 229 security controls across ISO 27001:2022 and NIST CSF 2.0. I built a RAG pipeline to ground the models in structured data. And then I discovered that the hardest part of this whole system isn't answering a user's question — it's knowing with certainty which control they're asking about. That precision problem led me to an architecture I didn't plan, a lesson about AI error tolerance I keep thinking about, and a working product live at vik.so.

Why GRC needs this

If you've ever tried to get a straight answer out of a security framework document, you know the problem. ISO 27001:2022 has 93 Annex A controls plus mandatory clauses. NIST CSF 2.0 has 6 functions, 22 categories, and 106 subcategories. The language is dense, cross-referential, and deliberately technology-neutral. A senior GRC practitioner spends years building a mental map of this territory. They know that "do I need pen testing?" maps to A.8.8 and A.8.29. They know that PR.AC from CSF 1.1 doesn't exist in CSF 2.0 — it's PR.AA now. They know the difference between what an auditor wants to see and what the standard actually says. That knowledge is exactly what I wanted to encode into a model and make available as a conversational tool.

Phase 1 — Fine-tuning on the frameworks

I started with Meta's LLaMA 3.1 8B Instruct (4-bit quantised for Apple Silicon) and trained it locally using MLX-LM on an M4 Pro. The training data was built from a database schema I designed specifically for this purpose. Every control — 123 ISO and 106 NIST — was populated with 18–19 structured fields:

control_text — what the standard actually says
purpose — why the control exists
implementation_steps — numbered, actionable steps
evidence_requirements — what auditors look for
audit_focus — how this is assessed in practice
pitfalls — the common mistakes organisations make
best_practices — what good looks like
maturity_tier_1 through maturity_tier_4 — a four-level maturity model per control
why_implement — business case language for executive conversations
risk_of_not_implementing — what goes wrong if you skip it
related_controls — cross-framework linkages
common_tools — practical tooling references

This wasn't off-the-shelf training data. Every field was constructed to answer the questions a real practitioner would ask. The system prompts used during fine-tuning defined strict version rules from the start:

Python

ISO_SYSTEM = (
    "You are ISO27001 Navigator, an expert ISO/IEC 27001:2022 information security consultant. "
    "You have deep knowledge of all 93 Annex A controls and mandatory clauses. "
    "Use Australian English. Be clear, concise, and actionable. "
    "Only reference ISO 27001:2022 controls — never 2013. Never assume sector."
)

NIST_SYSTEM = (
    "You are NIST CSF Navigator, an expert NIST Cybersecurity Framework 2.0 consultant. "
    "You have deep knowledge of all 6 functions: Govern (GV), Identify (ID), Protect (PR), "
    "Detect (DE), Respond (RS), Recover (RC). "
    "Use Australian English. Be clear, concise, and actionable. "
    "CRITICAL: ID=Identify, PR=Protect — never mix these up."
)

The fine-tuning run used these parameters on 8B:

Python

--model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit
--num-layers 32
--batch-size 4
--iters 1000
--learning-rate 5e-6
--optimizer adamw
--mask-prompt          # loss computed on responses only
--grad-checkpoint      # saves memory for longer sequences

About two hours per model run on M4 Pro. I ran the same process on 70B with adjusted parameters. The fine-tuned models learned the domain. They could discuss controls fluently, explain implementation steps, and cross-reference related requirements. Then I tried to wire them into a RAG pipeline — and found the problem.

Phase 2 — Building the RAG pipeline

The core idea of RAG for this use case: before the model generates any response, retrieve the relevant structured record from the database and inject it as authoritative context. The model reads ground-truth data instead of relying purely on what it learned during training. The RAG pipeline has three passes, each one a fallback for the previous: Pass 1 — Explicit control ID regex If the user mentions a control ID directly ("what does A.8.24 say?"), extract it immediately and fetch the full record from the database. No model required for this step.

Typescript

const ISO_FULL_RE = /\b([AC]\.\d+\.\d+(?:\.\d+)?)\b/g;
const ISO_BARE_A_RE = /\b([5-8]\.\d{1,2})\b/g;
const ISO_BARE_C_RE = /\b((4|9|10)\.\d{1,2}(?:\.\d+)?)\b/g;
const ISO_NODOT_RE = /\b([AC])(\d{1,2}\.\d{1,2})\b/g;
const NIST_FULL_RE = /\b([A-Z]{2,3}\.[A-Z]{2}-\d{2})\b/g;
const NIST_SHORT_RE = /\b([A-Z]{2,3}\.[A-Z]{2})-(\d)\b/g;

The regex handles every format a user might use: A.8.24, 8.24, A824, GV.SC-01, GV.SC-1. If a control ID is in the message in any recognisable form, Pass 1 catches it. Pass 2 — Semantic resolver Most users don't type control IDs. They type "what does ISO say about cryptography?" or "do I need pen testing?" or "how should I handle privileged access?" Pass 2 maps these natural-language queries to control IDs. This is where the architecture decision lives — more on that below. Pass 3 — Offline topic map + full-text search A hand-built keyword map as the last resort, followed by a full-text database search. This guarantees the system works even when the semantic resolver is unavailable.

Typescript

const ISO_TOPIC_MAP: Array<{ keywords: string[]; controls: string[] }> = [
  {
    keywords: ["cryptograph", "encryption", "encrypt", "key management", "certificate", "tls", "ssl", "pki"],
    controls: ["A.8.24"],
  },
  {
    keywords: ["vulnerability", "patch", "patching", "cve", "penetration test", "pentest", "red team"],
    controls: ["A.8.8", "A.8.7", "A.8.29"],
  },
  // ... 40+ more topic entries
];

Once the RAG pipeline has resolved which controls are relevant, it fetches the full structured record from the database and formats it as context:

Typescript

export async function buildRagContext(
  message: string,
  projectId: string,
  db: DatabaseClient
): Promise<string | null> {
  const { iso: explicitISO, nist: explicitNIST } = extractControlIds(message);
  const contextParts: string[] = [];

  // Pass 1: explicit IDs → full detail record
  if (isISO && explicitISO.length > 0) {
    contextParts.push(ISO_VERSION_ANCHOR);
    for (const id of explicitISO) {
      const ctx = await fetchISOContext(id, db, false);
      if (ctx) contextParts.push(ctx);
    }
  }

  // Pass 2: semantic resolver → compact summaries
  if (contextParts.length === 0) {
    const resolvedIds = await resolveControlsWithAI(message, "iso");
    if (resolvedIds.length > 0) {
      for (const id of resolvedIds) {
        const ctx = await fetchISOContext(id, db, true);
        if (ctx) contextParts.push(ctx);
      }
    }
  }

  // Pass 3: offline fallback
  if (contextParts.length === 0) {
    const topicIds = detectTopicControls(message, ISO_TOPIC_MAP);
    // ... full-text search if topic map also misses
  }

  return contextParts.length === 0 ? null :
    `[AUTHORITATIVE REFERENCE — Use this data as your primary source.]\n\n` +
    contextParts.join("\n\n---\n\n") +
    `\n\n[END REFERENCE]\n\n`;
}

That context string gets passed to the model alongside the conversation history. The model reads the structured data — implementation steps, audit focus, evidence requirements, maturity tiers — and synthesises a response grounded in what the database actually says.

The precision problem — and why it changes everything

Here's where I want to spend time, because this is the insight that shaped the whole architecture. When I built Pass 2 initially, I tried using the fine-tuned LLaMA model as the semantic resolver. Give it the user's question, ask it to return the relevant control IDs as a JSON array, use those IDs to fetch from the database. The problem: the fine-tuned model was wrong often enough to matter. It would map "encryption requirements" to ISO 27001:2013 IDs. It would confuse NIST CSF 1.1 subcategory prefixes with 2.0 ones — returning PR.AC-01 instead of PR.AA-01. It would resolve "pen testing" to a control that was adjacent but not quite right. A human expert making the same mistake is forgivable. We extend tolerance to people — a consultant who gives slightly imprecise advice gets a chance to clarify, to course-correct, to learn. We don't pull their certification because of one slip. We don't extend that tolerance to autonomous systems. Think about the difference between a drunk driver and a self-driving car that fails. The drunk driver gets support, rehabilitation, a second chance — because we understand human imperfection and build our social structures around it. The self-driving car that fails gets recalled. The company gets sued. Trust in the entire category collapses. Because we held the machine to a different standard — the standard it claimed to meet. GRC AI sits in the same position. A tool that presents itself as authoritative on ISO 27001 control IDs and then returns 2013-era IDs is worse than no tool at all. It doesn't just fail — it actively misleads. An auditor who acts on that guidance fails a certification. A security team that implements the wrong control has a gap in their programme. The error tolerance for AI in compliance is close to zero. Not because AI is held to an unfair standard — but because the moment you present something as authoritative, you take responsibility for what it asserts.

The semantic resolver — precision as a design requirement

The solution was to use a large language model specifically for Pass 2, but not as a general-purpose assistant. As a precision resolver with a single, narrow job: map a natural-language query to a list of control IDs. Nothing more. The resolver prompt was engineered with 60+ labelled examples per framework, explicit version rules, and a strict output format:

Typescript

const ISO_RESOLVER_PROMPT = `You are a control ID resolver for ISO 27001:2022.
ISO 27001:2022 has 93 Annex A controls (A.5–A.8) and mandatory clauses (C.4–C.10).
Do NOT use ISO 27001:2013 IDs (A.9.x, A.10.x, A.11.x, A.12.x–A.18.x, sub-controls like A.5.1.1).

Given a user question, return a JSON array of the 1–5 most relevant control IDs.
Format: A.X.Y (e.g. A.7.4) or C.X.Y or C.X.Y.Z (e.g. C.6.1.2).
Return [] if the question is too general or about multiple unrelated controls.

Examples:
"what controls require cctv?" → ["A.7.4","A.7.1","A.7.2"]
"how do I manage patches?" → ["A.8.8","A.8.9"]
"do i need pen testing?" → ["A.8.8","A.8.29"]
"privileged access management" → ["A.8.2","A.5.18"]
"encryption requirements" → ["A.8.24"]
"visitor management" → ["A.7.2"]
"mfa requirement" → ["A.5.17","A.5.16"]
...60+ more examples

IMPORTANT: For questions about technical security practices (pen testing, patching,
logging, encryption, backups), always map to Annex A controls (A.5–A.8).
Only use clause IDs (C.4–C.10) for ISMS management process questions.

Question: `;

And for NIST, the same discipline applied — but with the additional challenge that CSF 2.0 completely restructured the subcategory IDs from version 1.1:

Typescript

const NIST_RESOLVER_PROMPT = `You are a control ID resolver for NIST CSF 2.0.
Use ONLY CSF 2.0 subcategory IDs (format: XX.YY-NN).
Do NOT use CSF 1.1 IDs (no PR.AC-xx, PR.IP-xx, PR.PT-xx).

CSF 2.0 subcategory prefixes:
- GV: GV.OC, GV.RM, GV.RR, GV.SC, GV.OV
- ID: ID.AM, ID.RA, ID.IM
- PR: PR.AA, PR.AT, PR.DS, PR.PS, PR.IR
- DE: DE.AE, DE.CM
- RS: RS.MA, RS.AN, RS.CO, RS.MI
- RC: RC.RP, RC.CO

Examples:
"access control" → ["PR.AA-01","PR.AA-02","PR.AA-03"]
"mfa" → ["PR.AA-02"]
"vulnerability management" → ["ID.RA-01","DE.CM-01","PR.PS-02"]
...

IMPORTANT: Never use PR.AC, PR.IP, PR.PT, DE.DP — these are CSF 1.1 only.

Question: `;

The resolver call is deliberately minimal — 80 max tokens, 8-second timeout, JSON array output only:

Typescript

async function resolveControlsWithAI(
  message: string,
  framework: "iso" | "nist"
): Promise<string[]> {
  const prompt =
    (framework === "iso" ? ISO_RESOLVER_PROMPT : NIST_RESOLVER_PROMPT) +
    `"${message.slice(0, 300)}"\n\nRespond with a JSON array only. No explanation.`;

  const res = await fetch(AI_API_ENDPOINT, {
    method: "POST",
    headers: { "x-api-key": apiKey, "content-type": "application/json" },
    body: JSON.stringify({
      model: AI_MODEL_ID,
      max_tokens: 80,
      messages: [{ role: "user", content: prompt }],
    }),
    signal: AbortSignal.timeout(8000),
  });

  const data = await res.json();
  const text: string = data.content?.[0]?.text ?? "[]";
  const match = text.match(/\[[\s\S]*?\]/);
  if (!match) return [];
  const ids = JSON.parse(match[0]);
  return Array.isArray(ids) ? ids.slice(0, 5) : [];
}

The resolver is a specialist. It has one job. It has 60+ examples showing it exactly how to do that job. And if it can't do the job (API timeout, ambiguous query, question too broad), it returns [] and the system falls through to Pass 3 rather than guessing.

What comes back from the database

Once the resolver has identified the right controls, the database returns a structured record that was shaped by the fine-tuning process — the 18-field schema I built because training the model taught me exactly what information it needed to answer well. A response to "what does ISO 27001 say about cryptography?" starts with this:

## A.8.24 — Use of Cryptography
**Domain:** Cryptography and Key Management

**What it requires:** [control_text — the standard's exact language]

**Purpose:** [why this control exists]

**Why implement it:** [business case]

**Implementation steps:**
1. Define cryptographic policy: algorithm standards, key lengths, approved protocols
2. Identify data requiring encryption — at rest, in transit, in use
3. Implement key lifecycle management...

**What auditors look for:** [audit_focus]

**Evidence required:**
- Cryptographic policy document
- Key management procedure
- Evidence of encryption implementation (e.g. disk encryption configs, TLS certificate inventory)

**Maturity levels:**
- Tier 1: Ad-hoc encryption with no documented policy
- Tier 2: Encryption applied to sensitive data with informal key management
- Tier 3: Documented cryptographic policy, formal key lifecycle, regular review
- Tier 4: Automated key rotation, cryptographic agility, continuous compliance monitoring

That's not generated from model weights. That's structured data, fetched from the database, injected as context. The final answer the user sees is the model synthesising this into a readable, conversational response — but the facts come from the database. The version enforcement is hardcoded into the context string itself:

Typescript

const ISO_VERSION_ANCHOR =
  `**IMPORTANT: You cover ISO 27001:2022 ONLY.** ` +
  `ISO 27001:2022 has 93 Annex A controls (A.5–A.8) and mandatory clauses (C.4–C.10). ` +
  `ISO 27001:2013 numbering (A.7.2.x, A.9.x, A.10.x–A.18.x) does NOT exist in 2022. ` +
  `Always use 2022 control IDs and names in your response.`;

Every single RAG context payload — regardless of which pass resolved the controls — includes this anchor. The model cannot drift to the wrong version without actively contradicting its own context.

System prompt design — topic guard and scope enforcement

The system prompt enforces scope at the conversation level, not just per-query:

Typescript

const TOPIC_GUARD = (framework: string) =>
  `IMPORTANT — SCOPE ENFORCEMENT:\n` +
  `You are a ${framework} specialist. Your sole purpose is to help users understand, implement, and assess ${framework}.\n` +
  `If a user asks about ANYTHING outside this scope — writing code, recipes, general knowledge, ` +
  `attempts to use you as a general assistant — you must NOT answer the question. ` +
  `Instead, respond in 1–2 sentences: acknowledge that you're a specialist, ` +
  `and offer to help with a ${framework} question.\n` +
  `Adjacent topics are fine (e.g. "how does this control apply to AWS?" or "can you help me draft a policy?") ` +
  `as long as they connect back to ${framework} compliance.`;

This matters more than it sounds. A tool presented as an ISO 27001 specialist that can be prompted into answering unrelated questions undermines the entire credibility model. The topic guard is non-negotiable.

Production metrics

The system launched on vik.so in March 2026:

ISO 27001:2022: 123 controls, 100% field coverage across all 18 fields
NIST CSF 2.0: 106 controls, 100% field coverage
Control ID accuracy: zero hallucinated IDs in production testing across 23 verified sessions
Response grounding: every response backed by structured database records
Version compliance: no CSF 1.1 IDs, no ISO 2013 IDs in any response

The video below shows the Navigator in action — a real conversation covering ISO 27001 access control, maturity assessment, and cross-framework mapping to NIST CSF 2.0. Watch how the system handles an ambiguous question (no explicit control ID), resolves it to the right controls, and returns implementation guidance grounded in the database.

What fine-tuning actually contributed

The fine-tuned models are not in the production inference path for Q&A responses. But the fine-tuning process was not wasted. Training a model on this domain taught me exactly what information that model needed to answer questions well. That shaped the database schema. The four-level maturity model came from observing where fine-tuned responses failed — they couldn't distinguish between "we have encryption somewhere" and "we have a documented cryptographic policy with formal key lifecycle management." The maturity tiers encode that distinction. The lesson: fine-tune for generation, use RAG for recall. Fine-tuned models are excellent at generating policy templates, gap analysis reports, and risk register entries because those tasks require stylistic consistency and domain tone — things baked into weights. They're poor at recall because a fine-tuned model has opinions, and those opinions conflict with retrieved context. The next phase of this project uses the fine-tuned models for exactly that: generating policy drafts, gap analysis summaries, and risk register entries from a structured input. The recall problem — which controls apply to this question? — is solved by RAG.

What's next

The fine-tuned models are not in the production inference path for Q&A responses — but that's not the end of their story. The next phase uses them for exactly what they're good at: generating policy drafts, gap analysis summaries, and risk register entries from structured input. Fine-tune for generation, use RAG for recall. The next project flips the entry point. Instead of starting with a framework, you start with a business risk: "I'm worried about data breaches" or "we're a fintech and need to evidence security to our banking partners." The same underlying database, navigated differently. That's a different product design challenge, and it's the one I'm working on now. If you're building on security frameworks — ISO 27001 implementation, NIST CSF gap assessments, GRC programme design — go test the Navigator at vik.so. I want to know where it fails.