The fear about putting an LLM anywhere near a compliance determination is easy to state: the model hallucinates, writes “MET” on a control that isn’t, and a bad assessment ships. It is a reasonable fear. It is also avoidable, and the way you avoid it tells you exactly where the model belongs in the workflow.

The model belongs on the reading, not the deciding.

An assessor’s day is mostly reading. Read the policy, read the process document, read the screenshot, read the ticket, hold all of it against a requirement, and form a judgment. The reading is slow and the judgment is fast. An LLM inverts that ratio. It reads a fifty-page policy suite in seconds and surfaces what’s relevant to a given requirement, and then a human makes the call in the time it takes to agree or disagree. The model never writes the determination. It writes the brief the determination is made from.

That single boundary, the model reads and the human decides, is what makes AI-assisted compliance work defensible. But the boundary alone isn’t enough. Four disciplines make the difference between a tool you can put in front of a QA reviewer and a tool that quietly manufactures plausible nonsense.

Primary sources, not summaries

The failure mode that matters most is the model answering from the wrong document. Ask an LLM “does this organization meet the audit-logging requirement” and it will happily answer from a policy that describes audit logging, when the requirement is about logging that is implemented and operating. The fix is to feed it the actual evidence, scoped to the actual requirement, and to separate the evidence types so the right criteria apply to each. Policy documents get read against policy criteria. Process documents against process criteria. Implementation evidence against implementation criteria. When the pipeline collapses those distinctions, the model conflates them, and so would a rushed human.

This is not an AI-specific discipline. It is assessment discipline. The model just makes the cost of skipping it visible faster.

Self-correction against the source

Here is the behavior I most want to see from an AI-assisted review, and the one people rarely test for. Give the model a first-pass read from a summary, then give it the primary source, and watch whether it updates.

I ran an analysis recently where the model’s early read, working from secondary coverage, characterized a set of comment letters one way. After it read the letters in full, it reversed the specific claim and flagged the reversal rather than smoothing it over. The direction of the conclusion held, but the mechanism behind it changed, and it said so.

That is the tell for a trustworthy pipeline. A model that returns the same answer no matter what the source says is not reading the source. Build the review so that the primary document is always the last thing the model sees before it writes its notes, and so that a contradiction between the summary and the source produces a flagged change, not a silent average of the two.

Stated limits

A good analyst tells you what they could not determine. So should the tool. When the evidence doesn’t support a determination, the right output is “insufficient evidence, here is what’s missing,” not a confident guess dressed as a finding. When a judgment depends on something the model can’t see, environment context, a conversation that happened in an interview, a control that operates outside the documents provided, the output should say so and route it to the human.

The tools that get people in trouble are the ones that never say “I don’t know.” Force the “I don’t know” path to exist and to be used.

Human-in-the-loop on every determination

None of the above removes the assessor. It relocates the assessor’s time. Instead of spending sixty hours reading to reach a judgment, the assessor spends twenty-five hours reviewing structured first-pass notes and reaching the same judgment with the reading already done. Every MET, NOT MET, and PARTIALLY MET is confirmed by a person who can be named in the assessment. The model produces the draft. The human signs it.

That is not a hedge. It is the whole design. In compliance the determination is the thing being purchased, and a determination no human will stand behind has no value, no matter how fast it was produced.

Why this shape fits compliance specifically

Compliance work has a property most LLM applications don’t: an adversarial reviewer at the end. A HITRUST QA reviewer, a SOC 2 report user, an auditor, a regulator. That reviewer will ask “show me the evidence and show me your reasoning.” A pipeline built the way I’ve described answers that question by construction. The first-pass notes cite the evidence with page-level references. The determination is a human’s, on the record. The place where the model changed its mind is documented rather than hidden. The audit trail is a byproduct of the method, not a thing assembled afterward under deadline.

Contrast that with the tempting shortcut, an LLM that reads the whole environment and emits determinations directly. It is faster to build and impossible to defend. When the reviewer pulls a thread, there is no human reasoning underneath it, only a model output nobody examined. The first approach makes the review shorter. The second makes it a liability.

The rule of thumb

Put the model where the reading is, keep the human where the judgment is, and instrument the boundary so that the source always wins. The speedup is real, first-pass evidence review on comparable frameworks drops from the sixty-to-eighty-hour range to twenty-five-to-thirty-five, and it comes without the failure mode everyone fears, because the model was never allowed to make the call in the first place.

Use the LLM to read the source. Make the call yourself.

Running compliance assessments, or building the automation that supports them? Get in touch.

About the author

Gary Isaac is a HITRUST CCSFP and CISA. Fourteen years inside the ONC test lab and HITRUST practice at Drummond Group, where he helped launch the firm’s HITRUST service line. He builds AI-assisted compliance assessment tooling on an open-source cybersecurity assessment platform, with the assessor in the loop on every determination.