Deterministic vs. Probabilistic DLP: Why Regex and Luhn Checks Beat LLM-as-a-Judge for PII

It is interesting to me that the AI industry is trying to solve DLP (data loss prevention) using other LLMs, in what is called the “LLM-as-a-judge” technique. The basic idea is to have one LLM judge or review what the first LLM generated.

But doesn’t the fact that an LLM is reviewing the content itself constitute a security vulnerability?

Unless you are hosting the LLMs yourself, any information reaching an LLM is already compromised. By the time a second model gets a chance to “judge” whether sensitive data was present, that data has already left your network. The judge had to read the content to rule on it, and reading it is the breach.

SAFi takes a different approach. It handles DLP deterministically, using regular expressions and checksum algorithms such as Luhn validation before the data ever leaves your network. And if, by some chance, sensitive information does reach the LLM, SAFi blocks it on the way back to the user.

On the way in, SAFi evaluates the prompt for sensitive data before the model is called. On the way back out, it checks the entire generated response before it is delivered to the user. Both checks are plain Python running outside the model pipeline entirely. No LLM is invoked at any point in the detection path.

Picture a user who pastes a support ticket into a chat. The ticket body contains a credit card number. SAFi’s inbound scan catches it: the regex finds a sixteen-digit sequence that looks like a card number, the Luhn checksum confirms it is a valid one, and the turn is blocked before the model ever sees it. The user gets a message explaining the policy. Nobody had to send that number to a model to find out it was there. The arithmetic said so.

Now picture a different scenario. An agent with tool access reads a document from an internal system, finds an account number in it, and reproduces that number in its answer. Nobody typed it. The model produced it from a tool result. An inbound-only control would miss this entirely, because the sensitive data never appeared in the prompt. This is why the outbound check exists, and why it runs over the full generated response, not just a sample.

The distinction matters. Whether a credit card number satisfies the Luhn algorithm is arithmetic, and arithmetic has one correct answer that anyone holding the audit record can recompute. A language model asked the same question returns an answer that is usually right, occasionally not, carries no stable threshold, and cannot be independently verified afterward. For a control whose value is being checkable, that is the wrong tier.

And this is where the audit trail comes in. Every detection decision SAFi makes, inbound and outbound, is written to the Audit Hub with its stage and the reason it fired. The record shows what was caught, where, and why. It does not store the sensitive value itself, only the fact of the detection. So an auditor can pull the record, re-run the same regex and the same checksum against the same input, and arrive at the same answer. That is what “deterministic” buys you. Not just a faster check, but a check you can prove.

This is not a minor architectural preference. When a control’s purpose is to prove, after the fact, that sensitive data never reached a model, the control itself must be something an auditor can recompute and arrive at the same answer. A regex pattern matched against a Luhn checksum gives you that. A second LLM’s opinion does not.

I think the industry’s instinct to throw another model at the problem is understandable. Models are flexible, and flexibility is seductive. But DLP is not a flexibility problem. It is a correctness problem. And correctness problems are solved by math, not by asking a model how it feels about the math.