Field note · Analysis

Why audit logs aren't evidence

A log you ask people to trust is not the same as proof they can verify. The difference decides whether you pass an audit.

The claim

Most AI systems record what they did in application logs. When a regulator or customer asks "prove the control worked," teams export those logs. But a log is a record you control (you can append to it, edit it, or lose it), so it answers "what do you say happened," not "what can I independently verify happened."

The test: if you could silently alter a record after the fact and no one could tell, it is not evidence. It is a claim.

Why logs fall short

  • Mutable. Standard log stores allow edits and deletes; integrity depends on trusting the operator.
  • Unbound. A log line rarely binds the exact decision, the policy version, and the input that produced it.
  • Unverifiable by outsiders. An auditor cannot check a log without trusting your infrastructure, the very thing under review.

What makes evidence

Evidence is a record whose integrity can be checked by someone who does not trust you. Two properties get you there, both well established in security engineering:

  • Cryptographic signing. Each decision is signed (e.g. ES256) so anyone with the public key can confirm it came from the system and was not altered.
  • Hash-chaining. Each receipt commits to the previous one, so deleting or reordering any entry breaks the chain. Tampering becomes structurally detectable, the same idea behind transparency logs.

This maps directly onto the accountability expectations in frameworks like the NIST AI RMF (Govern/Manage) and the OWASP LLM Top 10's emphasis on traceable decisions.

In practice

In Intercept, every allow, block, and flag is signed and hash-chained the moment it is made. At review time you hand over the receipts and the verifier. The auditor checks the math, not your word.

Next: see Command for how receipts are signed and chained.