EU AI Act audit evidence architecture for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · EU AI Act · layout matrix_first · implementation executed in CI before publish

The control that resolves this is an Attestable Inference Plane, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is EU AI Act, and the question it actually asks is what risk class is this system, and where is the technical documentation. The shape below holds p95 ≤ 600ms over durable queue (SQS / Kafka) with an outbox, and ships with a standard-library implementation that runs as written.

Deployment parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

Controls and acceptance parameters

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Why this blocks production

The model works and the pilot passed. The rollout is stopped by a control gap: the team can show that inference happened, but cannot produce per-tenant, tamper-evident evidence of WHAT was processed, by WHOM, under WHICH retention rule — which is the evidence an auditor asks for and a shared application log cannot give.

What EU AI Act specifically requires

The EU AI Act regulates the SYSTEM, not the data. The first question is its risk classification, and a high-risk classification pulls in Article 11 technical documentation, Article 12 record-keeping and Article 14 human oversight — obligations that have to be designed in, not attached later.

The artifact that satisfies it is a model card plus Article 12 automatic logs. The way teams most often fail it: a system classified as limited-risk that is actually high-risk because it informs a decision about a person.

Implementation components

1. Policy decision point — Evaluates tenant, data class and purpose before the call. Fails closed.
2. Redaction stage — Strips sensitive values and substitutes digests, BEFORE anything is sealed.
3. Per-tenant hash chain — Each record commits to the previous digest, keyed per tenant.
4. HMAC seal — A KMS-held key makes records unforgeable by anyone with store write access.
5. Retention scheduler — Enforces the per-class retention clock and records the deletion.
6. Verifier endpoint — The function you run in front of the auditor to prove the chain is intact.

The request path

Attestable Inference Plane — the request path, with the decision points that must exist before the first model call:

flowchart LR
    C[Client / Agent] --> G[Gateway]
    G --> PDP{Policy Decision Point<br/>tenant - class - purpose}
    PDP -->|deny| D[403 + denial record]
    PDP -->|allow| R[Redaction stage<br/>values to digests]
    R --> M[Model provider]
    M --> S[Seal: sha256 chain + HMAC]
    S --> L[(Per-tenant audit store)]
    L --> V[Verifier endpoint]
    V --> A[Auditor evidence pack]
    L --> RT[Retention scheduler]
    RT -->|expiry| X[Documented deletion]

Reference implementation — Per-tenant tamper-evident audit chain

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind durable queue (SQS / Kafka) with an outbox; the failure mode it guards against is a consumer crash between side effect and acknowledgement. The output you hand the assessor is the Annex III classification record and model card.

"""Per-tenant, tamper-evident audit log for an LLM gateway.

Why this shape: a SOC 2 auditor does not ask "do you log?" — they ask whether a
log entry can be altered after the fact without detection, and whether tenant A
can ever read tenant B's entries. A plain append-only table answers neither.

This answers both with a per-tenant hash chain: each record commits to the
previous record's digest, so any edit or deletion breaks the chain at a
detectable point, and the chain is keyed per tenant so isolation is structural
rather than a WHERE clause someone can forget.
"""
import hashlib
import hmac
import json
import os
import time
from dataclasses import dataclass, field


GENESIS = "0" * 64


def _canonical(obj) -> bytes:
    # Digests must be stable across processes and Python versions, so the JSON
    # has to be canonical: sorted keys, no insignificant whitespace.
    return json.dumps(obj, sort_keys=True, separators=(",", ":")).encode()


@dataclass
class AuditChain:
    """Append-only, per-tenant hash chain with an HMAC seal.

    secret: a per-deployment key from your KMS. It makes the seal
    unforgeable by anyone who only has write access to the store.
    """
    secret: bytes
    heads: dict = field(default_factory=dict)   # tenant_id -> last digest

    def append(self, tenant_id: str, event: dict) -> dict:
        if not tenant_id:
            raise ValueError("tenant_id is required: an unattributed audit "
                             "record cannot satisfy per-tenant evidence")
        prev = self.heads.get(tenant_id, GENESIS)
        record = {
            "tenant_id": tenant_id,
            "ts": event.get("ts") or time.time(),
            "actor": event.get("actor"),
            "action": event["action"],
            "resource": event.get("resource"),
            "prev": prev,
            # Redaction happens BEFORE sealing. Sealing a payload you later
            # need to redact forces you to break your own chain.
            "payload": redact(event.get("payload") or {}),
        }
        digest = hashlib.sha256(_canonical(record)).hexdigest()
        seal = hmac.new(self.secret, digest.encode(), hashlib.sha256).hexdigest()
        record["digest"] = digest
        record["seal"] = seal
        self.heads[tenant_id] = digest
        return record

    def verify(self, tenant_id: str, records: list) -> tuple:
        """-> (ok, first_broken_index). This is the function you demo to the
        auditor; without it the chain is a claim rather than evidence."""
        prev = GENESIS
        for i, r in enumerate(records):
            if r.get("tenant_id") != tenant_id:
                return False, i          # cross-tenant record in the stream
            body = {k: r[k] for k in
                    ("tenant_id", "ts", "actor", "action", "resource",
                     "prev", "payload")}
            if r.get("prev") != prev:
                return False, i
            if hashlib.sha256(_canonical(body)).hexdigest() != r.get("digest"):
                return False, i
            expect = hmac.new(self.secret, r["digest"].encode(),
                              hashlib.sha256).hexdigest()
            if not hmac.compare_digest(expect, r.get("seal", "")):
                return False, i
            prev = r["digest"]
        return True, -1


SENSITIVE_KEYS = {"prompt", "completion", "email", "ssn", "dob", "mrn",
                  "card", "iban", "phone", "address", "name"}


def redact(payload: dict, keep_hash: bool = True) -> dict:
    """Drop sensitive values, keep a digest so you can still prove WHICH
    payload was processed without storing the payload itself.

    This is the control that lets you keep audit evidence under GDPR data
    minimisation and an HIPAA BAA at the same time."""
    out = {}
    for k, v in payload.items():
        if k.lower() in SENSITIVE_KEYS:
            out[k] = ("sha256:" + hashlib.sha256(
                _canonical(v)).hexdigest()[:16]) if keep_hash else "[redacted]"
        elif isinstance(v, dict):
            out[k] = redact(v, keep_hash)
        else:
            out[k] = v
    return out


if __name__ == "__main__":
    chain = AuditChain(secret=os.environb.get(b"AUDIT_HMAC_KEY", b"dev-only-key"))
    stream = [
        chain.append("tenant-a", {"actor": "svc:rag", "action": "inference",
                                  "resource": "gpt-4o",
                                  "payload": {"prompt": "patient MRN 9912",
                                              "tokens": 318}}),
        chain.append("tenant-a", {"actor": "u:42", "action": "export",
                                  "resource": "conversation/77"}),
    ]
    print("redacted payload:", stream[0]["payload"])
    print("chain verifies:", chain.verify("tenant-a", stream))
    stream[0]["payload"]["tokens"] = 1          # tamper
    print("after tamper:   ", chain.verify("tenant-a", stream))

The EU AI Act control, as code

Annex IV documentation completeness and Article 12 logging. This is the artifact an assessor asks for under EU AI Act specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""Annex IV technical documentation completeness + Article 12 logging.

A high-risk system under the EU AI Act needs technical documentation covering
specified elements, and automatic recording of events over the system lifetime.
The failure is rarely refusal to document — it is documentation that is stale,
partial, or cannot be tied to the exact model version that produced an output.

This checks the documentation set against the Annex IV elements and refuses to
mark a release conformant while any element is missing or stale.
"""
from datetime import date, timedelta

ANNEX_IV_ELEMENTS = {
    "general_description": "intended purpose, provider, version",
    "design_specification": "architecture, algorithmic logic, design choices",
    "training_data": "provenance, scope, characteristics, labelling",
    "human_oversight": "measures enabling natural-person oversight",
    "accuracy_metrics": "accuracy, robustness and cybersecurity metrics",
    "risk_management": "risk management system per Article 9",
    "change_log": "changes made through the lifecycle",
    "post_market_plan": "post-market monitoring plan",
}

STALENESS_DAYS = 180


class TechnicalFile:
    def __init__(self, system: str, model_version: str, today: date):
        self.system, self.model_version, self.today = system, model_version, today
        self.elements: dict[str, date] = {}

    def attach(self, element: str, updated: date) -> None:
        if element not in ANNEX_IV_ELEMENTS:
            raise KeyError(f"{element} is not an Annex IV element")
        self.elements[element] = updated

    def missing(self) -> list[str]:
        return sorted(set(ANNEX_IV_ELEMENTS) - set(self.elements))

    def stale(self) -> list[str]:
        cutoff = self.today - timedelta(days=STALENESS_DAYS)
        return sorted(k for k, v in self.elements.items() if v < cutoff)

    def conformant(self) -> dict:
        missing, stale = self.missing(), self.stale()
        return {"system": self.system, "model_version": self.model_version,
                "missing": missing, "stale": stale,
                "conformant": not missing and not stale}


class EventLog:
    """Article 12: records must tie an output to the version that produced it."""

    def __init__(self):
        self.events: list[dict] = []

    def record(self, model_version: str, event: str, ref: str) -> None:
        self.events.append({"model_version": model_version,
                            "event": event, "ref": ref})

    def trace(self, ref: str) -> dict | None:
        for e in self.events:
            if e["ref"] == ref:
                return e
        return None

    def untraceable(self, known_versions: set[str]) -> list[str]:
        return [e["ref"] for e in self.events
                if e["model_version"] not in known_versions]


if __name__ == "__main__":
    today = date(2026, 10, 16)
    tf = TechnicalFile("claims-triage", "v4.2.0", today)
    for element in ANNEX_IV_ELEMENTS:
        tf.attach(element, today - timedelta(days=30))
    assert tf.conformant()["conformant"], tf.conformant()

    partial = TechnicalFile("claims-triage", "v4.3.0", today)
    for element in list(ANNEX_IV_ELEMENTS)[:5]:
        partial.attach(element, today - timedelta(days=10))
    partial.attach("change_log", today - timedelta(days=400))
    report = partial.conformant()
    assert not report["conformant"], report
    assert "post_market_plan" in report["missing"], report
    assert "change_log" in report["stale"], report

    log = EventLog()
    log.record("v4.2.0", "inference", "out-9001")
    log.record("v4.1.7", "inference", "out-8800")
    assert log.trace("out-9001")["model_version"] == "v4.2.0"
    assert log.untraceable({"v4.2.0"}) == ["out-8800"]

    print("EU AI Act Annex IV + Article 12: OK")
    print(f"  complete file conformant={tf.conformant()['conformant']}, "
          f"partial missing {len(report['missing'])}, stale {len(report['stale'])}")

What the auditor will ask

• What is this system's risk classification under Annex III, who decided, and what happens to that classification when the use case changes?
  The classification is recorded against its Annex III category with a named owner, and a material change of use triggers re-classification before the change ships.
• Can you produce, for one named tenant, every inference record in a date range with proof none was altered?
  Yes — the verifier endpoint replays that tenant's hash chain for the date range and reports the first index at which the chain breaks. Each record commits to the previous record's digest, so an edit or deletion is detectable rather than merely unlikely.
• Where exactly in the request path are sensitive values removed, and what is stored in their place?
  In the redaction stage, before the record is sealed. Sensitive values are replaced with a sha256 digest of the value, so you can still prove which payload was processed without retaining the payload.
• Which key seals the records, who can use it, and what happens to verification after rotation?
  An HMAC key held in your KMS, usable only by the sealing service. Each record stores the key id it was sealed with, so the verifier resolves historical records against the KMS key history and rotation does not invalidate past evidence.
• What is the retention clock per data class, and what evidence proves deletion occurred?
  A retention clock per data class, enforced by the retention scheduler, which writes a deletion record when a record passes its horizon. The deletion log is what you show the auditor; reconcile it monthly against records in the store.
• Can tenant A's records ever appear in tenant B's evidence pack, and what makes that structurally impossible?
  No. The chain is keyed per tenant and the verifier rejects any record whose tenant_id does not match the stream being verified, so isolation is structural rather than a WHERE clause that can be forgotten.

Source discussions

This page exists because the problem is reported publicly. The threads below are the primary sources, linked unmodified and not affiliated with this site.

• Feature: Structured compliance audit logging for EU AI Act (Article 12) github · langchain-ai/langchain · 2026-02-20
• docs: Add EU AI Act compliance guide for LangChain deployers github · langchain-ai/langchain · 2026-03-23
• EU AI Act Compliance: Risk Classification & Transparency Documentation github · langchain-ai/langchain · 2026-02-21
• Article 6 Compliance: EU AI Act Risk Assessment & Documentation github · langchain-ai/langchain · 2026-02-21
• ToTra – open-source LLM gateway with GDPR/EU AI Act compliance hackernews · hn/story · 2026-06-06
• Show HN: AI Governance Architecture – DB-Governed, LLM-Agnostic, EU AI Act hackernews · hn/comment · 2026-03-05
• Open-source EU AI Act compliance middleware hackernews · hn/comment · 2026-02-09
• [Bug]: Compliance checks (GDPR / EU AI Act) report NON-COMPLIANT when guardrail_mode is a list (e.g. Presidio with outpu github · BerriAI/litellm · 2026-07-05

Related architectures

• Part of enterprise ai compliance architecture.
• For the related constraint, see EU AI Act vendor evaluation criteria for enterprise AI implementation.
• For the related constraint, see EU AI Act model release gate and golden set evaluation for enterprise AI.
• Where this interacts with another regime, see EU AI Act legacy integration patterns for enterprise AI agents.
• For the related constraint, see EU AI Act provider failover and degradation architecture for enterprise.
• If this is the blocker in front of you: enterprise AI infrastructure review.