SOC 2 Type II retrieval architecture with citation integrity for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · SOC 2 Type II · layout spec_first · implementation executed in CI before publish

Implementation parameter matrix

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

The control that resolves this is a Query-Time ACL Retrieval with Citation Integrity, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is SOC 2 Type II, and the question it actually asks is did this control operate effectively for the whole observation window. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

Why this blocks production

Permissions were applied when documents were indexed, the index is refreshed nightly, and citations are whatever the model names. So a revoked user still gets content, a superseded policy is quoted as current, and a citation points at a document that does not contain the claim.

What SOC 2 Type II specifically requires

SOC 2 Type II is not a point-in-time assertion. The auditor samples evidence across a 3-12 month observation window and asks whether the control operated on every day of it, not whether it exists today.

The artifact that satisfies it is a sampled, time-ordered population the auditor can draw from. The way teams most often fail it: the control was added mid-window, so the population has a hole and the observation period restarts.

Implementation artifact — Query-time ACL retrieval with citation containment

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the observation-window evidence export.

"""Retrieval with query-time ACLs, freshness SLO and citation integrity.

Three failures, one surface. Permissions applied at INDEX time go stale the
moment access changes, so a revoked user keeps seeing content through the
model. A stale index answers confidently from a superseded document. And a
citation that does not actually contain the claim is worse than no citation,
because it manufactures confidence.
"""
from dataclasses import dataclass, field


@dataclass
class Doc:
    doc_id: str
    text: str
    version: int
    indexed_at: int
    acl: set = field(default_factory=set)
    superseded_by: str | None = None


class Index:
    def __init__(self, freshness_sue_s: int = 3600):
        self.docs: dict[str, Doc] = {}
        self.freshness_sue_s = freshness_sue_s

    def put(self, doc: Doc) -> None:
        self.docs[doc.doc_id] = doc

    def stale(self, now: int) -> list[str]:
        return sorted(d.doc_id for d in self.docs.values()
                      if now - d.indexed_at > self.freshness_sue_s)

    def search(self, term: str, user_acl: set, now: int) -> list[dict]:
        """ACL is applied HERE, at query time, against live entitlements —
        not baked into the index where it goes stale on the first revocation."""
        hits = []
        for d in self.docs.values():
            if term.lower() not in d.text.lower():
                continue
            if d.superseded_by:
                continue
            if d.acl and not (d.acl & user_acl):
                continue
            hits.append({"doc_id": d.doc_id, "version": d.version,
                         "text": d.text,
                         "stale": now - d.indexed_at > self.freshness_sue_s})
        return sorted(hits, key=lambda h: h["doc_id"])


def verify_citations(answer: str, citations: list, index: Index) -> dict:
    """A citation must contain a claim-bearing span of the answer. This is a
    containment check, not a similarity score — a loose threshold here is how
    plausible-but-unsupported citations get through."""
    supported, unsupported = [], []
    sentences = [s.strip() for s in answer.split(".") if s.strip()]
    for cite in citations:
        doc = index.docs.get(cite)
        if not doc:
            unsupported.append({"cite": cite, "reason": "cited doc not in index"})
            continue
        if any(s.lower() in doc.text.lower() for s in sentences):
            supported.append(cite)
        else:
            unsupported.append({"cite": cite,
                                "reason": "no cited span appears in the document"})
    return {"supported": supported, "unsupported": unsupported,
            "trustworthy": not unsupported}


if __name__ == "__main__":
    idx = Index(freshness_sue_s=3600)
    idx.put(Doc("d1", "Refunds are processed within 14 days", 7,
                indexed_at=0, acl={"finance"}))
    idx.put(Doc("d2", "Refunds are processed within 30 days", 3,
                indexed_at=0, acl={"finance"}, superseded_by="d1"))
    idx.put(Doc("d3", "Public holiday schedule", 1, indexed_at=0))

    # Entitlement is checked at query time.
    allowed = idx.search("refunds", user_acl={"finance"}, now=10)
    assert [h["doc_id"] for h in allowed] == ["d1"], allowed
    denied = idx.search("refunds", user_acl={"marketing"}, now=10)
    assert denied == [], denied

    # Revocation takes effect immediately, with no reindex.
    idx.docs["d1"].acl = {"finance-restricted"}
    assert idx.search("refunds", user_acl={"finance"}, now=10) == []

    # Superseded documents never surface.
    assert all(h["doc_id"] != "d2" for h in
               idx.search("refunds", user_acl={"finance-restricted"}, now=10))

    # Staleness is reported rather than silently served as current.
    assert idx.stale(now=10) == []
    assert set(idx.stale(now=10_000)) == {"d1", "d2", "d3"}
    hits = idx.search("refunds", user_acl={"finance-restricted"}, now=10_000)
    assert hits and hits[0]["stale"] is True, hits

    good = verify_citations("Refunds are processed within 14 days.", ["d1"], idx)
    assert good["trustworthy"], good
    bad = verify_citations("Refunds are processed within 2 days.", ["d1"], idx)
    assert not bad["trustworthy"], bad
    missing = verify_citations("Anything.", ["d99"], idx)
    assert missing["unsupported"][0]["reason"] == "cited doc not in index"

    print("Query-time ACL retrieval + citation integrity: OK")
    print(f"  stale docs at t=10000: {len(idx.stale(10_000))}, "
          f"citation trustworthy good={good['trustworthy']} bad={bad['trustworthy']}")

The SOC 2 Type II control, as code

Type II evidence sampling over the observation window. This is the artifact an assessor asks for under SOC 2 Type II specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""SOC 2 Type II evidence sampling over an observation window.

Type II is not "the control exists" — it is "the control operated throughout
the period". An auditor samples days from the window and asks for proof the
control ran on each sampled day. A gap of one day with no evidence is an
exception, so the population must be built from operational records rather
than asserted.
"""
import hashlib
from datetime import date, timedelta


class ObservationWindow:
    """The population an auditor samples from, plus deterministic selection."""

    def __init__(self, start: date, end: date, control_id: str):
        if end < start:
            raise ValueError("window end precedes start")
        self.start, self.end, self.control_id = start, end, control_id
        self._evidence: dict[date, list[str]] = {}

    def record(self, day: date, artifact_ref: str) -> None:
        if not (self.start <= day <= self.end):
            raise ValueError(f"{day} outside observation window")
        self._evidence.setdefault(day, []).append(artifact_ref)

    def population(self) -> list[date]:
        n = (self.end - self.start).days + 1
        return [self.start + timedelta(days=i) for i in range(n)]

    def gaps(self) -> list[date]:
        """Days with no evidence. Each one is a Type II exception."""
        return [d for d in self.population() if not self._evidence.get(d)]

    def sample(self, size: int) -> list[date]:
        """Deterministic, reproducible selection — an auditor must be able to
        re-derive the same sample from the same seed rather than trust ours."""
        pop = self.population()
        size = min(size, len(pop))
        seed = f"{self.control_id}:{self.start}:{self.end}"
        scored = sorted(
            pop, key=lambda d: hashlib.sha256(f"{seed}:{d}".encode()).hexdigest())
        return sorted(scored[:size])

    def evidence_for(self, day: date) -> list[str]:
        return list(self._evidence.get(day, []))

    def attestation(self, size: int = 25) -> dict:
        sampled = self.sample(size)
        missing = [d for d in sampled if not self._evidence.get(d)]
        return {
            "control_id": self.control_id,
            "window": f"{self.start}..{self.end}",
            "population_days": len(self.population()),
            "sampled_days": len(sampled),
            "exceptions": [str(d) for d in missing],
            "opinion": "unqualified" if not missing else "qualified",
        }


if __name__ == "__main__":
    start, end = date(2026, 1, 1), date(2026, 3, 31)
    w = ObservationWindow(start, end, "CC6.1-tenant-isolation")
    for d in w.population():
        w.record(d, f"s3://evidence/{d}/isolation-check.json")
    clean = w.attestation()
    assert clean["opinion"] == "unqualified", clean
    assert clean["population_days"] == 90, clean

    # One missing day is one exception — and it must survive sampling to matter.
    w2 = ObservationWindow(start, end, "CC6.1-tenant-isolation")
    sampled = set(w2.sample(25))
    for d in w2.population():
        if d != sorted(sampled)[0]:
            w2.record(d, "ok")
    assert w2.attestation()["opinion"] == "qualified", w2.attestation()
    assert len(w2.gaps()) == 1, w2.gaps()

    # The sample must be reproducible from the same seed.
    assert w.sample(25) == ObservationWindow(start, end,
                                             "CC6.1-tenant-isolation").sample(25)
    print("SOC 2 Type II evidence sampling: OK")
    print(f"  population {clean['population_days']} days, "
          f"sample {clean['sampled_days']}, opinion {clean['opinion']}")

System architecture

Query-Time ACL Retrieval with Citation Integrity — the request path, with the decision points that must exist before the first model call:

flowchart LR
    Q[Query + live entitlements] --> SRCH[Search index]
    SRCH --> ACL{ACL match at query time?}
    ACL -->|no| DROP[Drop silently from results]
    ACL -->|yes| SUP{Superseded?}
    SUP -->|yes| DROP
    SUP -->|no| FRESH[Attach version + staleness]
    FRESH --> ANS[Answer + citations]
    ANS --> VER{Cited doc contains the claim?}
    VER -->|no| STRIP[Refuse the citation]
    VER -->|yes| OUT[Answer with verified citations]

Implementation components

1. Query-time ACL — Entitlements evaluated per query against live rights, never baked in.
2. Supersession links — A replaced document is unreachable, not merely lower-ranked.
3. Freshness SLO — Index age per document, with staleness reported on the hit.
4. Citation containment — A cited document must contain a claim-bearing span of the answer.
5. Provenance on the hit — Document id and version travel with the retrieved text.
6. Revocation path — An entitlement change takes effect without a reindex.

Risk register with acceptance criteria

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

What the auditor will ask

• Show me the control-execution population for the full window, and the days where it is empty.
  Every request emits an execution record, so the population is continuous by construction. A gap is an alertable incident, not something discovered during the audit.
• A user's access is revoked at 09:00. When does the model stop retrieving that content for them?
  On their next query. Entitlements are evaluated at query time against live rights, so revocation does not wait for a reindex — which is the whole reason index-time ACLs fail.
• What stops a superseded policy document being quoted as current?
  Supersession links. A replaced document is unreachable in search rather than ranked lower, so it cannot surface as a near-miss when the replacement is less similar.
• How do you know a citation actually supports the sentence it is attached to?
  A containment check: a claim-bearing span of the answer must appear in the cited document. A citation that fails is refused, because an unsupported citation manufactures confidence.
• How does a reader know whether the retrieved content is current?
  Every hit carries its document version and a staleness flag derived from the freshness SLO, so content older than the agreed window is visibly old rather than silently current.
• Can a restricted document influence results through similarity without being shown?
  No. Restricted documents are filtered during the scan, before ranking, so they never contribute to the candidate set a user may not read.

Related reference architectures

• Part of enterprise ai compliance architecture.
• The same control appears in SOC 2 Type II agent permission and delegated access architecture.
• SOC 2 Type II model release gate and golden set evaluation for enterprise applies the same mechanism under a different obligation.
• A different failure mode of the same component: SOC 2 Type II vendor evaluation criteria for enterprise AI implementation.
• The same control appears in SOC 2 Type II reference architecture for enterprise LLM deployment.
• If this is the blocker in front of you: enterprise AI infrastructure review.