PCI DSS 4.0 cost control architecture for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · PCI DSS 4.0 · layout walkthrough · implementation executed in CI before publish

The control that resolves this is a Pre-Flight Budget Governor, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is PCI DSS 4.0, and the question it actually asks is is cardholder data in scope, and can you prove it is not. The shape below holds p95 ≤ 900ms over durable queue (SQS / Kafka) with an outbox, and ships with a standard-library implementation that runs as written.

Where the rollout actually stalls

LLM spend is discovered in a monthly invoice and defended with a dashboard, so it is reported rather than controlled. Finance asks for a per-team cap and engineering cannot implement one, because nothing in the request path knows a budget exists.

What PCI DSS 4.0 specifically requires

PCI scope is contagious: any system that stores, processes or transmits cardholder data — or can affect the security of one that does — is in scope. The cheapest control is to keep the model tier out of scope entirely.

The artifact that satisfies it is a segmentation test proving the model tier is out of scope. The way teams most often fail it: a PAN reaching a prompt log, dragging the whole inference tier into PCI scope.

The parts you have to build

1. Conservative estimator — Deliberately over-estimates; under-estimating makes the ceiling decorative.
2. Two-phase budget — Reserve the estimate, settle the actual — closes the concurrency race.
3. Rate table in config — Prices change; a stale rate in code silently misprices every chargeback.
4. Tier router — Routes to the cheapest tier that meets the quality bar for the task.
5. Chargeback ledger — Per-tenant, per-request actuals — the artifact finance reconciles against.
6. Fail-closed default — A tenant with no budget is refused, never admitted.

Implementation artifact — Two-phase token budget with per-tenant chargeback

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind durable queue (SQS / Kafka) with an outbox; the failure mode it guards against is a consumer crash between side effect and acknowledgement. The output you hand the assessor is the segmentation test report.

"""Pre-flight token budget enforcement with per-tenant chargeback.

Why this shape: LLM spend is discovered monthly in an invoice and defended with
a dashboard, which means it is never actually controlled. The control has to run
BEFORE the call, on an estimate, with a hard ceiling per tenant and per period —
otherwise the first thing finance asks for is a cap you cannot implement.
"""
import math
import time
from dataclasses import dataclass, field


# USD per 1k tokens. Keep rates in config, not code: they change, and a stale
# rate silently misprices every chargeback downstream of it.
RATES = {
    "frontier":  {"in": 0.0030, "out": 0.0150},
    "mid":       {"in": 0.0008, "out": 0.0040},
    "small":     {"in": 0.0001, "out": 0.0004},
}


def estimate_tokens(text: str, chars_per_token: float = 3.8) -> int:
    """Deliberately a conservative OVER-estimate.

    Under-estimating defeats the purpose: the request gets admitted, the real
    usage exceeds the budget, and the ceiling is decorative."""
    return max(1, math.ceil(len(text or "") / chars_per_token))


def price(tier: str, tokens_in: int, tokens_out: int) -> float:
    r = RATES[tier]
    return (tokens_in / 1000.0) * r["in"] + (tokens_out / 1000.0) * r["out"]


class BudgetExceeded(Exception):
    def __init__(self, tenant, projected, ceiling):
        self.tenant, self.projected, self.ceiling = tenant, projected, ceiling
        super().__init__(f"{tenant}: ${projected:.4f} would exceed "
                         f"${ceiling:.2f} for the period")


@dataclass
class Budget:
    ceiling_usd: float
    period_seconds: float = 86400.0
    spent_usd: float = 0.0
    started: float = field(default_factory=time.time)
    committed: dict = field(default_factory=dict)   # request_id -> reserved USD

    def _roll(self, now):
        if now - self.started >= self.period_seconds:
            self.started, self.spent_usd = now, 0.0
            self.committed.clear()

    def reserve(self, request_id: str, projected_usd: float, now=None) -> float:
        """Two-phase: reserve the estimate, settle the actual.

        Without a reservation, N concurrent requests each see the same low
        `spent` and all get admitted — the classic race that blows a cap."""
        now = time.time() if now is None else now
        self._roll(now)
        outstanding = sum(self.committed.values())
        if self.spent_usd + outstanding + projected_usd > self.ceiling_usd:
            raise BudgetExceeded("tenant",
                                 self.spent_usd + outstanding + projected_usd,
                                 self.ceiling_usd)
        self.committed[request_id] = projected_usd
        return projected_usd

    def settle(self, request_id: str, actual_usd: float):
        self.committed.pop(request_id, None)
        self.spent_usd += actual_usd

    def release(self, request_id: str):
        self.committed.pop(request_id, None)


class Governor:
    def __init__(self):
        self.budgets: dict = {}
        self.ledger: list = []

    def set_budget(self, tenant: str, ceiling_usd: float, period_seconds=86400.0):
        self.budgets[tenant] = Budget(ceiling_usd, period_seconds)

    def admit(self, tenant: str, request_id: str, prompt: str,
              tier: str = "mid", max_output_tokens: int = 1024) -> dict:
        b = self.budgets.get(tenant)
        if b is None:
            raise KeyError(f"{tenant} has no budget: fail closed, never open")
        t_in = estimate_tokens(prompt)
        projected = price(tier, t_in, max_output_tokens)
        b.reserve(request_id, projected)
        return {"tenant": tenant, "request_id": request_id, "tier": tier,
                "tokens_in_est": t_in, "projected_usd": round(projected, 6)}

    def record(self, tenant: str, request_id: str, tier: str,
               tokens_in: int, tokens_out: int) -> dict:
        actual = price(tier, tokens_in, tokens_out)
        self.budgets[tenant].settle(request_id, actual)
        row = {"tenant": tenant, "request_id": request_id, "tier": tier,
               "tokens_in": tokens_in, "tokens_out": tokens_out,
               "actual_usd": round(actual, 6), "ts": time.time()}
        self.ledger.append(row)     # this is the chargeback record
        return row


if __name__ == "__main__":
    g = Governor()
    g.set_budget("bank-eu", ceiling_usd=0.05)
    adm = g.admit("bank-eu", "req-1", "Reconcile these 400 journal lines." * 20)
    print("admitted:", adm)
    print("settled :", g.record("bank-eu", "req-1", "mid", adm["tokens_in_est"], 310))
    try:
        for i in range(40):
            g.admit("bank-eu", f"burst-{i}", "x" * 4000, tier="frontier")
    except BudgetExceeded as e:
        print("ceiling held:", e)

The PCI DSS 4.0 control, as code

PAN tokenisation and a scope-reduction proof. This is the artifact an assessor asks for under PCI DSS 4.0 specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""PAN detection with tokenisation, and a scope-reduction proof.

PCI DSS 4.0 scope is everything that stores, processes or transmits account
data, plus anything connected to it. The cheapest compliance move is not better
controls — it is removing systems from scope. A model gateway that never sees a
PAN is out of scope; one that sees it once is in, along with everything it talks
to.
"""
import hashlib
import re

PAN_RE = re.compile(r"\b(?:\d[ -]?){12,18}\d\b")


def luhn_valid(digits: str) -> bool:
    total, parity = 0, len(digits) % 2
    for i, ch in enumerate(digits):
        d = int(ch)
        if i % 2 == parity:
            d *= 2
            if d > 9:
                d -= 9
        total += d
    return total % 10 == 0


class TokenVault:
    """Format-preserving surrogate. The vault is in scope; nothing it feeds is,
    provided the surrogate is not reversible outside it."""

    def __init__(self, pepper: str):
        self._pepper = pepper
        self._map: dict[str, str] = {}

    def tokenise(self, pan: str) -> str:
        digest = hashlib.sha256(f"{self._pepper}:{pan}".encode()).hexdigest()
        surrogate = "4" + str(int(digest[:15], 16)).zfill(15)[:15]
        self._map[surrogate] = pan
        return surrogate

    def last_four(self, pan: str) -> str:
        return pan[-4:]


class ScopeBoundary:
    def __init__(self):
        self.systems: dict[str, bool] = {}
        self.flows: list[tuple[str, str]] = []

    def add(self, system: str, sees_pan: bool) -> None:
        self.systems[system] = sees_pan

    def flow(self, src: str, dst: str) -> None:
        self.flows.append((src, dst))

    def in_scope(self) -> set[str]:
        scope = {s for s, sees in self.systems.items() if sees}
        changed = True
        while changed:                      # connected-to is also in scope
            changed = False
            for src, dst in self.flows:
                if src in scope and dst not in scope:
                    scope.add(dst); changed = True
                elif dst in scope and src not in scope:
                    scope.add(src); changed = True
        return scope


def redact(text: str, vault: TokenVault) -> tuple[str, int]:
    found = 0
    def _sub(m):
        nonlocal found
        raw = re.sub(r"[ -]", "", m.group(0))
        if not luhn_valid(raw):
            return m.group(0)
        found += 1
        return f"tok_{vault.tokenise(raw)}(last4={vault.last_four(raw)})"
    return PAN_RE.sub(_sub, text), found


if __name__ == "__main__":
    vault = TokenVault(pepper="hsm-managed")
    text = "Charge card 4111 1111 1111 1111 for invoice 9912, ref 1234567890123456"
    out, n = redact(text, vault)
    assert n >= 1 and "4111 1111 1111 1111" not in out, out
    assert "last4=1111" in out, out

    # A number that fails Luhn is not a PAN and must not be mangled.
    benign, n2 = redact("order 1234 5678 9012 3450 placed", vault)
    assert n2 == 0 and "1234 5678 9012 3450" in benign, benign

    sb = ScopeBoundary()
    sb.add("checkout", True); sb.add("token_vault", True)
    sb.add("llm_gateway", False); sb.add("analytics", False)
    sb.flow("checkout", "token_vault"); sb.flow("token_vault", "llm_gateway")
    scope = sb.in_scope()
    assert "llm_gateway" in scope, "connected-to is in scope — that is the point"

    clean = ScopeBoundary()
    clean.add("checkout", True); clean.add("token_vault", True)
    clean.add("llm_gateway", False)
    clean.flow("checkout", "token_vault")
    assert "llm_gateway" not in clean.in_scope(), clean.in_scope()

    print("PCI DSS 4.0 tokenisation + scope reduction: OK")
    print(f"  PANs tokenised {n}, naive scope {len(scope)} systems, "
          f"segmented scope {len(clean.in_scope())} systems")

Reference architecture

Pre-Flight Budget Governor — the request path, with the decision points that must exist before the first model call:

flowchart LR
    REQ[Request] --> EST[Conservative token estimate]
    EST --> PR[Price at tier rate]
    PR --> RES{Reserve against budget<br/>spent + outstanding + projected}
    RES -->|over ceiling| REJ[429 BudgetExceeded]
    RES -->|no budget| FC[Refuse - fail closed]
    RES -->|ok| TR[Tier router<br/>cheapest tier meeting quality bar]
    TR --> M[Model call]
    M --> ST[Settle actual usage]
    ST --> LED[(Chargeback ledger)]
    LED --> FIN[Finance reconciliation]

Technical parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

Risk and control matrix

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Review questions and answers

• Is the inference tier in PCI scope, and what segmentation test proves it is not?
  Cardholder data is tokenised before the model boundary and the segmentation is evidenced by an annual penetration test, so the tier stays out of scope by design.
• What refuses a request that would exceed this team's monthly cap, and where in the path does it run?
  The budget governor, at admission — before the model call. It prices a conservative token estimate against the tenant's remaining ceiling and refuses the request if the projection would exceed it.
• If 50 requests arrive at once against a nearly-exhausted budget, how many are admitted?
  Only those that fit. Admission counts outstanding reservations as well as settled spend, which closes the race where concurrent requests all read the same low figure and are all admitted.
• How does your chargeback figure reconcile to the provider invoice, and what is the variance?
  The chargeback ledger records per-request actuals priced from a versioned rate table with an effective date; reconcile it monthly and track the variance against the provider invoice.
• What happens to a tenant with no budget configured?
  It is refused. A missing budget fails closed — admitting an unbudgeted tenant is how the cap becomes decorative.
• Which rate table priced last month's chargeback, and when was it last updated?
  The rate table in versioned config, with its effective date recorded. A rate older than 90 days raises an alert, because a stale rate silently misprices every chargeback downstream.

Related architectures

• Part of enterprise llm cost control.
• A different failure mode of the same component: PCI DSS 4.0 model release gate and golden set evaluation for enterprise AI.
• This pattern also governs PCI DSS 4.0 retrieval architecture with citation integrity for enterprise.
• PCI DSS 4.0 provider failover and degradation architecture for enterprise applies the same mechanism under a different obligation.
• PCI DSS 4.0 data governance framework for enterprise LLM deployment applies the same mechanism under a different obligation.
• If this is the blocker in front of you: enterprise AI infrastructure review.