GDPR provider failover and degradation architecture for enterprise LLM deployment — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · GDPR · layout diagram_first · implementation executed in CI before publish

The control that resolves this is a Labelled Degradation Ladder, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is GDPR, and the question it actually asks is what is the lawful basis, and can you actually erase. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

System architecture

Labelled Degradation Ladder — the request path, with the decision points that must exist before the first model call:

flowchart LR
    REQ[Request + min_quality] --> T1{Tier 1 breaker closed?}
    T1 -->|open| T2{Tier 2}
    T1 -->|closed| CALL1[Call tier 1]
    CALL1 -->|fails| BRK[Record failure<br/>maybe open breaker] --> T2
    T2 -->|below min_quality| REFUSE[Refuse: no acceptable tier]
    T2 -->|ok| CALL2[Call tier 2] --> LAB[Label response<br/>degraded=true]

Component breakdown

1. Quality-ranked tiers — Explicit integer quality per provider, not an implicit order.
2. Circuit breaker — A failing tier leaves rotation instead of being retried into the ground.
3. Half-open probe — One probe after cooldown; recovery is tested, not assumed.
4. Tier label on response — Every answer names its tier and whether it was degraded.
5. Quality floor per caller — A caller declares a minimum; below it is a refusal.
6. Degradation events — Each failover recorded — the signal the incident needs.

Implementation parameter matrix

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

The architecture friction point

When the primary provider degrades, traffic fails over to a weaker model and the answers keep flowing. Nobody downstream is told the quality changed, so the incident is discovered in complaints.

What GDPR specifically requires

GDPR asks two things an audit log does not answer: under what lawful basis was this processed, and can you erase it on request. Erasure is the harder one — a prompt embedded into a vector index and a fine-tune has propagated beyond the row you deleted.

The artifact that satisfies it is a lineage graph from source record to every derived artifact. The way teams most often fail it: an erasure request that cannot be honoured because the data survives in embeddings, caches and eval sets.

Working implementation — Degradation ladder with circuit breaker and quality floor

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the data-subject lineage and erasure report.

"""Degradation ladder with a circuit breaker and labelled quality tiers.

The failure: a provider degrades, traffic silently fails over to a weaker
model, and nobody downstream knows the answer came from a lower tier. The
control is that degradation is explicit and labelled on the response, so a
consumer can refuse a tier it cannot accept, and that a failing provider is
taken out of rotation rather than retried into the ground.
"""
import time


class ProviderDown(Exception):
    pass


class CircuitBreaker:
    def __init__(self, threshold: int = 3, cooldown_s: float = 30.0):
        self.threshold, self.cooldown_s = threshold, cooldown_s
        self.failures, self.opened_at = 0, None

    def record_failure(self, now: float) -> None:
        self.failures += 1
        if self.failures >= self.threshold and self.opened_at is None:
            self.opened_at = now

    def record_success(self) -> None:
        self.failures, self.opened_at = 0, None

    def closed(self, now: float) -> bool:
        if self.opened_at is None:
            return True
        if now - self.opened_at >= self.cooldown_s:
            self.opened_at, self.failures = None, 0   # half-open: allow a probe
            return True
        return False


class Tier:
    def __init__(self, name: str, quality: int, call):
        self.name, self.quality, self.call = name, quality, call
        self.breaker = CircuitBreaker()


class DegradationLadder:
    """Tiers in descending quality. The response always names the tier that
    produced it; a caller declaring min_quality gets a refusal rather than a
    quietly worse answer."""

    def __init__(self, tiers: list):
        self.tiers = sorted(tiers, key=lambda t: -t.quality)
        self.events: list[dict] = []

    def complete(self, prompt: str, min_quality: int = 0, now=None) -> dict:
        now = now if now is not None else time.monotonic()
        attempted = []
        for tier in self.tiers:
            if tier.quality < min_quality:
                continue
            if not tier.breaker.closed(now):
                attempted.append({"tier": tier.name, "skipped": "circuit open"})
                continue
            try:
                text = tier.call(prompt)
            except ProviderDown:
                tier.breaker.record_failure(now)
                attempted.append({"tier": tier.name, "failed": True})
                continue
            tier.breaker.record_success()
            degraded = tier is not self.tiers[0]
            self.events.append({"tier": tier.name, "degraded": degraded})
            return {"text": text, "tier": tier.name, "quality": tier.quality,
                    "degraded": degraded, "attempts": attempted}
        raise ProviderDown(
            f"no tier at or above quality {min_quality} available: {attempted}")


if __name__ == "__main__":
    def primary(p): raise ProviderDown("502 from primary")
    def secondary(p): return f"secondary:{p}"
    def local(p): return f"local-small:{p}"

    ladder = DegradationLadder([
        Tier("frontier", 100, primary),
        Tier("secondary", 70, secondary),
        Tier("local-small", 30, local)])

    r = ladder.complete("summarise", now=0.0)
    assert r["tier"] == "secondary" and r["degraded"] is True, r
    assert r["attempts"][0]["failed"], r

    # A caller that cannot accept degradation gets an error, not a worse answer.
    strict = DegradationLadder([Tier("frontier", 100, primary),
                                Tier("local-small", 30, local)])
    try:
        strict.complete("x", min_quality=90, now=0.0)
        raise AssertionError("served below the declared quality floor")
    except ProviderDown:
        pass

    # The breaker opens after repeated failures and stops the retry storm.
    t = Tier("flaky", 50, primary)
    for i in range(3):
        try:
            t.call("x")
        except ProviderDown:
            t.breaker.record_failure(now=float(i))
    assert not t.breaker.closed(now=3.0), "breaker should be open"
    assert t.breaker.closed(now=40.0), "breaker should half-open after cooldown"

    print("Degradation ladder + circuit breaker: OK")
    print(f"  served tier={r['tier']} degraded={r['degraded']}, "
          f"events {len(ladder.events)}")

The GDPR control, as code

Article 17 erasure propagated into derived artifacts. This is the artifact an assessor asks for under GDPR specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""Article 17 erasure that propagates into derived artifacts.

The erasure request everyone implements is a DELETE on the source row. The one
that fails an audit is the derived copy: embeddings in a vector index, a cached
retrieval chunk, a fine-tuning corpus, a prompt log. Article 17(2) expects
controllers to take reasonable steps to inform processors of the erasure, and a
vector built from personal data is still personal data.

So erasure is modelled as a lineage walk with a receipt per derived artifact,
and the request is not complete until every downstream store confirms.
"""
from datetime import datetime, timezone


class Store:
    def __init__(self, name: str, kind: str):
        self.name, self.kind = name, kind
        self.rows: dict[str, dict] = {}
        self.erased: list[str] = []

    def put(self, key: str, subject_id: str, payload: str) -> None:
        self.rows[key] = {"subject_id": subject_id, "payload": payload}

    def erase_subject(self, subject_id: str) -> int:
        doomed = [k for k, v in self.rows.items() if v["subject_id"] == subject_id]
        for k in doomed:
            del self.rows[k]
            self.erased.append(k)
        return len(doomed)

    def holds(self, subject_id: str) -> int:
        return sum(1 for v in self.rows.values() if v["subject_id"] == subject_id)


class ErasureOrchestrator:
    """Lineage-aware erasure. A store that is not registered cannot be erased,
    so registration is the control — an unregistered derived store is exactly
    how personal data survives a 'completed' erasure."""

    def __init__(self):
        self.stores: list[Store] = []
        self.lineage: dict[str, list[str]] = {}

    def register(self, store: Store, derived_from: str | None = None) -> Store:
        self.stores.append(store)
        if derived_from:
            self.lineage.setdefault(derived_from, []).append(store.name)
        return store

    def execute(self, subject_id: str) -> dict:
        receipts = []
        for store in self.stores:
            held = store.holds(subject_id)
            removed = store.erase_subject(subject_id)
            receipts.append({"store": store.name, "kind": store.kind,
                             "held": held, "erased": removed})
        residual = {s.name: s.holds(subject_id) for s in self.stores
                    if s.holds(subject_id)}
        return {
            "subject_id": subject_id,
            "completed_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
            "receipts": receipts,
            "residual": residual,
            "complete": not residual,
            "derived_stores_covered": sorted(
                s.name for s in self.stores if s.kind != "source"),
        }


if __name__ == "__main__":
    orch = ErasureOrchestrator()
    src = orch.register(Store("crm_postgres", "source"))
    vec = orch.register(Store("vector_index", "embedding"), derived_from="crm_postgres")
    cache = orch.register(Store("retrieval_cache", "cache"), derived_from="vector_index")
    logs = orch.register(Store("prompt_log", "log"), derived_from="crm_postgres")

    src.put("c-1", "subject-77", "Jana Soor, contract renewal notes")
    vec.put("v-1", "subject-77", "[0.11, -0.4, ...]")
    cache.put("k-1", "subject-77", "chunk: contract renewal notes")
    logs.put("l-1", "subject-77", "prompt containing the same notes")
    src.put("c-2", "subject-90", "unrelated subject")

    result = orch.execute("subject-77")
    assert result["complete"], result["residual"]
    assert len(result["derived_stores_covered"]) == 3, result
    assert sum(r["erased"] for r in result["receipts"]) == 4, result
    assert src.holds("subject-90") == 1, "erasure must not over-reach"

    # An unregistered derived store is the failure this design exists to expose.
    orphan = Store("analytics_extract", "warehouse")
    orphan.put("a-1", "subject-77", "same personal data, different copy")
    assert orphan.holds("subject-77") == 1
    assert orphan.name not in {s.name for s in orch.stores}

    print("GDPR Article 17 lineage erasure: OK")
    print(f"  stores erased {len(result['receipts'])}, complete={result['complete']}")
    print(f"  UNREGISTERED store still holding data: {orphan.name} "
          f"({orphan.holds('subject-77')} row) — registration is the control")

Controls and acceptance parameters

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Control questions before sign-off

• A data subject requests erasure. Trace exactly what gets deleted, including embeddings and cached completions.
  Every derived artifact carries the source record id, so erasure traverses the lineage graph. The completion report lists each store touched and the rows removed.
• When the primary provider fails, how does a downstream consumer learn the answer came from a weaker model?
  The response names the tier that produced it and carries a degraded flag. A consumer reading the label can treat a degraded answer differently; silent failover is what this design exists to prevent.
• What stops a failing provider from being retried thousands of times a minute?
  The circuit breaker. After the failure threshold it opens and the tier is skipped entirely until the cooldown elapses, so a failing provider leaves rotation rather than absorbing retries.
• Can a caller that requires top-tier quality be served a degraded answer?
  No. The caller declares a minimum quality and tiers below it are never attempted — the ladder raises instead of serving, which is the correct outcome when no acceptable tier exists.
• How do you know a recovered provider is actually healthy before restoring full traffic?
  The breaker goes half-open after cooldown and admits a single probe. Full rotation is restored only after that probe succeeds, so recovery is tested rather than assumed.
• What prevents failover from moving data outside the contracted region?
  Each tier carries a residency attribute and policy excludes out-of-region tiers from the ladder, so a failover cannot quietly relocate processing.

See also

• Part of enterprise ai compliance architecture.
• Where this interacts with another regime, see SOX provider failover and degradation architecture for enterprise LLM.
• This pattern also governs EU AI Act provider failover and degradation architecture for enterprise.
• PCI DSS 4.0 provider failover and degradation architecture for enterprise applies the same mechanism under a different obligation.
• HIPAA provider failover and degradation architecture for enterprise LLM applies the same mechanism under a different obligation.
• If this is the blocker in front of you: enterprise AI infrastructure review.