GDPR vendor evaluation criteria for enterprise AI implementation — Reference Architecture

Reference architecture · published 2026-10-16 · regulated enterprise · GDPR · layout control_review · implementation executed in CI before publish

The control that resolves this is a Gated Vendor Evaluation Framework, evaluated before the request reaches a model provider rather than audited afterwards. In regulated enterprise the binding constraint is GDPR, and the question it actually asks is what is the lawful basis, and can you actually erase. The specific problem here is RFP and structured evaluation. The shape below holds p95 ≤ 600ms over REST over HTTPS, and ships with a standard-library implementation that runs as written.

The architecture friction point

A vendor comparison built as a weighted average lets a supplier average its way past a control it does not have. The scorecard ranks everyone, the committee cannot see which gates were failed, and the decision gets re-litigated after the contract is signed.

What makes RFP and structured evaluation specifically hard

An RFP scored as a weighted average lets a vendor average its way past a control it does not have. The scoring model, not the questions, is usually what makes the evaluation indefensible.

What resolves it is a two-layer scorecard: pass/fail gates, then weights. The failure to design against: the highest-scoring vendor failing security review after award.

What GDPR specifically requires

GDPR asks two things an audit log does not answer: under what lawful basis was this processed, and can you erase it on request. Erasure is the harder one — a prompt embedded into a vector index and a fine-tune has propagated beyond the row you deleted.

The artifact that satisfies it is a lineage graph from source record to every derived artifact. The way teams most often fail it: an erasure request that cannot be honoured because the data survives in embeddings, caches and eval sets.

Risk and control matrix

Each row carries an acceptance parameter rather than an adjective, because "low latency" cannot be signed off. Retention here is bound by per the data class retention schedule, enforced automatically.

Deployment parameters

Every value below is a parameter you can hold a vendor or an internal team to. Adjectives are not acceptance criteria.

Questions to answer before approval

• A data subject requests erasure. Trace exactly what gets deleted, including embeddings and cached completions.
  Every derived artifact carries the source record id, so erasure traverses the lineage graph. The completion report lists each store touched and the rows removed.
• Which criteria are pass/fail, and which vendor failed one?
  Gates are evaluated first and independently of score: SOC 2 Type 2 currency, DPA and subprocessor disclosure, tenant isolation evidence, residency. A failure on any of them disqualifies.
• Which criteria are pass/fail, and which vendor failed one?
  The gate layer is pass/fail: SOC 2 Type 2 currency, DPA and subprocessor disclosure, tenant isolation evidence, and data residency. Gates are evaluated before any weighted scoring, so a vendor cannot average past a control it lacks.
• For each passed gate, what artifact proves it and what is its date?
  Each gate names the artifact that satisfies it — the report, the signed DPA, the architecture document — with its date. A claim is not evidence.
• Is the latency target in the signed agreement with a remedy, or on a status page?
  It must be in the signed agreement with a remedy. A status page is not a commitment; the scorecard scores the remedy, not the promise.
• What does exit look like — specifically, what can you export and has it been rehearsed?
  Export of prompts, embeddings, eval sets and fine-tunes, rehearsed during the pilot rather than promised in the contract. Portability that has never been executed is not portability.
• Was the reference deployment in the same vertical and regulatory regime as ours?
  It must be — a reference in the same vertical and the same regulatory regime, worked through a structured question set, not a vendor-selected happy customer.

System architecture

Gated Vendor Evaluation Framework — the request path, with the decision points that must exist before the first model call:

flowchart TB
    V[Vendor responses] --> G{Gates - pass/fail}
    G -->|any gate failed| DQ[Disqualified<br/>with named gates]
    G -->|all gates passed| W[Weighted scoring]
    W --> TCO[3-year fully loaded TCO]
    W --> REF[Reference call<br/>comparable regulated deployment]
    TCO --> R[Ranking]
    REF --> R
    R --> DR[Decision record<br/>score + reason per criterion]
    DQ --> DR
    DR --> SC[Steering committee]

Component breakdown

1. Gate layer — Pass/fail controls: SOC 2 Type 2 currency, DPA, tenant isolation, residency.
2. Weighted layer — Ranks only vendors that cleared every gate.
3. Evidence requirement — Each gate names the artifact that satisfies it, not a claim.
4. TCO model — Fully loaded three-year: licence, run cost, internal effort.
5. Reference protocol — A structured call with a comparable regulated deployment.
6. Decision record — Ranking plus failed gates plus a written reason per score.

Working implementation — Weighted scorecard with disqualifying gates

Standard-library Python, no external dependencies, runs as written. Executed by the publish gate: if it raises, this page does not ship. Wire it behind REST over HTTPS; the failure mode it guards against is a retried POST creating a duplicate record. The output you hand the assessor is the data-subject lineage and erasure report.

"""Weighted vendor scorecard with disqualifying gates.

Why this shape: a procurement scorecard that is a weighted average lets a vendor
average its way past a control it simply does not have. Real evaluations have
two layers — GATES that are pass/fail, and weights that only ever rank the
vendors that already cleared every gate.

This produces the artifact a steering committee signs: a ranking, the gates each
vendor failed, and a written reason per score.
"""
from dataclasses import dataclass, field


@dataclass(frozen=True)
class Criterion:
    key: str
    weight: float          # relative; normalised at scoring time
    gate: bool = False     # True -> pass/fail, cannot be averaged away
    note: str = ""


DEFAULT_CRITERIA = [
    Criterion("soc2_type2_current", 0.0, gate=True,
              note="Type 2 report inside 12 months, not a Type 1 or a roadmap"),
    Criterion("dpa_and_subprocessors", 0.0, gate=True,
              note="DPA signable as-is and a disclosed subprocessor list"),
    Criterion("tenant_isolation_evidence", 0.0, gate=True,
              note="Architectural isolation, not a WHERE clause"),
    Criterion("data_residency", 0.0, gate=True,
              note="Processing stays in the contracted region"),
    Criterion("exit_and_portability", 2.0,
              note="Export of prompts, embeddings, evals; no model lock-in"),
    Criterion("latency_slo_contractual", 3.0,
              note="p95 in the agreement with a remedy, not a status page"),
    Criterion("integration_depth", 3.0,
              note="Works against the real system of record, not a demo schema"),
    Criterion("implementation_capacity", 2.0,
              note="Named engineers, not a bench claim"),
    Criterion("tco_3yr", 3.0, note="Fully loaded: licence, run, internal effort"),
    Criterion("reference_in_our_vertical", 2.0,
              note="A reference call with a comparable regulated deployment"),
]


class Disqualified(Exception):
    pass


@dataclass
class Vendor:
    name: str
    gates: dict = field(default_factory=dict)    # key -> bool
    scores: dict = field(default_factory=dict)   # key -> 0..5
    reasons: dict = field(default_factory=dict)


def evaluate(vendors, criteria=None):
    """-> (ranked, disqualified). Gates are evaluated first, always."""
    criteria = criteria or DEFAULT_CRITERIA
    gate_keys = [c.key for c in criteria if c.gate]
    weighted = [c for c in criteria if not c.gate and c.weight > 0]
    total_w = sum(c.weight for c in weighted) or 1.0

    ranked, disqualified = [], []
    for v in vendors:
        failed = [k for k in gate_keys if not v.gates.get(k, False)]
        if failed:
            disqualified.append({"vendor": v.name, "failed_gates": failed})
            continue
        got = 0.0
        breakdown = []
        for c in weighted:
            s = float(v.scores.get(c.key, 0))
            if not 0 <= s <= 5:
                raise ValueError(f"{v.name}.{c.key}={s} outside 0..5")
            contribution = (c.weight / total_w) * (s / 5.0)
            got += contribution
            breakdown.append({"criterion": c.key, "score": s,
                              "weight": c.weight,
                              "contribution": round(contribution, 4),
                              "reason": v.reasons.get(c.key, ""),
                              "standard": c.note})
        ranked.append({"vendor": v.name, "score": round(got * 100, 1),
                       "breakdown": breakdown})
    ranked.sort(key=lambda r: -r["score"])
    return ranked, disqualified


if __name__ == "__main__":
    passing = {k: True for k in
               ("soc2_type2_current", "dpa_and_subprocessors",
                "tenant_isolation_evidence", "data_residency")}
    a = Vendor("Vendor A", gates=dict(passing),
               scores={"exit_and_portability": 4, "latency_slo_contractual": 5,
                       "integration_depth": 4, "implementation_capacity": 3,
                       "tco_3yr": 3, "reference_in_our_vertical": 5},
               reasons={"latency_slo_contractual": "p95 450ms with 10% credit"})
    b = Vendor("Vendor B", gates={**passing, "tenant_isolation_evidence": False},
               scores={k: 5 for k in
                       ("exit_and_portability", "latency_slo_contractual",
                        "integration_depth", "implementation_capacity",
                        "tco_3yr", "reference_in_our_vertical")})
    ranked, dq = evaluate([a, b])
    for r in ranked:
        print(f"{r['vendor']}: {r['score']}")
    print("disqualified despite perfect scores:", dq)

The GDPR control, as code

Article 17 erasure propagated into derived artifacts. This is the artifact an assessor asks for under GDPR specifically — it is not the same control as the one above, and a system that implements one does not thereby implement the other. It executes under the same publish gate.

"""Article 17 erasure that propagates into derived artifacts.

The erasure request everyone implements is a DELETE on the source row. The one
that fails an audit is the derived copy: embeddings in a vector index, a cached
retrieval chunk, a fine-tuning corpus, a prompt log. Article 17(2) expects
controllers to take reasonable steps to inform processors of the erasure, and a
vector built from personal data is still personal data.

So erasure is modelled as a lineage walk with a receipt per derived artifact,
and the request is not complete until every downstream store confirms.
"""
from datetime import datetime, timezone


class Store:
    def __init__(self, name: str, kind: str):
        self.name, self.kind = name, kind
        self.rows: dict[str, dict] = {}
        self.erased: list[str] = []

    def put(self, key: str, subject_id: str, payload: str) -> None:
        self.rows[key] = {"subject_id": subject_id, "payload": payload}

    def erase_subject(self, subject_id: str) -> int:
        doomed = [k for k, v in self.rows.items() if v["subject_id"] == subject_id]
        for k in doomed:
            del self.rows[k]
            self.erased.append(k)
        return len(doomed)

    def holds(self, subject_id: str) -> int:
        return sum(1 for v in self.rows.values() if v["subject_id"] == subject_id)


class ErasureOrchestrator:
    """Lineage-aware erasure. A store that is not registered cannot be erased,
    so registration is the control — an unregistered derived store is exactly
    how personal data survives a 'completed' erasure."""

    def __init__(self):
        self.stores: list[Store] = []
        self.lineage: dict[str, list[str]] = {}

    def register(self, store: Store, derived_from: str | None = None) -> Store:
        self.stores.append(store)
        if derived_from:
            self.lineage.setdefault(derived_from, []).append(store.name)
        return store

    def execute(self, subject_id: str) -> dict:
        receipts = []
        for store in self.stores:
            held = store.holds(subject_id)
            removed = store.erase_subject(subject_id)
            receipts.append({"store": store.name, "kind": store.kind,
                             "held": held, "erased": removed})
        residual = {s.name: s.holds(subject_id) for s in self.stores
                    if s.holds(subject_id)}
        return {
            "subject_id": subject_id,
            "completed_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
            "receipts": receipts,
            "residual": residual,
            "complete": not residual,
            "derived_stores_covered": sorted(
                s.name for s in self.stores if s.kind != "source"),
        }


if __name__ == "__main__":
    orch = ErasureOrchestrator()
    src = orch.register(Store("crm_postgres", "source"))
    vec = orch.register(Store("vector_index", "embedding"), derived_from="crm_postgres")
    cache = orch.register(Store("retrieval_cache", "cache"), derived_from="vector_index")
    logs = orch.register(Store("prompt_log", "log"), derived_from="crm_postgres")

    src.put("c-1", "subject-77", "Jana Soor, contract renewal notes")
    vec.put("v-1", "subject-77", "[0.11, -0.4, ...]")
    cache.put("k-1", "subject-77", "chunk: contract renewal notes")
    logs.put("l-1", "subject-77", "prompt containing the same notes")
    src.put("c-2", "subject-90", "unrelated subject")

    result = orch.execute("subject-77")
    assert result["complete"], result["residual"]
    assert len(result["derived_stores_covered"]) == 3, result
    assert sum(r["erased"] for r in result["receipts"]) == 4, result
    assert src.holds("subject-90") == 1, "erasure must not over-reach"

    # An unregistered derived store is the failure this design exists to expose.
    orphan = Store("analytics_extract", "warehouse")
    orphan.put("a-1", "subject-77", "same personal data, different copy")
    assert orphan.holds("subject-77") == 1
    assert orphan.name not in {s.name for s in orch.stores}

    print("GDPR Article 17 lineage erasure: OK")
    print(f"  stores erased {len(result['receipts'])}, complete={result['complete']}")
    print(f"  UNREGISTERED store still holding data: {orphan.name} "
          f"({orphan.holds('subject-77')} row) — registration is the control")

Related architectures

• Part of enterprise ai vendor evaluation.
• The same control appears in PCI DSS 4.0 vendor evaluation criteria for enterprise AI implementation.
• For the related constraint, see SOX vendor evaluation criteria for enterprise AI implementation.
• This pattern also governs ISO/IEC 27001:2022 vendor evaluation criteria for enterprise AI.
• A different failure mode of the same component: HIPAA vendor evaluation criteria for enterprise AI implementation.
• If this is the blocker in front of you: enterprise AI infrastructure review.