How our data is built

Everything here is exactly how the service works in production — no black box. PROOViD AML screens names against sanctions and PEP data; this page shows where that data comes from, how it is refreshed, how PEP levels are worked out, and — honestly — what the free bundled data does and does not guarantee. For the deeper technical diagrams see Architecture; for live source counts see Coverage & sources.

1. Where the data comes from

Two tiers. The bundled tier ships with the product; the bring-your-own (BYO) tier is a commercial feed a customer plugs in with their own key for guaranteed, exhaustive coverage.

flowchart LR
  subgraph Free["Bundled — free, daily refresh"]
    S1["UN / OFAC / EU / UK
sanctions (official gov lists)"]
    S2["Wikidata PEPs (CC0)
leaders, ministers, ambassadors,
central bankers, legislators + families"]
    S3["Adverse media
(GDELT + NER filter)"]
  end
  subgraph Byo["Bring-your-own — commercial, guaranteed"]
    S4["OpenSanctions / World-Check /
Dow Jones … (customer key)"]
  end
  Free --> ENG["PROOViD engine
(one normalized model)"]
  Byo --> ENG
      

2. The ingest pipeline (self-healing)

Each daily refresh runs these stages. The last stage is the important one: we accumulate — we never wipe the list and rebuild it.

flowchart TB
  A["Phase A — resolve the political offices
(head of state, gov, ministers, ambassadors,
central bankers, legislators)
by hierarchy AND by direct example"]
  A2["Phase A2 — office metadata
(jurisdiction country + IGO flag)"]
  L["Current leaders — sitting holders only
(no end date, alive, recent start)"]
  ACCL["+ accumulated leaders from earlier runs
(kept by self-healing)"]
  P["Rank by prominence
(Wikipedia reach — most-notable first)"]
  R["Relatives / RCA — leaders' families → PEP4"]
  B["Bulk — the broad PEP set (best-effort)"]
  ACC["Self-healing merge
add / refresh, stamp 'last seen',
KEEP anything missing this run"]
  PR["Age-out — prune only what's been
unseen for 30 days"]
  A --> A2 --> L --> P --> R --> B --> ACC --> PR
  ACCL --> P
      

Why "self-healing" matters

Country-anchored leaders index

The most robust safeguard doesn't rely on how the public source organises offices at all. We start from the fixed, closed list of the world's ~200 sovereign countries and follow each country's own link to its head-of-state and head-of-government office. Because we start from the country, no country's leader can be missed by a data-structure quirk — and it's self-checking: one row per country, so a gap is obvious. We capture every holder with the dates they were in office (current and historical) plus their families, and keep the link to the source record on every row.

flowchart TB
  C["~200 sovereign countries
(fixed, closed list)"]
  O["each country's head-of-state
+ head-of-government office"]
  H["every holder + term dates
(current AND historical)"]
  F["families / associates → PEP4"]
  M["merge into self-healing accumulate"]
  C --> O --> H --> F --> M
      

3. How PEP levels are worked out

A PEP's level is derived from the public office they hold, graded against your tenant's home country — it is not a flat label. Foreign-vs-domestic is inherently relative to you, so it is computed at screen time.

flowchart TB
  O["Office the person holds
(from Wikidata / OpenSanctions)"]
  O --> Q1{"Family / associate
relationship?"}
  Q1 -- yes --> P4["PEP4 — Family / associate (RCA)"]
  Q1 -- no --> Q2{"Intergovernmental
org role?"}
  Q2 -- yes --> P3["PEP3 — International-org"]
  Q2 -- no --> Q3{"Office country ==
your home country?"}
  Q3 -- yes --> P1["PEP1 — Domestic"]
  Q3 -- no --> P2["PEP2 — Foreign"]
  Q3 -- "unknown / home not set" --> PD["Documented PEP1 default"]
      

Set your home country on the risk profile to activate Domestic/Foreign grading, and use coverage presets (or the per-screen pepTiers filter) to choose which levels count. See the PEP-tier column on Coverage.

4. What happens when you screen a name

flowchart LR
  N["Name (+ DoB / nationality / aliases)"] --> C["Candidate retrieval
(trigram + phonetic, tenant lists,
optional freshness window)"]
  C --> SC["Fuzzy + phonetic + nickname scoring
(+ DoB confirmation)"]
  SC --> CL["Classify — TP / potential / false-positive"]
  CL --> TR["Derive PEP tier + adverse-media category"]
  TR --> D["Decision — Pass / Review / Fail
(+ 0–100 risk score, per-tenant policy)"]
  D --> CASE["Case created + explainable factors"]
      

5. Refresh & data freshness

• Cadence: every source refreshes on a ~24-hour cycle; the self-healing merge means each refresh improves coverage rather than risking it.
• Last-seen: every entity carries the date it was last confirmed in a source refresh.
• Per-tenant freshness filter: a tenant can require "only match data seen in the last N days" — the shared dataset can't be pruned per-tenant, so this is your screen-time freshness knob (maxDataAgeDays). A conservative global age-out keeps the shared set clean regardless.

6. The honest boundary — free vs guaranteed