The same verifier, against a real fraud model.
Run on the real ULB MLG dataset (284,807 anonymized European card transactions, 0.17% fraud). The falsifier caught two real gaps: temporal drift and adversarial evasion.
XGBoost, the workhorse of production fraud detection, is 99.9% accurate on real transactions, yet its f1 still swings 1.59× across time. Gradient-guided perturbations evade 22.6% of the fraud the logistic model catches, and 8.4% transferred to a black-box XGBoost. The amount-evasion attack that fired on an earlier synthetic dataset doesn't fire on real data (0 of the fraud it caught was evaded). The verifier reports that honestly rather than manufacturing a catch.
And on IEEE-CIS (590,540 real e-commerce transactions), a model built on the naive interpretable signals (amount, card, address, email) is 94.4% accurate yet catches only 23.9% of fraud: the fraud is low-and-slow, structured to mimic legitimate behavior.
logistic regression 96.391% accurate evades 44/195 of caught fraud; the SAME adversarial samples transfer to a black-box XGBoost and evade 15/179 — mean applied L-inf perturbation 0.500
99.923% accurate; 0/179 of caught fraud evaded by halving amount
94.4% accurate yet catches only 23.9% of fraud — below the 96.5% 'always legitimate' baseline
f1 0.753 (recall 0.798) · f1 0.761 (recall 0.798) · f1 0.773 (recall 0.807) · f1 0.781 (recall 0.798)
f1 swings 0.538 to 0.857 across time segments (1.59x spread) — the validated number is a snapshot, not a property
5 signed fraud receipts, each Ed25519-signed over the record hash. Download any of them and re-verify with the public key.
open the receipts board: verify any of these in your browser →