Decision deck · purchase-card fraud

Stopping the card,
not the transaction

Should the program deploy a fraud score, and how fast should it act on an alert?

Jay Yu · UCLA Anderson, MS in Business Analytics

Card transactions 2010 · evaluated on Nov–Dec, two months the model never saw1
Recommendation2

Score every purchase and block the card the same day an alert is confirmed: the model stops of fraud dollars for under a tenth of an analyst.

of fraud transactions land in the top 3% of scores on months the model never saw (Nov–Dec).

of

of fraud stopped with same-day blocking; next-day blocking stops .

alerts/day

at this program's volume, about minutes of analyst time a day.

Decision requested

Approve a 60-day shadow pilot: score live purchases, work alerts the same day, and go live if the top 3% of scores catch at least 45% of fraud.

Out-of-time replay, Nov 1 – Dec 31 2010 · model trained Jan 15 – Aug 31 · cutoff set on Sep – Oct · 10 min per reviewfraudscore decide
The problem3

Fraud is concentrated by card and front-loaded: of fraud dollars land on a card's first fraud day.

Compromised cards by number of fraud purchases, 2010

cards carry all fraud purchases, out of cards.

of fraud dollars sit on the ten costliest cards.

of purchases are fraud: rare per purchase, costly per card.

purchases, Jan–Dec 2010, after exclusionsfraudscore export
Approach4

Five steps from raw purchases to a staffing decision, every one judged on later months.

1

Clean

Purchases only; merchant, state and ZIP recovered without touching the label.

purchases kept
2

Describe behavior

Point-in-time counts, spend and bursts per card, merchant and pairing, 0–30 days back.

variables
3

Model on time order

Boosted trees tuned on two forward-in-time folds; Nov–Dec never used to choose.

settings compared
4

Replay card blocks

Nov–Dec purchase by purchase: alert, confirm, block, and count what the block stops.

compromised cards
5

Price the decision

Net value by review budget, analyst capacity and response speed, with every cost varied.

simulated outcomes

Everything is one reproducible pipeline: fraudscore run → tune → decide → export, with unit tests for every rule that could leak the future into the past.

github.com/J-A-Y-0/card-fraud-scoringfraudscore
How much to trust the number5

A random test split promised detection. On later months the same model found , so we tune and report on time order only.

test set claimed out-of-time, Nov–Dec FDR@3%, mean of 5 seeds

of the random split's test frauds sat on a card whose other frauds were in training. The model recognized the card, not the pattern.

What changed

Settings were chosen on two forward-in-time folds; Nov–Dec was held out from every choice, including the alert cutoff.

Boosted trees on 40 point-in-time variables · same out-of-time window for every splitfraudscore run
Performance on unseen months6

The top 3% of scores hold of Nov–Dec fraud, worth about twice the best rule an analyst could write.

Share of fraud found, by review budget

Sep–Oct validationNov–Dec out-of-time

Net value over Nov–Dec, next-day blocking

The rule is a card's spend over the last 3 days, the strongest single variable, tuned on the same months.

Model trained Jan 15 – Aug 31 · rule and model cutoffs both set on Sep – Octfraudscore decide
Where the value is7

Speed is the lever: with the same alerts, blocking the same day stops of fraud dollars instead of .

Every compromised card, Nov 1 – Dec 31, next-day blocking

loststoppedcard blocked

stopped when the card is blocked the next morning.

when alerts are worked and cards blocked the same day: more, at the cost of more good purchases declined.

with real-time declines, but good purchases declined. A later step, not the first.

Same model and cutoff for every policy · good purchases declined include the 5-day reissue lagfraudscore.value.replay
Rollout and risks8

Start with a 60-day shadow pilot whose go-live bar matches what 16 compromised cards can prove.

Days 1–60

Shadow

Score every purchase, work the top 3% the same day, log what a block would have stopped. Go live if the top 3% catches ≥ 45% of fraud.

From day 61

Same-day blocking

Block confirmed cards; keep 10% of cards scored but not blocked as the comparison group.

Once false declines are known

Real-time declines

Decline at the point of sale only for the highest scores.

Small samples

A 60-day pilot pins detection to ±. It can confirm the score works, not fine-tune it.

Value range

– net over two months (90%), never negative; driven by which cards get compromised.

Drift

Score stability is PSI today. Review above 0.10, retrain above 0.25 or quarterly.

Full report: Stopping the card · Model & decision report, §7–§98