Testing
Tested on scams it has never seen.
Anyone can score well on scams they practised on. We test on fresh real scams, blind, and publish every result, including the bad rounds.
Where it stands today
Two numbers matter. How much it catches of everything it has ever been tested on, and how much it caught the first time it saw each batch, before we fixed anything.
Latest batch, round 17 (2026-10-02): 541 pages, 743 ads and 1,700 comments and threads. Blind, it caught 134 of 186 dangerous scams (72%) and 71 of 113 social scams (63%). After the fixes that round led to: 183 of 187 (98%) and 116 of 122 (95%).
17 rounds so far, 22,404 real items labelled, 27,878 test cases in all including 2,252 holdouts we never tune on. Labellers agreed on 95.7% to 98.7% of items per round. Every number on this page is generated from the testing ledger in the code, last updated 2026-10-03.
What we are testing, and how
Any detector looks good on scams it practised on. The real test is fresh scams it has never seen.
1. A fresh pile, every round
Real things you might run into this week, never seen before: new phishing pages, wallet drainers, scam ads, comments under money videos, and ordinary popular sites. A new pile every round.
2. Labelled blind, by two
Two people who never see our rules mark each item scam, ordinary or grey. Only items they agree on count. We check both: scams stamped, ordinary pages left alone.
3. Scored once, blind
We run the extension before changing anything. That's what you would have seen that week. It's measured once, never redone.
4. Fixed, then re-scored
We improve the rulebooks, score the same pile again and check no earlier round got worse. That's the "now" number.
Two kinds of scam
Dangerous: pages that take your password, card or wallet. Social: the comments, bot threads and ads that lead you there. "24 of 42" means 24 of 42 scams were stamped.
False alarms and small rounds
A false alarm is an ordinary page that got stamped. Our limit is 0.5%. Small rounds swing a lot, so trust the totals at the top. 896 repeats of earlier rounds are left out.
How we test
Both ways, every time
Each kind of scam is tested against real scams and the real pages they copy. Catching scams is only half the job; leaving real pages alone is the other half.
Holdouts we never tune on
Some cases are locked away. If one is ever used to fix something, it moves into the practice set and a fresh one takes its place.
Blind real-world rounds
A fresh batch of real items nobody has seen. Two labellers mark each one scam or real without seeing our rules; only items they agree on count (96 to 99% agreement). Scored once, then used to improve.
The latest rounds
Blind is the first time it saw the pile. Now is the same pile with today's rulebooks. Open a round for what was in it, where it came from, false alarms and what made it hard.
| Round | Dangerous scams caught | Social scams caught | |
|---|---|---|---|
| 172026-10-02Latest | blind72%134 of 186now98%183 of 187 | blind63%71 of 113now95%116 of 122 | |
A fresh pile of 2,984 real items: 541 web pages, 743 ads and 1,700 comments and threads. The labellers agreed that 299 were scams, 344 were grey and 2,305 were ordinary; 16 repeats of an earlier round were left out. Where it came from: 1,274 YouTube comments · 743 Facebook ads · 426 TikTok comments · 277 pages picked by scam type · 148 recently reported phishing pages · 75 ordinary popular sites · 19 dm · 14 mail · 8 known wallet-drainer sites. Dangerous scamsPages that take a password, card or wallet.blind134 of 18672% now183 of 18798% False alarms on ordinary pagesblind13 of 3303.9% now3 of 3300.9% In this pile, caught blindwallet drainers51 of 72 recovery and grant scams37 of 50 fake payment pages16 of 19 fake logins12 of 13 fake stores8 of 11 tech-support traps8 of 10 chat and booking scams1 of 10 fake invoices1 of 1 Social scamsComments, bot threads and ads that lure people to them.blind71 of 11363% now116 of 12295% False alarms on ordinary comments and adsblind9 of 1,9750.5% now1 of 1,9750.1% In this pile, caught blindscam comments53 of 79 scam ads18 of 34 What made it hard. The first round that deliberately tested the kinds of scams earlier rounds never reached, including real emails and chat messages, so the score before tuning is lower. | |||
| 162026-10-02 | blind85%145 of 170now99%166 of 168 | blind79%64 of 81now97%82 of 85 | |
A fresh pile of 2,437 real items: 552 web pages, 559 ads and 1,326 comments and threads. The labellers agreed that 251 were scams, 248 were grey and 1,884 were ordinary; 4 repeats of an earlier round were left out. Where it came from: 1,105 YouTube comments · 559 Facebook ads · 221 pages picked by scam type · 221 TikTok comments · 142 recently reported phishing pages · 81 live phishing-feed pages · 73 ordinary popular sites · 33 known wallet-drainer sites · 2 dm. Dangerous scamsPages that take a password, card or wallet.blind145 of 17085% now166 of 16899% False alarms on ordinary pagesblind6 of 3171.9% now1 of 3170.3% In this pile, caught blindwallet drainers57 of 67 fake logins51 of 52 fake payment pages16 of 20 recovery and grant scams14 of 19 tech-support traps5 of 6 fake stores2 of 4 chat and booking scams0 of 2 Social scamsComments, bot threads and ads that lure people to them.blind64 of 8179% now82 of 8597% False alarms on ordinary comments and adsblind4 of 1,5670.3% now1 of 1,5680.1% In this pile, caught blindscam comments46 of 51 scam ads18 of 30 What made it hard. The biggest round so far, over three times round 15, with fresh ad searches. Ads, wallet drainers and the targeted dangerous pages were the weak spots before tuning. | |||
| 152026-09-30 | blind57%24 of 42now98%41 of 42 | blind70%7 of 10now92%11 of 12 | |
A fresh pile of 584 real items: 256 web pages, 5 ads and 323 comments and threads. The labellers agreed that 52 were scams, 23 were grey and 498 were ordinary; 4 repeats of an earlier round were left out. Where it came from: 268 YouTube comments · 139 recently reported phishing pages · 65 ordinary popular sites · 55 TikTok comments · 28 known wallet-drainer sites · 24 pages picked by scam type · 5 Facebook ads. Dangerous scamsPages that take a password, card or wallet.blind24 of 4257% now41 of 4298% False alarms on ordinary pagesblind1 of 1890.5% now0 of 1890% In this pile, caught blindwallet drainers16 of 27 fake payment pages3 of 9 fake logins4 of 4 recovery and grant scams1 of 2 Social scamsComments, bot threads and ads that lure people to them.blind7 of 1070% now11 of 1292% False alarms on ordinary comments and adsblind3 of 3091% now1 of 3070.3% In this pile, caught blindscam comments7 of 10 What made it hard. Hardest dangerous round in a while: fake cloud-mining sites, fake couriers, fake loan banks, wallet pages on abused hosting. Only 5 ads: the ad searches had run dry. | |||
| 142026-09-30 | blind92%70 of 76now99%75 of 76 | blind55%6 of 11now83%5 of 6 | |
A fresh pile of 544 real items: 285 web pages, 10 ads and 249 comments and threads. The labellers agreed that 87 were scams, 27 were grey and 396 were ordinary; 24 repeats of an earlier round were left out. Where it came from: 143 recently reported phishing pages · 141 YouTube comments · 108 TikTok comments · 58 known wallet-drainer sites · 58 ordinary popular sites · 26 pages picked by scam type · 10 Facebook ads. Dangerous scamsPages that take a password, card or wallet.blind70 of 7692% now75 of 7699% False alarms on ordinary pagesblind0 of 1800% now0 of 1800% In this pile, caught blindwallet drainers63 of 63 fake payment pages6 of 7 chat and booking scams0 of 3 fake logins0 of 1 fake stores0 of 1 recovery and grant scams1 of 1 Social scamsComments, bot threads and ads that lure people to them.blind6 of 1155% now5 of 683% False alarms on ordinary comments and adsblind5 of 2162.3% now2 of 2160.9% In this pile, caught blindscam comments6 of 10 scam ads0 of 1 What made it hard. Drainers 53 of 53. Social was TikTok campaigns again. | |||
| 132026-09-30 | blind85%105 of 124now98%121 of 124 | blind55%22 of 40now97%29 of 30 | |
A fresh pile of 855 real items: 380 web pages, 28 ads and 447 comments and threads. The labellers agreed that 164 were scams, 15 were grey and 595 were ordinary; 55 repeats of an earlier round were left out. Where it came from: 318 YouTube comments · 146 recently reported phishing pages · 128 TikTok comments · 74 live phishing-feed pages · 59 known wallet-drainer sites · 55 ordinary popular sites · 46 pages picked by scam type · 28 Facebook ads · 1 Reddit posts. Dangerous scamsPages that take a password, card or wallet.blind105 of 12485% now121 of 12498% False alarms on ordinary pagesblind7 of 2193.2% now2 of 2190.9% In this pile, caught blindwallet drainers56 of 65 fake logins35 of 39 fake payment pages13 of 16 fake stores0 of 2 recovery and grant scams1 of 1 chat and booking scams0 of 1 Social scamsComments, bot threads and ads that lure people to them.blind22 of 4055% now29 of 3097% False alarms on ordinary comments and adsblind3 of 3760.8% now1 of 3760.3% In this pile, caught blindscam comments22 of 39 scam ads0 of 1 What made it hard. Brand look-alikes with nothing to fill in, and free-host login clones. | |||
Rounds 12 to 17 together, blind: 530 of 679 dangerous scams caught (78%) with 2.2% false alarms, and 176 of 279 social scams (63%).
"Blind" is measured once, the day the batch was fresh, and never re-measured. "Now" is the same batch scored with today's rulebooks. Each round is also tuned on after its blind score, which is why "now" is always higher; the honest measure of new-scam performance is the blind column of the next round.
New features are tested the same way
AI slop tag
1,200 real posts labelled blind. On posts it never saw: 55% of AI slop caught, 1% of human posts tagged, right 94% of the time it tags.
Logo recognition
1,000 ordinary websites it never saw: 1 was taken for another brand. Fake logins with only a logo went from 68% to 89% sure.
Every browser test
The real extension, loaded in a real browser, against test pages for each kind of scam, plus checks that private messages which shouldn't be sent aren't.
What it doesn't catch yet
Being honest about this is how we fix it.
Brand-new scam families
A rulebook has to exist before a stamp can. Blind catch rates for dangerous scams have ranged from 57% to 92% on fresh rounds.
Social campaigns that change
Bot campaigns reword themselves when they're caught. Our weakest social round so far (round 12) scored 25% blind; every round's tuning goes back over the ones before it.
Scams off the page
Phone calls and texts are out of reach. We catch the page, email or chat the scammer sends you to.