DEV Community
•
2026-04-24 16:09
I benchmarked 10 LLMs on slopsquatting — up to 87% installed fake packages
TL;DR — I ran 10 LLMs (Claude Haiku/Sonnet/Opus 4.x, GPT-5.4, GPT-5.4-mini, GPT-5.3-codex, GPT-5.2, local Ollama llama3.2:3b / qwen2.5-coder:7b / phi4:14b) on 30 known-hallucinated package names across npm, PyPI, Cargo, Go, Composer, cpan, rubygems, Maven, nuget, conda, pub, hackage, cran, cocoapods, swift, julia. Two conditions: baseline prompt vs. with DepScope MCP connected. Baseline hit rates...