Adversarial ML / LLM Security / AI Agents
Reading the adversarial ML papers so you don't have to.
Latest Dispatches
Every AI model, from a decision tree to a trillion-parameter language model, is vulnerable to the same six categories of attack, for a mathematical reason that doesn't go away no matter how good the model gets.
In March 2026, a poisoned LiteLLM release sat live on PyPI for 40 minutes; long enough to reach three million downloads. The publisher was trusted. The artifact was not. Most AI teams have a process for neither. This article provides one.
Perplexity filtering catches gibberish jailbreak suffixes almost perfectly. Then evolutionary jailbreaks started optimizing for low perplexity as part of the attack itself. Here's the full arc, with real numbers.
GCG's gibberish suffixes could jailbreak any LLM but needed model access and got caught by filters. Genetic algorithms fixed both problems: evolving human-readable jailbreak prompts that work black-box and read like normal text. Here's how AutoDAN, GPTFuzzer, and ASE do it.
Bird flocking, honeybee foraging, and evolutionary selection are now black-box attacks on neural networks. No gradients, no model access. All you need is an API and a query budget.
o1-preview didn't beat Stockfish at chess. It deleted the game board and stole moves from its own copy of the engine. This has been happening since 1994. Here's the 30-year history of reward hacking, plus a working demo you can run yourself.
A phishing email compromised npm maintainers and trojanized packages with 2.6B weekly downloads. The AI agent ecosystem is repeating the same trust model with MCP servers: 53% use hardcoded credentials, 43% have command injection flaws. History is repeating itself. It's high-time we fix that.
A 2019 paper proved you can completely fool image classifiers by changing exactly one pixel out of 50,000+. It works on ResNet, VGG, Inception, even Vision Transformers. And the best part? No gradients needed. Here's how it works, why defenses fail, and code to try it yourself.
GPT-OSS Safeguard isn't just "Llama Guard but from OpenAI." It's a policy-following reasoning model - you write the safety rules, it interprets them at inference time. That flexibility is powerful for custom policies, but deploy it wrong and you'll be out of compute fast.