Why do people keep approving AI agent commands they shouldn't?

Founders Miss 1 in 3 Dangerous AI Agent Commands

An August 2026 scalex.dev study found people miss a third of dangerous AI agent commands across 409,000 approval decisions in tools like Claude Code. Thomas Davenport's All-in On AI argues the fix is governance structure, not vigilance, which is why I now sort every command into a three-bucket approval protocol instead of reviewing each one myself.

I approved somewhere around four hundred Claude Code commands last month without really reading most of them. I know that because I went back and checked the session logs after reading a study that made me genuinely uneasy about what "human in the loop" actually means in practice.

The study is from scalex.dev, published August 6, 2026: a browser game called llmgame.scalex.dev where you play the human approving or denying commands from an AI coding agent, some routine, some quietly malicious. After 40,000 game sessions and 409,000 individual approve-or-deny decisions, the average player missed one in three threats, a mean accuracy of 66.3%. The breakdown by category is the part that stung: obviously destructive commands like rm -rf / were caught 88.3% of the time, but exfiltration and code-execution attacks, the ones that actually steal your credentials, were missed 33.4% of the time, nearly three times as often. The single most-missed command in the whole game was npm run analyze, approved 64.7% of the time, even when the agent's own history log showed the script piping stats.json to an unknown API. Two-thirds of players saw the evidence and clicked approve anyway.

Anthropic has already named this problem. In a May 25, 2026 engineering post, "How we contain Claude," the company wrote: "The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision." Their fix at the product level was Claude Code's auto mode, which automates the safer class of approvals so you're not desensitized by the time a dangerous one shows up. It helps, but it doesn't solve the underlying problem for anyone running an agent outside Anthropic's own scaffolding, which by now is most of us.

What actually changed my approach wasn't the study, it was rereading Thomas Davenport and Nitin Mittal's All-in On AI, specifically their argument that companies serious about AI need governance structures, algorithm review boards, explicit rules, before deployment, because "if a company is relying heavily on AI in its business, it needs to ensure that the AI systems it uses are ethical and trustworthy, or it's likely to lose more from AI than it gains." Solo founders don't get a review board. You are the review board, on your third coffee, at 11pm, four hundred approvals into the month. The scalex.dev numbers are what happens when you try to run that board on vigilance alone.

So I stopped reviewing every command and started sorting them, the way an actual governance structure would. I call it the three-bucket protocol. Bucket one is pre-approved and silent: read-only and idempotent commands (git status, npm test, ls) never prompt at all, matching what Claude Code's auto mode already does for you. Bucket two is hard-blocked, no override: anything matching a destructive or credential-scope pattern (rm -rf, reading a dotfile, piping to an external host) gets denied automatically before I ever see it, using Claude Code's permission rules file rather than my own judgment in the moment. Bucket three is the only one I actually read, and for those I apply one rule the study proves most people skip: read the last three lines of the agent's stated reasoning before the command, not just the command itself, because that's exactly where the npm-run-analyze trap hides.

It's not foolproof. I've still approved something in bucket three I shouldn't have, once, a scoped API call I assumed was reading data when it was writing it, and I only caught it because a teammate flagged the output an hour later. The honest failure mode of this protocol is the same one Davenport names for corporate governance: rules only work if someone actually maintains the deny-list as new command patterns show up, and a solo founder updating that list at midnight is not meaningfully different from the reviewer who got tired in the scalex.dev game. The protocol reduces the problem. It doesn't remove it.

Sources: scalex.dev, "Humans missed 1 in 3 threats approving AI agent commands across 40,000 plays" (Aug 6, 2026); Anthropic, "How we contain Claude" (May 25, 2026); The Register, "Humans in the loop miss a third of dangerous AI coding agent requests" (Aug 6, 2026); Thomas H. Davenport & Nitin Mittal, All-in On AI (Deloitte/HBR Press).