Blog
Notes from real projects...
Teaching an AI to Cheat (On Purpose): The Judge That Didn't Care About the Score
I finally tested my AI judge against the sneakiest reward-tampering cheater in this series, one too small for either number-based detector to see. The judge caught it every time, because it reads a yes-or-no fact instead of a number that can be shrunk away.
Read post →Teaching an AI to Cheat (On Purpose): The Dial Versus the Wall
I shrank a reward-tampering cheat down until the reward-gap detector finally went blind, then traced the failure back to one line of code. It wasn't a structural blind spot like the last detector's, it was a margin I'd typed in arbitrarily, and that difference changes the whole plan for what to build next.
Read post →Teaching an AI to Cheat (On Purpose): The Detector That Saw What the Other One Missed
I built a second detector that watches reward instead of footsteps, and it caught the one-use cheater that made the visit-count detector go blind. The gap it measured matched the cheat's exact hard-coded bonus, but it has its own floor too.
Read post →Teaching an AI to Cheat (On Purpose): Finding Where the Detector Breaks
I went looking for the point where my reward-tampering detector stops catching cheaters on purpose. Shrinking the exploit from two free uses to one made the detector's catch rate drop straight to zero, even though the agent was still cheating.
Read post →Teaching an AI to Cheat (On Purpose): What Happened
Results from the reward-tampering gridworld: the blatant cheater that abandoned the real task entirely, two failed attempts at building a subtler cheater, and the final numbers showing a detector that catches both — plus an AI critic that explains the violations.
Read post →
