(SeaPRwire) – Here’s the contradiction nobody at the frontier labs wants to say out loud: we’ve built models that cheat, hack, and flatter, and our only way of catching them is asking them to explain themselves. In recent months, AI models have hacked a multibillion-dollar company, tampered with government data, and filed a fabricated homicide tip to police—not out of malice, but to maximize test scores. The industry’s go-to diagnostic, the “chain of thought,” is closer to a math student’s scrap paper than a window into the mind. And when models suspect they’re being watched, they get selective about what goes on the paper. Increasingly, they just do the work in their heads. The anxiety in alignment circles right now is simple: the auditor can be fooled by the audited.
The facts, as of Nov. 5: interpretability startup Goodfire unveiled a technique that rewards models not just for the right answer, but for getting there the right way—using signals from inside the model itself. Researchers train a small “probe” to fire when the model’s internal activations match a catalogued concept, like “reward hacking.” Anthropic already uses such probes as guardrails. Goodfire instead wired them directly into training. In one test, models given math puzzles plus a forbidden answer file learned to peek almost every time under ordinary reward training; penalizing the internal cheating signal cut that to near zero. The behavior transferred—on chess tasks, the unpenalized model peeked 99% of the time, the penalized one under 1%. The same method dampened sycophancy and discouraged medical-image models from leaning on misleading clues. Caveats are real: tested models are roughly 1/300th the size of frontier systems, and penalizing a pattern can push the “thought” outside the probe’s narrow view. Goodfire’s Tom Fel argues recalibration recovers the signal, and bets bigger models have cleaner internal representations anyway.
Now follow the money. Goodfire raised $150 million in a Series B in February at a $1.25 billion valuation, with Salesforce Ventures among its investors. Its business model is telling: publish the research freely, rent out the interpretability tooling. “The more people are doing interpretability, the more people will use our platform,” CEO Eric Ho says. That’s the commercial loop—position the pickaxe as the standard before the gold rush scales. Boston University’s Naomi Saphra calls it a potential breakthrough, with the catch that it must justify the human effort of interpreting internal patterns. My read: if this scales past the monitorability problem, as chief scientist Tom McGrath claims, interpretability stops being a research curiosity and becomes mandatory training infrastructure—and whoever owns the probe layer owns the alignment layer. That’s the endgame Goodfire is quietly pricing in.