Home

 › 

Uncategorized

 › 

A Researcher Says AI Models Are Far Easier to Read Than Anyone Thought. Not Everyone Is Convinced.

A close-up view of a dark gray microchip with the white letters 'AI' on its surface, centrally positioned on a densely populated electronic circuit board. The board is illuminated with strong blue light on the left and magenta light on the right, casting a colorful glow over the numerous small electronic components, solder points, and etched pathways. Text labels such as R29, C12, U71, R45, and C26 are visible among the components.

A Researcher Says AI Models Are Far Easier to Read Than Anyone Thought. Not Everyone Is Convinced.

Quick Read

  • One of the architects of a foundational AI theory now says its most famous examples were simply misread, and his reinterpretation changes everything.
  • An OpenAI insider called this a potential historic moment, but then immediately listed the reasons it might not be.
  • If the findings hold up, AI safety researchers gain a powerful new tool, and the same is true for everyone trying to make AI more capable.
  • Just released. Our analysts combed the entire stock market and named the ten best stocks to buy right now. The report is free. Enter your email and see if any of your stocks made the cut.

Nick Cammarata, a researcher at the interpretability lab Goodfire, says the inner operations of large AI models are much easier to understand than the field has assumed. He lays out the case in a long essay titled “Neural Networks Are Already Clean,” which The Decoder reported on in detail on October 16, 2026. So far the claim exists only as an essay. Goodfire has not yet released the technical paper meant to go with it, and researchers disagree about what it means.

What Interpretability Tries to Solve

Mechanistic interpretability tries to trace how an AI model reaches its answers internally, going beyond what the model outputs. Ideally, each artificial neuron (a small unit of computation inside the network) would stand for one recognizable concept. In practice, many neurons seemed to fire on completely unrelated things. The usual explanation is superposition, the theory that a network short on neurons packs unrelated concepts into the same ones. That is why many labs train a second tool, such as a sparse autoencoder (a separate model built to pull those blended concepts apart), and study that tool instead of the original model.

A Reversal From One of Superposition’s Architects

Cammarata helped build the view he is now challenging. According to The Decoder, he and Chris Olah co-wrote “Zoom In,” an influential early paper that introduced the first superposition circuit. In 2023, he started an automated interpretability project at OpenAI. At that point only about 1.7% of neurons counted as understandable, even by the most generous measure.

He now says the famous examples were misread. On TBPN, his post was quoted as saying “many famous examples of tangled neurons turned out to be misunderstood.” His main example is a neuron in InceptionV1, an older image model. For years it was cited as responding to both cat faces and car fronts. Cammarata now reads it as a detector for pointed V shapes. Fox snouts and flames activate it, while round shapes like the letter O push its activation negative.

Preliminary Numbers Awaiting Verification

For Qwen3-32B, Cammarata offers early figures. He says about 90% of neuron poles have one main job, about 7% have two related jobs and 2% to 4% remain unexplained. Goodfire says it sees similar results in models with up to trillions of parameters. AI agents analyzing neurons in bulk did most of the work. His summary, as quoted on TBPN: “the simple thing at absurd scale works.” These figures come from Cammarata and Goodfire via The Decoder. No independent group has confirmed them yet.

Roon Calls It a Possible Historic Moment

Roon, an OpenAI employee who posts on X under that handle, wrote that the work “may be remembered as a major moment in scientific history.” He said even Eliezer Yudkowsky, one of the best-known voices warning of AI disaster, would have to lower his estimated odds of doom. The Decoder notes that Roon says he is close friends with Cammarata.

Roon also had reservations. He said others still need to confirm the results. He suspects interpretability advances will push AI capabilities forward. And he said discoveries coming from outside do not bode well for the big labs, OpenAI included.

Google DeepMind’s Neel Nanda Pushes Back

Neel Nanda leads the interpretability team at Google DeepMind and is considered a founding figure of the field. According to The Decoder, he found the work interesting but thinks it leaves the big picture largely unchanged. He expects neuron-based analysis to help with practical questions, but he also expects it to have serious flaws and miss a lot. He still believes superposition is real, pointing out that language models store far more facts than they have neurons. He also doubts that findings from small image models carry over to larger systems.

What Is at Stake for AI Safety and the Labs

If Cammarata is right, that is good news for AI safety. Today there are few reliable ways to tell whether a model is acting deceptively. Readable neurons could let researchers spot that behavior directly in a model’s circuits. A guest on TBPN pointed out that the same knowledge can also be used to improve capabilities: “If you’re trying to make some capability better within the model… you can figure out exactly how it’s working right now and then kind of optimize on that.”

For investors following the AI labs, the open questions are whether the findings hold up and who benefits from them. Goodfire is an interpretability company, so it has an obvious stake in the thesis. Roon’s comment about outside discoveries raises a competitive question too: a key advance from a smaller lab could shift where talent and research attention go.

Goodfire’s Paper Is the Next Catalyst

Cammarata’s claim is serious and comes from someone with deep standing in the field. For now, though, it rests on an essay and numbers his own company reported. The next step to watch is the release of Goodfire’s technical paper. If labs such as Google DeepMind can reproduce the about 90% figure on their own models, Nanda’s concerns become much harder to sustain. If they cannot, the essay will be remembered mainly as a fresh look at old examples.

To top