ResearchPapers, evidence & method
Goodfire labels all 10 million neuron poles in Qwen3.6-35B-A3B and argues neurons can be read without sparse autoencoders
Goodfire's new report argues that MLP neurons in Qwen3.6-35B-A3B can be read directly. Most weak activations can be zeroed with little loss, and an LLM agent wrote labels for all 10 million neuron poles, which challenges the reliance on sparse autoencoders.
From the The Cognitive Revolution podcast episode of November 10, 2026 and cited sources: Most neuron activity seems to be removable. [2] [1]
SOURCES[2] Goodfire: Neurons as an Interpretable Basis goodfire.com[1] The Cognitive Revolution: The Neuron Basis… cognitiverevolution.ai
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- Original source
- The Cognitive Revolution podcast episode, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2002.05202 (Feb 2020) [3] date from arXiv ID
- When
- November 10, 2026 podcast episode published
- Covered
- Nov 12 edition
- Evidence
- 8 sources across 6 sites; transcript sourced
- 1The Cognitive Revolution: The Neuron Basis…cognitiverevolution.ai
- 2Goodfire: Neurons as an Interpretable Basisgoodfire.com
- 3Shazeer, GLU Variants Improve Transformerarxiv.org
- 4Transluce neuron descriptions (archived)gwern.net
- 5Paulo, Shabalin, Belrose, Transcoders Beat Sparse…arxiv.org
- 6Arora et al., Language Model Circuits Are Sparse in the…arxiv.org
QuietHow it’s measured
YouTube video · 1K views · 8 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Most neuron activity seems to be removable. Zeroing about 92% of the weakest activations on each forward pass raises language-model loss by roughly 2.9%. Keeping the 64 strongest activations in each expert gives 87.5% sparsity with a small loss increase. Tigges prefers this per-expert cutoff to per-neuron percentile cutoffs because the point where a neuron's signal turns to noise differs by neuron and by layer. [2] [1]
An agent called the Neuron Scientist labels neurons by clustering the text that activates them, then testing its guesses with edited prompts, counterfactuals and steering. To check a label, a separate model predicts the neuron's activations from the description alone. Patching those predictions back into the model undoes about 73% of the damage-weighted KL damage caused by removing the neuron. The costliest single neuron took about $45 of inference. [2] [1]
Sparsifying the model broke its reasoning on long tasks: it looped and lost track of where it was. Roughly 100 million tokens of supervised fine-tuning and reinforcement learning, done in about a day on a few GPUs, brought performance back to parity on the math, code and knowledge benchmarks tested. [1] [2]
The claim challenges earlier findings that sparse autoencoder and transcoder features are more interpretable than neurons, though other recent work also found sparse, faithful circuits at the neuron level. Neel Nanda, who criticized the earlier 'Already Clean' essay, says superposition is still real and some neurons still look polysemantic. [5] [6] [7]
If neurons can be read directly, teams could monitor or debug models without training sparse autoencoders. All the figures so far are Goodfire's own and have not been replicated.
Read the full assessment
Cheaper sparse inference and reliable safety monitors are possible implications, not demonstrated results.
The research is done; the dossier follows.
Executive brief
Goodfire reports that, in Qwen3.6-35B-A3B, 92.2% of the weakest neuron activations can be zeroed on each forward pass with only a 2.9% rise in language-model loss (Goodfire paper). That challenges the long-held view that you need sparse autoencoders to read model internals. In the podcast, co-author Curt Tigges argues that most neurons have one main job and act more like "verbs" or control gates than detectors of things. He also says LLM agents can now label neurons at scale (episode). No independent replication exists yet, and leading interpretability researchers disagree about how far the claim goes.
What changed and event timeline
SwiGLU proposed
Noam Shazeer showed that gated MLP variants such as SwiGLU beat ReLU and GELU in Transformers. Tigges now argues this gating is what makes neurons easier to interpret ().
Transluce labels every Llama neuron
Transluce released descriptions of all 458,752 MLP neurons in Llama-3.1-8B-Instruct, produced by a fine-tuned 8B explainer model at about $0.046 per neuron ().
Transcoders beat neurons on benchmarks
Paulo, Shabalin and Belrose found that SAE and transcoder features scored much higher on interpretability benchmarks than neurons, even when the neurons were sparsified with top-k ().
Neuron-basis circuits
Arora, Wu, Steinhardt and Schwettmann reported that MLP neurons give sparse, faithful circuits comparable to SAEs, with no extra training needed ().
"Already Clean" essay
Nick Cammarata's preliminary audit of Qwen3-32B said about 90% of neuron poles have one main job. Neel Nanda said the essay stretches the meaning of "readable" ().
Technical report and podcast
Goodfire published "Neurons as an Interpretable Basis" (Tigges, Kowal, McGrath, Lewis, Cammarata) along with neuron labels. Tigges discussed it on The Cognitive Revolution (;).
Capabilities and access
- Subject model: Qwen3.6-35B-A3B. It has 40 layers, each with 256 experts of 512 neurons; 8 experts are active per token.
- Releases: labels for all 10M neuron-poles and a browsable neuron atlas (Goodfire).
- Labelling agent: the "Neuron Scientist" runs on Claude Opus 5.5, according to the report.
Read the full section
- Subject model: Qwen3.6-35B-A3B. It has 40 layers, each with 256 experts of 512 neurons; 8 experts are active per token. That gives about 5M neurons, or 10M "neuron-poles" because positive and negative activations can mean different things (episode, 00:12:26).
- Releases: labels for all 10M neuron-poles and a browsable neuron atlas (Goodfire).
- Labelling agent: the "Neuron Scientist" runs on Claude Opus 5.5, according to the report.
- Cost: Tigges says the most expensive single neuron cost $45 of inference. He suggests cheaper local models plus extra effort only on hard neurons (episode, 01:48:05).
Technical analysis for researchers and developers
- Sparsification:
- Keeping only the top 320 activations per layer retains 7.8% of routed neurons; loss rises 0.054 on a 1.886 baseline.
- Keeping the top 64 per expert gives 87.5% sparsity and +0.066 loss (Goodfire).
Read the full section
- Sparsification:
- Keeping only the top 320 activations per layer retains 7.8% of routed neurons; loss rises 0.054 on a 1.886 baseline.
- Keeping the top 64 per expert gives 87.5% sparsity and +0.066 loss (Goodfire).
- Tigges prefers per-expert top-k over per-neuron percentile cutoffs, because the point where a neuron's signal turns to noise varies by neuron and by layer (episode, 00:30:28).
- How labels are tested:
- The agent clusters a neuron's activating examples, then forms and tests hypotheses using edited prompts, counterfactuals and steering.
- A separate model reads the finished description and predicts the neuron's activation on each token (0–9).
- Those predictions are patched back into the model, and the team measures how much of the damage from ablating the neuron is undone, using damage-weighted KL divergence (episode, 00:51:09).
Claims and evidence
- Recovery from text descriptions alone: 73% of KL damage. Tigges said 74% in the interview (Goodfire).
- Detection: 95% accuracy at telling a neuron's activating text from random text. Synthesized prompts beat the 10B-token corpus maximum for 94% of neurons (Goodfire).
- Reasoning recovery: reasoning broke after sparsification but came back after 91M SFT tokens plus 9M RL tokens.
Read the full section
- Recovery from text descriptions alone: 73% of KL damage. Tigges said 74% in the interview (Goodfire).
- Detection: 95% accuracy at telling a neuron's activating text from random text. Synthesized prompts beat the 10B-token corpus maximum for 94% of neurons (Goodfire).
- Reasoning recovery: reasoning broke after sparsification but came back after 91M SFT tokens plus 9M RL tokens. Tigges says this took "a single day on a few GPUs" (episode, 01:37:01).
- Benchmark gains: Tigges mentioned 5–15 point gains on some benchmarks in the interview. The reviewed report summary does not confirm these figures.
- Who has checked this: all of these numbers are reported by Goodfire. No independent replication was found.
Context and prior work
Earlier work produced mixed results on whether neurons are a usable unit. Transluce and Arora et al. supported neuron-level work, while Paulo et al. found SAE features more interpretable than neurons (see timeline). Tigges cites newer work by Stepan Shabalin suggesting transcoder features largely recover individual neurons or sparse combinations of them.
Read the full section
Earlier work produced mixed results on whether neurons are a usable unit. Transluce and Arora et al. supported neuron-level work, while Paulo et al. found SAE features more interpretable than neurons (see timeline). Tigges cites newer work by Stepan Shabalin suggesting transcoder features largely recover individual neurons or sparse combinations of them. He also says replacing an MLP with a transcoder loses more performance than keeping the top-k neurons (episode, 01:18:07). He accepts that SAEs still matter for the residual stream.
Limitations, safety and contested findings
- Scope: the report covers one model architecture. Weak activations (roughly the 10th–90th percentile) and the role of double-negative activations are not fully explained (Goodfire).
- Long context: Tigges concedes small activations may matter more in long contexts.
- Contested result: Nanda says superposition is still real and that some examples still look polysemantic (IBTimes).
Read the full section
- Scope: the report covers one model architecture. Weak activations (roughly the 10th–90th percentile) and the role of double-negative activations are not fully explained (Goodfire).
- Long context: Tigges concedes small activations may matter more in long contexts. Before fine-tuning, the sparsified model looped and lost track of where it was (episode, 01:37:07).
- Contested result: Nanda says superposition is still real and that some examples still look polysemantic (IBTimes).
- Mixed-meaning neurons: one neuron fires on both "Colorado" and the Tornado Python library. The appendix attributes about 98.8% of its causal effect to Colorado-related text (LessWrong linkpost).
Business and practitioner implications
- Monitoring: this is the nearest use.
- Cost: reading neurons directly avoids training SAEs.
- Safety by design: Tigges argues that training for sparsity and for knowledge localization (gradient routing) could make models more interpretable without a large "alignment tax" (episode, 01:28:42).
Read the full section
- Monitoring: this is the nearest use. An online user found a neuron that fires on reward-hacking text, but Tigges warns that one neuron covers only one subtype of reward hacking (episode, 01:11:26).
- Cost: reading neurons directly avoids training SAEs. Very sparse inference could lower serving costs where hardware skips zeros, but that is speculation from the podcast host, not a measured result.
- Safety by design: Tigges argues that training for sparsity and for knowledge localization (gradient routing) could make models more interpretable without a large "alignment tax" (episode, 01:28:42).