🔍 Read the full analysis: How AI Approached Neural Networks Through Their Parameters on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra identified patterns in small neural networks, repaired selected failures and wrote networks from scratch by editing their weights directly. The results are documented in a public blog post, not a peer-reviewed paper, and the experiments were limited in scale and scope.
The researchers gave Astra the raw weights of other networks and asked it to infer what they did, correct failures or produce new networks. Their approach did not use a training loop or gradient descent to make the edits. The source article says that, in some tests, Astra received weights in plain text and had to respond in one turn; in others, researchers ran code it wrote inside a restricted sandbox.
In one repair test, a car-control network that had recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights. In another, an edit affecting 0.016% of the weights in Qwen3.5-0.8B coincided with a rise in its MATH-500 score from 7.9% to 48.1%. The source says a Hugging Face engineer later rebuilt that edited model from the released logs and obtained a score of 51.0%.
The team also reported that Astra identified the games represented by four small game-playing networks and the structures of three small image generators from their weights. In three tiny language models, it found every “copying head,” a component associated with repeating patterns in context. For network creation, Astra produced small game-playing models and a 9-by-9 Go network from zero-valued weights. The Go model beat a same-size comparison network trained on about 20.5 million games in 50 test runs, according to the source.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
What Weight-Level Editing Could Change
If the findings hold up in broader tests, direct weight editing could offer developers another way to inspect and make targeted changes to small models. Instead of running further training, an AI system might identify a component linked to a specific behavior and alter a limited set of parameters. That could be useful for debugging or auditing models whose internal operations are difficult to interpret.
The results do not show that this method is ready for routine deployment. The source reports that Astra could not distinguish an honest model from a sycophantic one by weights alone, and that its changes did not improve mathematics performance in Gemma 3 270M, a model that was not looping. The researchers’ stated limitation was that editing could not add an ability a model lacked. The tests suggest a possible method for targeted fixes, not a general substitute for training or model evaluation.
For model users, the source says nothing changes immediately. For developers and organizations, the work raises practical questions about how model weights are produced, reviewed and changed. A model modified outside a conventional training process may need clear records of what was edited and how the resulting behavior was tested.
Products Worth Considering
Apple MacBook Air 15-inch with M5 chip, 16GB Memory, 512GB SSD, and Liquid Retina Display
LG UltraGear 34GX90SA-W 34-inch WQHD Curved OLED Gaming Monitor
Samsung Galaxy Tab S10 Lite – 6 GB RAM, 128 GB storage, up to 2 TB microSD, 10.9‑inch LCD, Gray
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in learned weights rather than written directly by people. As the source article recounts, Karpathy argued that manually coding millions of weights would be difficult. Conventional training instead adjusts those values repeatedly to optimize performance against data and a defined objective.
The MIT researchers’ proposal is to treat the weights more like a program that a language model can inspect and modify. The source names the team as Rikiya Takehi and Jayden Teoh, who contributed equally, along with Alex Zhang and Omar Khattab. The post says Astra was accessed through OpenAI’s Codex command-line tool with built-in tools such as shell access, web search and file editing disabled. In sandboxed tests, researchers ran Astra-written Python code without network access and imposed fixed test budgets.
The source describes eight experiments and says the researchers checked 1,125 turns in the released logs for tool use. It also notes that the post was not peer-reviewed and was not posted on arXiv at the time of publication. The reported results should be read as an initial set of demonstrations rather than a settled assessment of the method.
““This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.””
— The MIT researchers, quoted in their November 2 blog post
Limits of the Reported Tests
The available evidence has notable limits. The work is a blog post, not a peer-reviewed paper, and the source says it was not available on arXiv. The reported experiments came from one research group, used small networks and, according to the article, included one run per setting. The range of tasks and model families was limited.
It is not yet clear whether the results will reproduce across independent teams, larger models or different model architectures. The source does not establish how reliably weight edits would work in real-world deployments, how often they might cause unwanted behavior, or whether performance gains would persist under broader evaluation. The Qwen re-run by a Hugging Face engineer is an additional check reported by the source, but it does not resolve those wider questions.
Independent Tests Will Matter
The immediate next step is independent replication of the logged experiments, including checks on whether the repairs and performance changes persist across repeated runs and wider test sets. Tests across more model families and larger networks would show whether the method extends beyond the small systems described in the post.
The released logs provide material for further scrutiny, while the source article does not give a schedule for a peer-reviewed paper or a broader study. Until more evidence is available, weight-space programming remains an experimental research approach. Developers considering similar methods would need to document edits and test for regressions as well as improvements.
Key Questions
What is weight-space programming?
It is an approach in which a language model reads or edits another neural network’s weights directly, aiming to infer or change behavior without relying on a conventional training loop for those edits.
What did Astra repair in the reported tests?
The source reports a car-control network that went from zero successful reversals in 256 episodes to 1,024 successes in 1,024 episodes after a weight edit. It also reports a change to Qwen3.5-0.8B associated with an increase in its MATH-500 score from 7.9% to 48.1%.
Was the research peer-reviewed?
No. The source describes the findings as a blog post published by the researchers on November 2, 2026, and says it was not peer-reviewed or posted on arXiv at that time.
Did the method work in every test?
No. The source says Astra could not identify whether a model was honest or sycophantic from its weights alone, and edits did not improve maths performance in Gemma 3 270M, which was not looping.
Does this change how people use AI models now?
The source says there is no immediate change for users. The experiments are preliminary, and the broader reliability and safety of editing weights this way remain unestablished.
Source: ThorstenMeyerAI.com
