01 / A DIFFERENT OBJECT OF OPTIMIZATION
Same scientists.
Different rules of the game.
Imagine two scientific communities with the same researcher models and access to the same literature. One filters manuscripts before sharing them. Another circulates work first, then evaluates it. Funding, information and attention travel differently. What happens to the research that gets done?
Making an individual AI scientist more capable is one way to change that world. We wanted to investigate another: change the system that organizes scientific activity.
That is the motivation behind Science Utopia Lab. Start from a persistent, simulated researcher world. Treat its institutions as executable designs. Choose an explicit objective. Propose a system, run it, measure its consequences, and feed the result back into the next proposal.
The long-term question is not simply “Which acceptance rate is best?” It is “What kinds of institutions and connections does a chosen objective lead us to discover?” Peer review is a useful example, not the boundary of the project.
This also changes what a convincing result would look like. A novel-looking program is not enough. We need to see what it changes in the world, how the search gets there, and whether it outperforms a strong, understandable alternative.
02 / FROM A FIXED MENU TO EXECUTABLE INSTITUTIONS
The institution
becomes a program.
In a parameter study, the rule is fixed and its numbers move. We wanted a larger design space: changing the rule itself, adding mechanisms, and rewiring how they exchange information.
These are separate, composable freedoms. A funding fraction is a parameter. An allocation algorithm is a rule. A knowledge commons is a mechanism. Publishing before evaluation is a change in process. None should be confused with a stronger researcher model.
What can change?
THE DESIGN MOVE
Change a value, keep the rule.
Adjust a funding fraction without replacing the allocation algorithm.
funding.fraction = 0.30 → 0.50
FIXED RESEARCHER MODEL WEIGHTS
RESPONSIVE RESEARCHER ACTIONS
We implemented a representation with typed state, nodes, ports, wires and execution schedules, connected to the original researcher lifecycle and its actual consumers. An LLM designer can edit source and configuration, probe components, submit an executable design, and receive a measured loss.
There is an important boundary. A custom node label is not a magic new native capability. Its implementation needs an executable consumer and the appropriate authority. The framework broadens design freedom; it does not make every conceivable scientific activity real by naming it.
The original ScienceUtopia world is a contained instance and starting point—not something to replace with an easier numerical toy. The ambition is to extend it, while being specific about what each experiment actually exercises.
- 01Choose a lossMake the objective explicit.
- 02Propose a systemNumbers, code or graph.
- 03Run the worldFixed models, responsive agents.
- 04Observe the lossKeep the trajectory and failures.
- 05ReviseUse measured feedback.
03 / PUTTING THE LOOP TO WORK
We ran the search.
Here are the actual paths.
One completed corrected study used a four-year native world, five founding researchers and 103,107 literature records. Researcher models stayed fixed at Qwen3-8B; the free-design role used a local 35B model. Three methods each made four proposals for each of two losses.
Uniform priority search sampled numerical policies. Adaptive priority search updated using feedback. Free code/graph search gave the designer broader editing authority. Three freshly evaluated reference worlds supplied strong comparisons.
Follow the optimization, not just the endpoint.
Select a plotted point to inspect its actual submitted loss.
Every point is a submitted candidate. The missing execution has no numeric loss; no line bridges it.
Read all submitted values
The result was not the breakthrough we hoped for. Of the 23 measured own attempts, eight tied their assigned strong reference and fifteen were worse. None was better. The remaining attempt failed execution; we kept its loss unavailable rather than replacing it with an artificial penalty.
Some paths recovered from poor proposals. That is real movement within a search history—but recovering a supplied strong rule is not inventing a better one. The distinction matters when an optimization plot looks encouraging.
Not an impossibility proof.
Not evidence of useful invention.
04 / THE MORE INTERESTING SURPRISE
Changing the code
didn’t always change the world.
A broad editor creates the possibility of exploration. It does not guarantee that the designer will use it meaningfully.
Across the eight free-design sessions, we recorded 112 completed actions and 25 applied source edits. Among their incremental changed-node rows, 19 preserved the parsed syntax tree and six changed it. Those counts are not 25 independent institutions or six useful algorithms. No wire or schedule edits were generated.
Then we looked beyond the text of the program, at the complete model inputs the researchers actually received.
6 same observed input path
1 changed path, worse utility
1 execution failure
Even a different syntax tree can leave the relevant decision unchanged. A new parameter can be inert if nothing consumes it. A different internal score can preserve the same ranking. Looking only at code diffs or aggregate scores misses these distinctions.
The lesson is narrower—and more useful—than “LLMs cannot design institutions.” In this setup, nominal design freedom did not translate into effective behavioral exploration. That is a diagnosis of the observed search, not proof that the space has no better systems.
05 / THE OBJECTIVE IS A SUBSTANTIVE CHOICE
What, exactly,
did we ask it to improve?
“Better science” is not a single number waiting to be discovered. A loss selects some features of a world and leaves others outside it. That choice is part of the research question.
In the study above, U is a sum of centered abstract-panel opinions, not scientific truth. K is collected activity resources in the simulator, not GPU time, money or actual work. We minimized either −U, or −U + 0.1K.
A separate two-world, hand-designed control makes the consequences unusually concrete. It kept the same researcher inputs and choices, the same accepted content, and the same utility. Collected resources fell from 200 to 160. Unpaid costs rose from zero to 40. Requested expense stayed at 200.
The loss improved. The requested expense did not.
This slider changes the arithmetic for two observed policies. It does not run a simulation, optimize a new system, or establish another experimental result.
That is a valid improvement under the declared objective. It is not evidence of reduced requested work or better science. The right response is not to secretly change the scoring rule until a preferred outcome wins. It is to state what the objective measures, keep the other consequences visible, and interpret the result accordingly.
For future objective-conditioned search, the interesting question remains: when the loss changes, does the discovered institution change—and through what mechanism? Re-scoring an archive is not a substitute for conducting that search.
06 / A WORLD YOU CAN LOOK INSIDE
What if a research record
felt like a place?
Tables are useful, but they make a scientific ecosystem hard to inhabit. We also built NOCTILUCA: an English pixel-art, cyberpunk interface over research records.
Researchers walk between institutional workspaces. Manuscripts, reviews, funding and policy signals appear as flows. The important part is not just the atmosphere: the interface distinguishes canonical events, derived displays and presentation-only choreography.
It is a game-like replay, design and observation interface, not a finished management game. The more ambitious version would let people understand and reshape the same system they are studying. That goal remains distinct from proving that an optimizer can discover a better institution.

07 / OPTIMIZATION DOESN’T REQUIRE A DIFFERENTIABLE UNIVERSE
Use the information
the world actually gives you.
At first, a tempting extension was to make the entire scientific system differentiable. Then backpropagation could, in principle, optimize institutional parameters through the world.
But an original researcher world includes discrete choices, text, arbitrary programs and opaque model calls. Freezing the researcher weights does not make that whole computation differentiable. We do not want to distort the world just to fit one optimization method.
There are several tools, and they apply at different boundaries.
Black-box evaluation
Propose a system. Execute it. Use its measured loss without differentiating through its internals.
Gradients of an expectation
Under the required conditions, differentiate a distribution over designs using likelihood-ratio estimates—even when the sampled world is not differentiable.
Pathwise / differentiable representations
Differentiate the parts actually represented by a suitable differentiable computation. Keep the unrepresented boundary explicit.
Likelihood-ratio and reparameterization estimators are established ideas, not inventions of this project. Parmas and Sugiyama’s unified treatment is a useful way to understand their relationship. The research goal is to optimize flexible scientific institutions using the signal we can obtain—not to insist that everything must be differentiable.
08 / WHAT WE ARE LEAVING ON THE TABLE
A platform is here.
The central result is still open.
We can now express and execute institutional designs, run objective-conditioned searches in the native researcher world, inspect actual behavioral paths, preserve failures, and view the records through a game-like interface.
What we cannot yet claim is useful autonomous invention with stable superiority to strong simple rules. We have not completed unused-condition evaluation of the latest corrected frozen selections. We have not demonstrated recursive improvement of the inventor, or an improvement in real-world science.
The next scientifically meaningful step, if the research resumes, is not merely “more seeds.” It is to make proposals that actually alter relevant behavior, understand their effects, and test any apparent gain against strong controls and genuinely unused conditions. A follow-up information-feedback comparison was started, then stopped at the user’s request before any own outcome. It contributes no improvement result here.
This essay is a way to make the work inspectable before asking anyone to read a long manuscript. The code, original records and working paper remain available behind the pictures.
What scientific systems
could we build,
if their design were
itself a research problem?
FOLLOW THE EVIDENCE
Behind the pictures.
This is a retrospective of existing work, not a new experiment. Interactive diagrams are labeled illustrations; search charts use the original measured values; the accounting slider only re-scores an archived contrast. Earlier parameter, structure, workflow and estimator studies remain in the research index and the 117-page integrated working manuscript. We do not pool different protocols as independent replications.
- Completed corrected search: protocol, outcomes and closure24 own slots, 3 reference worlds; 23 measured own outcomes and one execution null.
- Author actions and observed behavioral outcomesLiteral program edits, actual inputs, losses and retained failures.
- Two-world terminal-cost controlHuman-designed rule; unchanged content/input path and a collected/unpaid cost contrast.
- NOCTILUCA: implemented interface and evidence boundariesIncludes the difference between canonical events and presentation-only movement.
- Parmas & Sugiyama (2021), A unified view of likelihood ratio and reparameterization gradientsPrimary methodological reference; not a new result of this project.
Snapshot: November 17, 2026. Research experiments are paused. Article data and source hashes ↗ · Original media provenance ↗
