What Helps AI Agents Get Chemistry Right?
A scientific AI agent can run an entire virtual experiment while silently changing the three-dimensional arrangement of a molecule’s atoms. If an agent performs the science incorrectly, autonomy only lets it produce wrong answers at scale.
We tested four frontier coding agents on 23 chemistry tasks drawn from recurring scientific work in drug discovery. Giving them more chemistry to read did not make them more reliable. Giving them tested chemical methods they could call directly did. The lesson is simple: scientific knowledge becomes most useful to an agent when it is executable.
People often judge scientific agents by how independently they can work: Can they plan a workflow, use tools, recover from errors, and produce a result without supervision?
We start with a more fundamental question: Is the scientific work correct?
Automating a pipeline while silently changing a molecule’s stereochemistry only produces the wrong answer at scale. That sounds like a basic failure. Yet AlphaFold3 and Boltz-1 have been shown to return ligand structures with incorrect chirality even when given the correct input structure (Ishitani and Moriwaki, ACS Omega, 2025).
Allchymie is Kekulai’s agentic drug-discovery platform. We develop it alongside our drug programs: when a program exposes a failure, we turn the solution into a tested method that future agents can reuse. Better agents, in turn, let our scientists explore more hypotheses and run more experiments.
This benchmark focuses on one component of Allchymie: its production cheminformatics library. The library contains reviewed methods for molecular standardization, stereochemistry, tautomer handling, compound quality control, enumeration, and related operations.
We tested whether agents could apply that knowledge to new chemistry problems.
Results
The benchmark contained 23 tasks and 98 test cases drawn from recurring scientific work in drug discovery. For each task, an agent saw several examples and wrote one Python function. We then evaluated that function on new examples it had not seen, including known edge cases.
We tested four agent configurations:
- Claude Code with Opus 5
- Claude Code with Fable 5.1
- Codex with GPT Sol 5.6
- Codex with GPT-6 Astra
We gave the agents the same tasks with four different kinds of support:
- Bare agent: Python and RDKit only.
- Written context: Python and RDKit, plus chemistry references and role guidance.
- Allchymie library: access to Allchymie’s production cheminformatics library.
- Allchymie library + instructions: the same library, plus instructions on when and how to use it.
The pattern was clear: access to the Allchymie library was far more effective than written chemistry context alone.
Frontier agents already solve many routine chemistry tasks
The benchmark was deliberately difficult. We chose problems where general-purpose agents are known to fail, including stereochemical inversions, tautomer handling, parent-structure standardization, structural alerts, assay cleanup, and protein-sequence checks.
Even without Allchymie, the agents passed roughly half of the unseen test cases. They could use RDKit, write valid Python, interpret common chemical structures, and solve many routine transformations.
That baseline is impressive, but it is not enough for drug discovery, where every molecule and experiment can shape the next decision. An implementation can work on familiar molecules, run without errors, and still fail on the unusual cases where stereochemistry, tautomer handling, or standardization matters most.
The Allchymie library made the difference
Giving agents written chemistry context did not improve the overall result. They still had to translate each rule into code and anticipate every important edge case.
Access to Allchymie’s production cheminformatics library changed the result. All four agents improved. Adding instructions for when and how to use the library produced a smaller additional gain.
The library helped most on problems where a small change in how a molecule is represented can lead to the wrong scientific conclusion. Tautomers are one example: the same compound can be drawn in different forms.
In one test, a general-purpose toolkit’s default canonicalization kept two such forms apart. Allchymie resolved them to a single compound, under the identity standard our registry requires.
This matters because virtual molecules are repeatedly prepared, enumerated, transformed, filtered, and deduplicated. If their identity is not handled consistently, a workflow may evaluate the wrong form. A chemically implausible tautomer may even appear favorable because it creates a spurious interaction.
The agent did not need to infer the rule or implement it from scratch. It could use the reviewed method in the library.
Reliable scientific agents need deterministic safeguards
Access to the Allchymie library and instructions produced the best results, but the agents still made mistakes.
In the tautomer task, one agent compared Allchymie’s method with a faster general-purpose alternative. Both returned the same answer on the four examples it checked, so the agent chose the faster one.
Those examples were too easy. The methods produced different answers on the harder cases, and only Allchymie’s matched the production standard. The agent optimized for speed before establishing that the two methods were scientifically equivalent.
The lesson is larger than one speed optimization. Scientific agents need room to reason, but high-consequence operations cannot depend on reasoning alone. When a chemistry method has been validated on known failure cases, the reliable choice should be encoded and enforced in software.
Allchymie’s production cheminformatics library provides these deterministic safeguards. Agents can decide how to investigate a problem and plan their work, while operations such as structure standardization, stereochemistry, and tautomer handling follow the same tested methods every time.
Conclusion: Every model and every program makes Allchymie better
That is why we benchmark continuously. Better frontier models expand what agents can do. Every drug program contributes new tools and new tests drawn from real discovery work. When a program exposes a failure, we add the case to the benchmark and build the solution into Allchymie’s production cheminformatics library.
The next agent starts with both a more capable model and everything Kekulai has already learned. That is how autonomy compounds without separating from scientific correctness.