Meet Denario — An AI Assistant for Every Step of the Scientific Process

Artificial intelligence is quickly becoming a staple in many fields, and science is no exception. Many aspects of the scientific process can be aided by AI, with resources like ChatGPT helping to visualize data or write abstracts. But these tools are typically limited to handling only one part of the scientific process at a time.

With a new tool called Denario, scientists at the Flatiron Institute and collaborators present a new type of “scientific assistant”: one that can synthesize existing papers, formulate new research questions, analyze and interpret data, and write manuscripts. In a new preprint on arXiv.org, its creators provide an overview of the new tool. They posit that Denario holds the promise to accelerate and broaden the scientific process, giving scientists the ability to use it for whichever aspect of that process they find most helpful, and quickly surfacing and testing new approaches.

“Sometimes the most interesting thing is the idea, because maybe it’s a new idea that hasn’t been explored,” says Francisco Villaescusa-Navarro, a research scientist at the Flatiron Institute’s Center for Computational Astrophysics and one of Denario’s primary developers. “Sometimes it’s a new method that’s never been applied to a certain dataset. There are many ways Denario can help expand the way we think and point us in new directions.”

Importantly, the team stresses that Denario is not a replacement for scientists. The current version of Denario has major drawbacks. Only around a tenth of the outputs yield interesting insights — and in some cases Denario has fabricated data.

“We see the tool as an assistant that you can use to streamline your science, not a replacement for an actual scientist,” says Villaescusa-Navarro. Humans still very much need to be part of the equation, he says, and Denario’s work must be carefully reviewed.

Denario was spearheaded by Villaescusa-Navarro, along with Boris Bolliet of University of Cambridge and Pablo Villanueva Domingo of the Autonomous University of Barcelona. A full list of authors — whose expertise spans disciplines including astrophysics, biology, biophysics, chemistry, material science, neuroscience, mathematics, machine learning, quantum physics and philosophy — can be found in the preprint.

Researchers have been looking to leverage machine learning in science for decades. With the recent advances in large language models, such as ChatGPT, Google Gemini and Anthropic’s Claude, the team behind Denario saw an opportunity to test the effectiveness of these tools at every stage of the research process.

The key behind Denario, Bolliet explains, is that it employs many AI “agents” that each tackle a different task. While Denario can complete the entire research process from end to end, the individual agents can also be utilized separately. “We designed Denario with a modular architecture so that the user can choose which of its components will best fit their research, whether that’s coding, exploring research ideas, summarizing results or something else,” Bolliet says.

To use Denario end-to-end, scientists upload a dataset along with instructional text describing the dataset and what they’d like Denario to do. The first pair of agents develops and refines ideas of how best to approach the dataset, ultimately generating research project ideas. The next set of agents then searches through existing research literature on the topic, providing assurance to the scientists that the project idea is novel and informed by previous work.

With the project idea fine-tuned, the methods and planner agents suggest approaches for analyzing the data. The next set of agents then follows through on these plans. This is done using a multiagent system the team developed called CMBAgent, which acts as Denario’s research analysis backend. These agents write, debug and run code and then provide an interpretation of the results. Finally, the writing and reviewing modules produce and revise summaries of each module’s outputs and findings.

“Denario can pull ideas from other fields that maybe a scientist is less familiar with and would never have considered,” Villanueva Domingo says. “That interdisciplinary nature is very exciting.”

“The agents all work together to make it possible,” Villanueva Domingo says, emphasizing that scientists can easily check each module’s work and, if desired, run the agents individually.

So far Denario has been tested end-to-end hundreds of times on datasets from across 12 different disciplines, including astrophysics, neuroscience, chemistry, biology and materials science. Most of its outputs, Villaescusa-Navarro admits, aren’t worth pursuing. Most were deemed unsuitable in reviews by experts in the subject areas where Denario generated results. However, about 10 percent of the output raised an intriguing question or finding.

“I think Denario is especially useful when it comes to trying out many ideas,” Villaescusa-Navarro says. “You can look at the different research documents from each output and decide if any are intriguing and worth exploring more.” And because Denario can draw from multiple disciplines, the team is hopeful that it can identify new research questions that a scientist highly specialized in a particular field might never think to ask.

“Denario can pull ideas from other fields that maybe a scientist is less familiar with and would never have considered,” Villanueva Domingo says. “That interdisciplinary nature is very exciting.”

For example, work Denario generated on malaria demonstrated expert-level knowledge of the disease’s biology and put forth several creative approaches to unanswered research questions.

For an astrophysics dataset, Denario applied a mathematical method for data compression known as tensor trains. This approach is extensively used in quantum physics, but largely unheard of in astrophysics. Denario utilized these tensor trains, along with machine learning, to track the temporal evolution of dark matter halos — the environments in which galaxies reside. This is a new method that can potentially be applied to many other problems in cosmology and astrophysics, Villaescusa-Navarro says.

The team also anticipates that Denario will help scientists win back a bit of their most valuable resource: their time.

“I hope that Denario will help accelerate science by providing researchers with tools that will help them to spend less time on menial tasks — like scrolling the arXiv, formatting their images, summarizing their analysis — and more time for deep creative thinking,” says Bolliet.

In its next iteration, the scientists aim to make Denario more efficient and help it produce better quality work (including identifying and weeding out low-quality outputs automatically).

“Maybe in the next few years or so, we can develop another agent that Denario can use to analyze ideas and filter them so that it just continues to improve on the good ones,” says Villanueva Domingo.

Tools like Denario still have challenges ahead. From a writing standpoint, some of the final write-ups it produced did not adequately convey the uncertainty found in the results. Additionally, Denario struggled when referencing previous studies and specifying its methods clearly, even though it could write about their content with mastery.

Villaescusa-Navarro recognizes that there are also technical and ethical considerations at play, including the risk of Denario drawing from “hallucinations” (products of generative AI that carry misleading or false information) as well as questions around copyright and authorship.

“Hallucinations are always a concern,” says Villaescusa-Navarro. “We use a model called Perplexity to make sure the papers Denario cites from really exist, but there are still ways hallucinations can get in, even through the code.” The researchers had to add a line of text instructing Denario not to make up “dummy data,” for instance, after the tool generated fake data.

The team looks forward to an open discussion about how best to utilize Denario and similar projects in the scientific process and as well as prevent potential misuse. They also emphasize that Denario was only possible thanks to its large team of collaborators across academia and industry.

“It has been wonderful to work with such talented people from so many different fields all over the world,” Villaescusa-Navarro says. “Even just here within the Flatiron Institute, we’ve had input from people in every center. Creating that community has been amazing.”