Ok so I tried out the tool on an area where I’m an expert where I already have very strong knowledge of the literature and understanding of the prevailing wisdom of the field. I also input a dataset I generated and analyzed myself (literally with my hands) so I knew the methodology backwards and forwards. Quick aside to define some terms: nearly all of your cells have an antenna, a cilium, that does two way communication to sense and transmit signals and its length has everything to do with its function. Hundreds of genes are involved in the formation, maintenance, and function of this antenna. When they’re mutated, you get a whole host of multi-symptom disorders from blindness to sterility to developmental disorders to kidney disease and and and and …. The mechanisms that affect cilium length/function are therefore of great translational interest. The dataset I generated (like 15+ years ago) was a chemical screen of 1280 FDA approved drugs with known targets to identify mechanisms affecting cilium length and function. The hypothesis it generated with the greatest surprisal score was an interesting one. The prior was indeed a vague opinion I held, that on the basis of existing research, there was a potential mechanistic tie between cilium resorption and autotomy (self-severing in response to stress). Think shrinking vs. severing. Various studies showed the activities occur in tandem or in sequence. And number of cellular processes involve both. The two activities can be independent and separable but the relationship is unclear and suggestive. The tool suggested due to insufficient evidence (not statistically significant) of enrichment of shortening drugs among severing set and vice versa that the data don’t support a shared mechanism. And there’s relatively little overlap between the compounds that result in both outcomes. So despite the suggestive evidence of a link, these specific data don’t support it. So what do I think about this conclusion? A hypothesis presumed to be true for which the null based on the data cannot be rejected. As always with AI, I’m not using it to replace my thinking but it’s making me think differently and more deeply. About the prior evidence too. And indeed even the evidence of shared mechanism in shrinking and severing are vaguely suggestive but not clearly demonstrated. But also the lack of mutual enrichment given the bias in the library and relatively small N is certainly no nail in the coffin. So interpretation of imperfect data via other imperfect data is well…imperfect. But this exercise is immensely useful and it has meaningfully changed my perspective. The lab experiments I’d propose to tease it apart are different. And I wouldn’t just throw up my hands and be opinionated about the parsimonious model based on what’s known (objectively little) but would instead be more precise in the perception of likelihoods in my mind. Once again, I implore scientists to not live or die by the specific outputs of these models and tools but enjoy the absolute richness of the experience of being given a jet pack. That we get to live in this technological moment as scientists is hard to fully appreciate. Making scientists better and ask harder questions previously not sufficiently explored IS making science better even if it doesn’t yet replace us.