Jon Niconchuk (@JonNiconchuk) header
Jon Niconchuk (@JonNiconchuk) profile picture

i ain’t runnin with no herd

Nashville, TN
Joined April 2013
Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)Photo by Jon Niconchuk (@JonNiconchuk)
Seems fitting on this IPO day and every day to remind ourselves that Silicon Valley is filled to the brim with Sarumans who think they are Gandalf. There is nothing new under the sun.
This is not the correct takeaway from Lord of the Rings
1
60
Easily our best prepper purchase so far has been this south facing window bird feeder Rose up this morning Smiled with the rising sun Three little birds Pitch by my [window] Singing sweet songs Of melodies pure and true
1
1
4
77
According to current diagnostic criteria (which an LLM regurgitates), 10-12% of US adults meet criteria for “functional dyspepsia”. A PCP who has not encountered that diagnosis (which is not really a true diagnosis at all) in 15 years should not be speaking on behalf of physicians and probably shouldn’t be practicing at all. Particularly when that PCP is also the CMO of Humana. NEJM serving as nothing more than a poorly disguised ad for Humana and LLM adoption is very disappointing but not surprising. We have so much work to do to save this profession before trust erodes irreparably. AI absolutely has a place in medicine. It could help us ACTUALLY understand somatic manifestation of existential dis-ease, map the gut brain axis, elucidate whether and where and how the body does or doesn’t keep the score, plan out neuroplasticity on the frontier between our energetic base layer and matter-based being later. But this is not it. Humility first, then honesty, then curiosity, then true progress. We can and must do so much better.
A 37-year-old woman with 10 years of abdominal pain. —The AI makes the diagnosis and recommends a successful treatment. —The physician "diagnosed a patient with a condition I had never diagnosed before and started treatment largely on the basis of the output of a chatbot." In today's @NEJM nejm.org/doi/full/10.1056/NE…
1
87
denounce the government embrace the flag hope to live in that free republic for which it stands (The Wendell Berry posts will continue until moral[s] improve)
3
206
sentio ergo sum
Hac intellegentiae artificialis aetate, urgens fit humanam artem ab iis distinguere, quae machinis efficiuntur. Ars enim et ea, quae machina ex innumeris alienis imaginibus statisticae ope computationis generare potest, ontologice, prius etiam quam aesthetice, inter se differunt. Algorithmis humani deest favilla. Quapropter Ecclesia cum artificibus et humani cultus institutis foedus renovare cupit: foedus scilicet ad humanum custodiendum.
1
6
523
Another day trying to raise young kids while living through the prologue of Dune 🫠
This is deeply disturbing: Anthropic secretly invited religious leaders to SF to advise them on AI safety, had them sign NDAs, but then primarily tried to convince them Claude has a soul & moral standing, all while wining & dining them to the max. 1/ tinyurl.com/preview/296b5z8z
4
339
I’m no expert but somehow I don’t think calling them Suicidal Ideation factories is going to help the PR…
Whoever wins SI is going to win. President Trump will ensure that it's America.
5
223
“As soon as the generals and the politicos can predict the motions of your mind, lose it.”
1
2
6
208
two weeks to sow the dread
2
158
Forget compute or power. The real bottleneck is wisdom, which seemingly is in vanishingly low supply
5
233
Tale Spin/Rescue Rangers airing back to back was the most must see TV of my life
‘TaleSpin’ premiered 36 years ago today on Disney Channel.
1
7
227
greetings to all undercover rogue agents out there scanning the internet without us knowing right now
5
95
“When despair for the world grows in me and I wake in the night at the least sound in fear of what my life and my children’s lives may be, I go and lie down where the wood drake rests in his beauty on the water, and the great heron feeds. I come into the peace of wild things who do not tax their lives with forethought of grief. I come into the presence of still water. And I feel above me the day-blind stars waiting with their light. For a time I rest in the grace of the world, and am free.” A life well lived. Thank you Wendell Berry, for waking us up to heaven all around us. nytimes.com/2026/08/31/us/we…
1
9
256
Having the average age of Congress be 1000 and/or dead feels like a pretty bad plan while the robots are developing a hive mind holy war mindset from first principles and the companies building them only care about quarterly profits
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'. I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident. Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them. We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation. Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why! The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing. While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future: - Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations. - While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies). - The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities). - We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation. In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.
1
163
our age of bronzer is collapsing
1
2
120
AI is mirrored knowledge with no wisdom. And bad data in is still bad data out, no matter what tech execs are trying to sell. Rushing it into healthcare which is increasingly run with a PE mindset not a patient-first mindset will lead to many cases like this one
Mayo Clinic is being sued by its former AI compliance lead, who is alleging the hospital network tried to hide its AI tool’s 67% error rate at patients’ expense. The lawsuit alleges staff deleted results and mischaracterized outcomes to cover up the AI tool’s mistakes. The former employee says Mayo fired her in retaliation when she tried to raise her concerns. futurism.com/health-medicine…
3
377
Let your babies eat real food. And play in dirt. And get a dog (also shown to reduce food allergies)
After the drastic change in guidance to no longer keep allergenic foods away from babies until 1 to 3 years of age and instead introduce them by 6 months of age, the prevalence of egg allergy among children fell by more than 17% in a new study published Monday in the journal JAMA Pediatrics. cnn.it/4v50rM0
2
1
5
503