Random Libertarian Tech Lead (@someRandomDev5) profile picture

Shipping to prod in a $100M+ ARR unicorn startup w/ local AI. 4x DGX Spark (GLM 5.3 Flash), 1x M5 Max 128GB (Qwen 3.8 Flash Next), 1x RTX 5090 (Qwen 3.8 27B).

Joined February 2018
Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)Photo by Random Libertarian Tech Lead (@someRandomDev5)
Damn, didn't have "Inkling winning in the 27B category before Qwen" on my Bingo card.
🚨‼️Alert: Inkling 2 Mini from Thinking Machines: 26B looped transformer available on Hugging Face. Leads every open model under 100B on DeepSWE and SWE-bench Pro. huggingface.co/thinkingmachi…
1
32
On a second glance though, the "lead" is pretty slim. And with Qwen 4 27B expected in the coming weeks, it probably won't last.
38
"So proud of what our team has done" is the passive-aggressive way to humblebrag on work-related social media. It's trying to communicate one message ("I did such a good job") while leaving plausible deniability for another on the surface ("I'm so proud of others I work with").
57
When you first develop a nose for AI slop, you start smelling it everywhere, which feels discouraging and makes you cynical. It's an isolating, lonely feeling: "Are humans even building anything anymore?" But then, if you continue to refine your senses, you start to notice more and more AI-specific patterns, and you start to make finer, more nuanced distinctions. Eventually, you get to a point where you can read all of the AI patterns in a piece of content, and suddenly you can distinguish them from human direction. As long as a human's direction was involved at all in the generation of that content, you start to become able to smell it (separately from the heavy AI stench that might otherwise cover it up). There's definitely still a lot of AI-generated content on social media that no human was involved with. For example, all of the "reply bot accounts" that respond with the same formulaic pattern, where they have two sentences: one short sentence lightly agreeing with you, and then one short sentence following up with a softball question in order to bait engagement from you. Hopefully, most of you can intuitively detect these scenarios by now, even if you couldn't quite put it in words, how you were able to recognize it. But since AI is going to be involved with pretty much everything moving forward, it's useful to learn to be able to find the human in content that was obviously AI-assisted.
40
I have some theories of why Claude writes in this peculiar "Claudish" way, and I have an intuitive theory of how it relates to some of the tradeoffs with Claude's training that lead to its particular strengths/weaknesses compared to e.g. GPT, Gemini, Grok, etc.
1
1
75
I can tell that this O'Reilly book was also written with Claude's assistance. Basically, it likes to select the token "the" too often. It's almost as if it had a train of thought that was shaken a little bit off course, and then it ends up correcting itself afterwards. Instead of saying "the parts that move," it's more natural for a human to say "moving parts." Another of my theories is that this tends to happen more often with more difficult topics where the model is less certain. If you're not sure which substantive word should be next, then a word like "the" is a safe choice for a model that can only plan one word at a time (whenever it's grammatically correct). If not for modeling the rules of proper grammar, you could imagine that it could even get stalled repeating the word "the" over and over.
1
7
So, in other words, at the risk of anthropomorphizing this too much, when you see Claude utter a phrase that says "the _____ that _____," you can almost think of this as being similar to a "nervous tic" in a human. Like a small "tell" that the model "felt" uncertain.
6
You can tell in advance that a stranger is going to be overly-sensitive when you hear them "uptalk" at the end of every sentence. When every single statement they make sounds like a question, it's a sign that they don't feel secure in their frame or their standing.
29
Claude Code is never beating the allegations of being literally the worst popular harness by a wide margin. > "I'll check the actual snapshots in the engine rather than answering from memory" > immediately consults memory
42
I've done informal tests in the past that demonstrated that the models have different "value systems" when they're instructed to think in Chinese instead of English. Something I haven't measured, however, is whether they perhaps score differently on benchmarks when allowed and/or forced to think in English, Chinese, or a mix. 🤔
19
Find ways to get your CPU involved in improving your agentic tasks! For this game I'm building as a side project, I have my CPU running batches of 24 simultaneous parallel game playthroughs at once in order to feed my LLM data that it can use to iteratively improve game balance.
1
1
29
The equivalent proportionate cost breakdown of running DeepSeek V4.1 Flash on my 4 Sparks compared to official API pricing. I could get a bit more precise than this if I sample wattage a bit more granularly. This is based on my actual energy bill (not just the electrical and delivery charges, but also all of the annoying little socialist extra fees added on top, like the low-income assistance charges, random variable city fees, and so on).
2
147
At my grid electrical pricing and at the $4,000 per unit I bought in at, my stack of 4 covers about one unit worth of inference per year if you keep us running 24/7 at a single stream. Concurrent streams are more cost-efficient, and so this could be reduced down to just under 3 years if you keep multiple threads running around the clock.
1
21
This is obviously based on current pricing and this specific model, and both are liable to change over time. 4 years from now, this cluster will probably be running something stronger than DeepSeek V4.1 Flash, and it remains to be seen what its equivalent would be priced at.
13
I've also got 12 kW of grid-tied solar on my roof and 2.4 kW of ground-deployed off-grid solar attached to a shed. During daytime on a sunny day, the ground-deployed off-grid solar generates much more than is needed to run and cool the cluster.
2
1
66
I don't recommend @GoalZero, by the way. That's what I've got for the off-grid shed solar, but their prices have always been high, and their quality control and customer support have taken a nosedive recently. They gave me a Yeti Pro 4000 and a Yeti Tank 4000 that had problems, and it was a major pain trying to get any sort of packaging to return them. They gave me return boxes, but nothing padded to return them in. When I tried to call the shipping company to pick them up, they told me that the labels that they provided weren't valid and that I would have to arrange pickup a different way. Huge hassle, headache, and I've got $8,000 worth of equipment just sitting here for over a year that I've essentially just written off as broken and unreturnable. The most recent time I've tried to reach out to them about it, they've just ghosted me. At least my other Yeti Pro 4000 and Tank still work, so I've got 8,000 W of battery storage for that shed. But I paid for 16,000, so needless to say, I feel pretty ripped off. Probably better to just build it yourself from parts with ChatGPT's help or whatever.
19