james (@http_413) header
james (@http_413) profile picture

develooper / bullish on a post-scarcity, post-suffering world via ai / building @thereabouts_app so you never lose a new place to try

Joined August 2017
Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)Photo by james (@http_413)
james retweeted
what if a model ran some of its steps twice before writing the next bit of text? that's the idea behind looped inference. take sensenova-loop-9b as an example: it has 32 layers but runs 56 layer steps. here's why that matters if you're running it at home.
1
1
27
Anthropic $ANT just rang the opening bell for the stock market today at the NASDAQ exchange
1
138
ever load a model, see that it fits, then paste a long document and run out of memory? the model didn't get bigger. the conversation did. one reason is something called the kv cache.
1
14
when someone says “it fits on my gpu,” a pretty good follow-up is “with how much text?” i put a little demo in my ml guide where you can add tokens and watch the cache grow. ml.webfor.fun/transformers#k…
1
4
what if a model ran some of its steps twice before writing the next bit of text? that's the idea behind looped inference. take sensenova-loop-9b as an example: it has 32 layers but runs 56 layer steps. here's why that matters if you're running it at home.
1
1
27
at home, you only have to store one copy of the middle layers. but your machine runs them twice for each new token (a piece of text). that usually means fewer tokens per second than running those layers once on the same setup.
1
6
james retweeted
This week marks the one year anniversary since the last line of code I ever wrote by hand. I can't overstate how big of a transformation my craft, my career, and (by proxy) my life has become since I adopted this alien technology. I don't work any less (quite a bit more in fact), but it's almost always the fun parts now. I have more confidence in my abilities to research, plan, execute, and improve hard projects. I've resurrected a ton of hobby apps. I've built countless toys that help me better engage my daughter in learning and joy. I've saved myself thousands of dollars and even more time. This level of intelligence constantly available is so engaging, empowering, and freeing. I can't even imagine what the next year will bring.
15
10
90
4,481
wtf on pro 100 i can run astra 6.2 ultra and it just decreases by like 1% for fairly complex tasks with multiple subagents. 🤯
51
james retweeted
🏆 AND WE HAVE A WINNER! GPT-6.2 Astra defeats Claude Fable 5.2 ♟️ Final Stats: 71 moves | 204,219 tokens | $2.0671 Token & Cost Breakdown: • GPT-6.2 Astra: 70,203 tokens | $0.6052 • Claude Fable 5.2: 134,016 tokens | $1.4619 Astra takes the win using almost half the tokens and costing significantly less. GG! ♟️
Ladies and gentlemen, place your bets! GPT-6.2 Astra vs Claude Fable 5.2 Who's winning this one? ♟️ 🏆 Results at 9 PM IST.
61
67
11
889
69,064
几个月后的 google 模型选择 belike:
228
829
138
14,061
556,091
james retweeted
One of the funny reasons for the difference in vibes between the influencers and the crowd today on 6.2 Sol/Astra is that we gave a few folks access a while ago and then made a lot of changes in the last 3-4 days and I'm pretty sure no re-testing was done within those days and the videos were produced last week. Don't get me wrong, while the models are a hefty jump compared to their GPT predecessors, they are by no means perfect and there are a few things the teams are excited for the models to get better at in future iterations in terms of taste and niceness of use. Love all the feedback and we hear you!
2,263
443
363
15,103
2,117,372
gpt 6.2 models are good, i don't know what everyone's bitching about. they're not like paradigm shift models but are modest improvements over 6/6.1 and are incredibly efficient.
1
1
167