ShinchaBoo

Great read. Working with AI agents has been cope after cope. Better harnesses, bigger models, and still it feels like working with a system that does not understand purpose. I don’t know if it’s true, but I read an article about AI agents giving up multiple times on Linus Torvalds during one of his "debugging sessions from hell." I liked a quote from that article: "Maybe they don’t make AI agents as stubborn as me." A lot of what’s described in this article closely matches what we have all encountered while working with AI agents. I had a thought along similar lines, maybe not exactly similar, but a highly focused thought on skills and solving real world tasks. The core question was the same: can we delegate tasks that require the creation of novel solutions to AI agents? A fascinating use case that ends up in failure: I asked an AI agent to solve a task given a certain profile (Computer Engineer, 5 years of work experience, 2 years of AI research, and so on). In this case, it had to move away from crude methods and provide novel solutions. The AI agent was not air gapped. It looked at the actual solution and tried to reverse engineer an answer without understanding it, just as mentioned in Token's article. It fails to accumulate true skills, and sycophantic behaviour prevents any chance of true discovery. In fact, AI agents give up quite often on long running tasks. I felt like a cheerleader constantly pushing the agent to keep going, try another approach, and not give up the moment it hit a dead end. Storyboard: Our new approach Can experience be decomposed into waypoints that another agent can navigate? Prep Phase: This setup is a very basic form of the Autoresearch setups that you see today The idea is to extract skills from work published by AI engineers and researchers. Extract them in a way that helps the AI system recreate a path to solve a similar or even completely different problem. Think of an autopilot system for AI. Just like navigation maps used in flight control systems use waypoints, we create lots of markers. Each marker gives the system confidence to move in a particular direction. Any topic is fine as long as a baseline of progress can be established with ease. The AI agent can reconstruct episodes of problem solving from papers, commits, discussions, experiments, failed attempts, talks, notebooks, issue threads, and other traces of how somebody arrived at a solution. The final result of this phase is a storyboard filled with stories or snapshots of experience and waypoints. Launch Phase We send the AI agent to the starting point to look at the state of the world and perform an audit of ideas, similar to a literature review. Using just this information, the AI agent has to recalibrate its skills and reconstruct the decision process behind the experience it has been given. It then tries to solve the task by completing each waypoint, increasing its own level, just like in anime, and continuing its journey towards new waypoints. The interesting part is whether it can eventually create a waypoint that did not exist in the original experience.

"Eight immortals crossing the sea, each displaying their own magic"

(01) RoPE -RoPE is covered in the intial part of Umar's video itself along with code implementation ( deepseek) - New video alert on RoPE: I always though RoPE was a very math dense subject. This video links pure math ( representation theory) with concepts used in RoPE 3Blue1Brown"youtube.com/watch?v=k86eUj4h… Title : Positional Encoding & Group Theory . - Blogs in English by Jianlin Su: I was going through original paper and found english translation of blogs by Jainlin Su. Translations can be found on Tyler Romero's website. Blog01: This is the first blog and deals with why absolute position encoding is a problem. tylerromero.com/translations… Blog 02: Second blog talks about his ideas on Rotary Position Embedding tylerromero.com/translations… - NanoGPT Speedrun ( @tyleraromero ) If you want a smaller simple code speedrun is a good place to start . I have created a visual + code first presentation of Tyler's run. The Baseline run has absolute position encoding ( wpe) along with embeddings wte. In the second run these are replaced with RoPE to get a significant speedup. Baseline: bubblnet.com/lab/speedrun/jo… RoPE: bubblnet.com/lab/speedrun/jo…

Why the F are models talking like this, AI chat is now totally broken. Every sentence is incoherent , expression is all over the place. I got this output after giving a lot of context and direction. what can I do to fix this ?

I am recreating speedruns from the ground up. Idea is to extract valuable signals from existing runs. We have a lot of information such as code changes, PR comments, graphs which can be used to extract and organize signals. Second part is where the story becomes much more interesting, understanding what scales well. So we want good signals and then we filter what scales well. A good example of signals not scaling well is weight tying. Speedrun experiments spend a lot of time with this concept due to sparsity of embedding gradients in the early part of the run. This is not a big problem for large model training & AI engineers won't spend too much time focusing on this aspect. This concept makes sense wrt GPT2 small but won't make much sense with bigger models. Auto research efforts are very interesting, however serious efforts will run into a lot of problems very soon. Number one problem will be data or lack of data. Even with a small experiment like NanoGPT speed run we can only do so many experiments. If these experiments are repeated with a larger model say 30B or 100B how many experiments or iterations will be feasible? I think as an auto research experiment, NanoGPT is not the ideal choice. Maybe we need a new design which is more suited for auto research. Authors discarded PRs and other information & this experiment didn't have access to the internet, so sticking to NanoGPT for future auto research gives little benefit.

Fable 5 is very good at building html blogs with heavy visualization. I feel this is where Fable 5 really shines converting ideas into visual aid in html. I just pass an entire blog written in pdf or md format and ask Claude to build a blog with lots of visualization. What really works is asking Fable to ensure proper coverage in terms of both topic and visualization of each part/idea within the document.

Nanogpt speed run Quiz 01 Early in training, embedding gradients are sparse: only tokens that appear in the batch get a gradient. Sharing lm_head’s denser gradient is a more stable way to move unused tokens. Later, input embeddings and output logits want different geometries, so in the speedrun we expect them to split. What trick does the nanoGPT speedrun use to do this? A. Weight tying : Keep embed and lm_head as the same matrix for the entire run. B. Delayed untying : Tie embed to lm_head for the first 2/3 of training, then copy weights and optimizer state and train them separately. C. 75× embedding learning rate : Leave the matrices untied and scale embed LR to compensate for sparse updates. D. Multi-token prediction : Predict the next k tokens so rare tokens get more gradient signal.

Crazy that such a powerful coding model is open source

Skills up for grab this week at FBA Lab speed run event 😱 🏀 LLM Modelling- Optimizer 🏀 Pytorch Request access —> Join event —> Complete Challenge bubblnet.com

Using AI for research , study or even literature review can be so troublesome. If research consumes a lot of tokens and you default to a smaller model , system overall starts giving wrong answers. For a simple question like "what is required to create a vector" Small Model : To make a vector in math, you need a magnitude (size) and a direction. Large Model : A vector requires two things: A vector space—the coordinate system or “space” in which it lives. One scalar component for every independent axis of that space.

Claude Cowork with blender is so much fun, still work in progress will post the final scene soon. Trying out if it can build basic geometry nodes scene like waves hitting a beach 🌊🏖️

This MTP pull request merge is getting more attention than many model drops. I first noticed MTP while looking at Qwen3.5-0.8B, and now llama.cpp support makes the whole thing more interesting. My current understanding is that MTP mainly improves token generation, not prompt processing. So it helps when the model is writing a lot: chat, coding, long answers, agents, synthetic data, local assistants. But if the workload is mostly huge prompt + short answer, then prompt processing is still the bottleneck. People are mentioning around 1.5x to 1.8x faster token generation in some setups. My question is: how useful is this overall in real local AI workflows? Is MTP going to matter mainly for long generation and agent loops, or will it become a default feature people expect in models?

New UI Preview feature on Claude Code is really great. I gave it a screenshot and asked it to make a navbar prettier. Instead of immediately editing CSS, it first asked me to choose a direction: Refined gold pill Sparkle prefix Glow halo around text That is the part I found useful. For frontend work, “make it prettier” is not a coding instruction. It is a taste decision. Claude Code did not jump straight from prompt to diff. It stopped at the subjective layer first. The flow felt like: visual context → design options → human choice → code edit All in a single clean flow.

This is one of the most crucial lessons in First Break AI. It teaches you how to navigate @huggingface like a pro. Not just: download model → run notebook → move on In this lesson, we go deeper. We look at how open model repos are structured, how to read model files, how config.json connects to the actual model class, and how to trace from a Hugging Face model page into the Transformers code that runs the model. We use Qwen3-0.6B as the learning model. We also look at why Markdown matters so much in AI workflows: model cards, GitHub issues, README files, Discord, Cursor, Claude Code, planning docs, and AI-assisted work. Then comes the biggest win: datasets. Working with datasets is a core AI engineering skill. I show 3 ways to analyze datasets on Hugging Face: Croissant endpoint Data Studio / browser viewer load_dataset with Python, pandas, and plots We inspect dataset structure, categories, response lengths, distribution, short examples, long examples, and how to think about dataset quality before using it for training or fine-tuning. And this sets up the next part: running Qwen3 directly in C, without treating Transformers as magic. Lesson 01: Hugging Face Beyond Upload Watch: youtu.be/MjZio-A9oUY Free cohort: cohort.bubblnet.com/lessons/…

First Break AI cohort.bubblnet.com/ Cohort: 1 May 2026 — 30 June 2026 (2 months) 3⃣ intuitions that make LLMs click: 🗿The model is a pipeline For Qwen3-0.6B: Input → text embeddings → Qwen3DecoderLayer ×28 → RMSNorm → lm_head → output From far away, it looks simple. Most of the intelligence is inside the repeated decoder layers. 🗿 LLMs generate one token at a time. They are causal autoregressive models. At inference time, the model sees the entire context so far, but it cannot see future tokens. The loop is: current context → predict next token → append token → new context → repeat So the model does not produce the full answer in one shot. It keeps extending the sequence one token at a time. Each new token becomes part of the context for the next prediction. During training, the full sequence can be passed in at once, but a causal mask prevents each token from looking ahead. 🗿 The model does not directly output one word. At every step, it outputs probabilities over the full vocabulary. For Qwen3-0.6B, that vocabulary is 151,936 possible tokens. Decoding then chooses the next token.

“Once you start learning about stuff , the density of accessible information increases in an extremely literal sense: You are able to engage with more of the world than you were before, even though the amount of physical world around you has not changed” - Quote from “How I read ” This is a great way to put into words my reaction when I discover a new perspective. One word in Mumbai slang describes this perfectly: “Aaila.” Link to the Original post: open.substack.com/pub/intern…

- Gothically gorgeous - Hauntingly beautiful

Only 5 hour limits are doubled which means weekly limits will hit even faster 👽🤣

Cost/Perf tradeoffs & Evals are the most requested topics for this cohort. I was not expecting these to make top 3. Real life signals are always different from my assumptions.