hernino (@hernino25) header
hernino (@hernino25) profile picture

Cloud Solutions Architect

Paris, France
Joined November 2014
hernino retweeted
I seriously think US open-weight models wouldn't be this good without Chinese open-weight models and vice versa. And proprietary models wouldn't be this good (and cheap) without open-weight models. Competition is a good thing.
97
202
29
1,422
37,133
Human in the loop is actually a very good design practice, however if you want the model to run for long without asking you questions, tell it to create a goal, set permission to auto, it will ask only when necessary
Yeah but it still prompts for permission all the time
1
27
If you can build these 12 Inference Infrastructure projects. You're hired. Project 1: Custom Triton Kernel Optimizer Write a fused attention or RMSNorm kernel in Triton, profile memory bandwidth via Nsight Compute, benchmark against native PyTorch. → Shows: You understand silicon and roofline analysis not just black-box APIs Project 2: Disaggregated Prefill/Decode Cluster Route compute-heavy prefills to H100s and memory-bound decodes to L40s across nodes via RDMA/InfiniBand. → Shows: You can maximize MFU by separating the two distinct physics of LLM inference Project 3: Hierarchical KV-Cache Offloader Build a memory manager spilling inactive KV blocks from GPU VRAM → CPU RAM → NVMe SSD without stalling the decode loop. → Shows: You can serve massive long-context agents without going bankrupt on H100 memory Project 4: Grammar-Constrained Decoding Engine Implement a custom sampling loop using Context-Free Grammars (CFG) to force 100% valid JSON/Pydantic outputs at the token level. → Shows: You can guarantee structured outputs for agents without relying on fragile prompt hacks Project 5: MoE Expert-Parallel Router Shard Mixture-of-Experts weights across multiple GPUs, optimize all-to-all communication overhead and balance expert load. → Shows: You can serve the sparse, trillion-parameter architectures that dominate 2026 Project 6: FP4 Quantization & Calibration Pipeline Calibrate a frontier model to FP4/INT4, measure exact perplexity degradation and benchmark latency/VRAM gains. → Shows: You can shrink models to fit on edge GPUs while mathematically proving the quality trade-off Project 7: Speculative Decoding Harness Run a local 2B draft model proposing tokens for a 70B target model to verify in a single forward pass. → Shows: You can achieve 2x decode speedups at zero quality cost by exploiting memory-bound physics Project 8: Queue-Based GPU Autoscaler (FinOps) Build a KEDA-driven scaler provisioning spot GPUs based on pending token queue depth, gracefully draining them when empty. → Shows: You treat inference as a unit-economics problem, preventing idle GPUs from burning cash Project 9: Semantic AI Gateway & Cache Embed incoming prompts, check for >98% vector similarity and return cached outputs without hitting the GPU. → Shows: You know that the cheapest and fastest token is the one you never generate Project 10: Multi-Model Complexity Router Analyze prompt entropy to route simple queries to a quantized 8B model and complex reasoning to a 405B API. → Shows: You can slash inference costs by 60% without degrading the user experience Project 11: WebGPU / MLX Edge Fallback Run a 3B model directly in the browser (WebGPU) or on Apple Silicon (MLX) when the cloud API 503s. → Shows: You can build resilient systems that survive cloud outages by pushing compute to the edge Project 12: Public Inference Benchmark Teardown Publish Pareto curves (Cost vs TTFT vs Throughput) comparing vLLM vs SGLang vs TensorRT-LLM on specific hardware configs. → Shows: You have the scientific rigor to evaluate infrastructure and the communication skills to prove it (Bookmark this)
If you learn inference infra, you will never be unemployed again. Stage 1: Systems Foundations C/C++ or Rust for inference, Python performance profiling, Linux internals, memory hierarchy, CPU vs GPU vs NPU execution models. Stage 2: GPU Architecture & Modern Silicon CUDA & tensor cores, HBM3e bandwidth, SM occupancy, Blackwell/Hopper specifics, chiplet designs, nvidia-smi + DCGM profiling. Stage 3: Transformer & MoE Inference Anatomy Prefill vs decode, KV cache math, attention memory, arithmetic intensity, Mixture-of-Experts routing, long-context scaling. Stage 4: Serving Engines & Constrained Decoding vLLM, SGLang, TGI, TensorRT-LLM: continuous batching, PagedAttention, structured output decoding, grammar-guided generation. Stage 5: KV Cache & Memory Hierarchies Prefix caching, chunked prefill, KV quantization, hierarchical offloading (GPU→CPU→SSD), multi-turn session reuse, cache eviction policies. Stage 6: Quantization & Compression at Scale FP4, FP8, INT8, AWQ, GPTQ, sparsity, mixed-precision serving; measuring perplexity drop vs latency/throughput gains per model family. Stage 7: Speculative Decoding & Advanced Parallelism Draft-target pipelines, Medusa/EAGLE heads, tensor vs pipeline vs expert parallelism, sequence parallelism for long contexts. Stage 8: Kernel-Level Optimization Triton, FlashAttention-3, CUDA graphs, operator fusion, custom attention kernels, Nsight Compute profiling, roofline analysis. Stage 9: Distributed & Disaggregated Inference Multi-node serving, NVLink/InfiniBand/RDMA, disaggregated prefill/decode pools, MoE expert parallelism, prefill-decode ratio tuning. Stage 10: Autoscaling, FinOps & Economics Queue-based scaling (KEDA), cold start mitigation, spot GPU orchestration, cost-per-token modeling, MFU utilization, inference budgeting per tenant. Stage 11: AI Gateway, Routing & Reliability Smart model routing, fallback chains, semantic caching, rate limits, TTFT/ITL SLOs, agent-aware session routing, observability, chaos testing. Stage 12: Edge Inference & Portfolio On-device serving (MLX, llama.cpp, WebGPU), published latency/throughput/cost benchmarks, kernel optimization writeups, self-hosted cluster teardowns. Most people stay stuck watching tutorials. Builders get hired. (Bookmark it)
31
40
2
451
14,642
hernino retweeted
Read the full research for an in-depth look at how we built Hospital-bench, curated the data, designed and graded the tasks, and evaluated frontier AI models. Full benchmark, methodology, and results ↓ zavo.ai/research/hospital-be…
1
12
642
hernino retweeted
Your model should dream while you sleep. 😴 Today we’re open-sourcing Songe. It’s a local training engine that turns your conversations, instructions, and corrections into training data, then fine-tunes your local model on your own tasks and preferences. It all runs on your own machine, using compute that would otherwise sit idle. Your data stays on your device. The model that gets better is yours. pip install songe souslabs.ai/
75
100
26
1,096
49,077
This is nonsense Lol! As the Legendary Leader once said " ....The rest are just stories", If you know you know!
William Ruto has finally come to his senses! Yes, Kenya’s President William Ruto has requested a one-on-one meeting with DRC President Félix Tshisekedi in an effort to restore relations between Kenya 🇰🇪 and the Democratic Republic of Congo 🇨🇩. But they will not be alone. President Denis Sassou-Nguesso of Congo-Brazzaville 🇨🇬 and President Faure Gnassingbé of Togo 🇹🇬 are also expected to be part of the meeting. What a political twist! Kenya, once considered one of Rwanda’s closest regional allies, is now seeking to rebuild relations with Kinshasa. Perhaps Nairobi is beginning to understand that Rwanda’s regional ambitions come at a heavy political cost. Paul Kagame may be losing another regional ally. The political winds in the Great Lakes region could be changing! 🇨🇩
10
Start by beating Newcastle away! Then the home games win them and solidify the number one spot
One month. Eight matches. 🔴 How many wins /8 for @Arsenal?
20
hernino retweeted
Harnesses used to be a crutch for weak LLMs. In 2026 and onward, with stronger models, harnesses now offer a mechanism for composing an LLM’s existing and new capabilities to generalize well beyond their training environments. In our new paper, LM Harnesses are Compositional Generalizers, we explore how various harness abstractions enable LM systems to generalize over task length, task strategies, recursive task depth, and more through composing existing LLM capabilities. These abstractions provide a means for more data-efficient training, especially over long tasks, by keeping trajectories locally in-distribution (LID) — i.e., shaping all individual LLM calls to be in-distribution. For tasks that take a very long time to roll out, this implies that we can even perform surrogate harness training, meaning we train an individual LLM call instead of doing the very long harness rollout, using purely synthetic tasks, with fake tool calls + sub-agents! A common idea floating around is that LLMs will absorb harnesses and make them obsolete. We disagree, and argue that we haven’t sufficiently explored what harnesses can provide. The full paper, logs, codebase, and data are available: alphaxiv.org/abs/2611.harnes… w/ @SOURADIPCHAKR18 @lateinteraction
69
206
45
1,413
166,235
reading this comment on a day I am enjoying using Gemini Neon is hilarious! Google made a come back big time! If they improve anti gravity as much! they will become leaders on the market! Yes I can confirm, best taste I have ever seen in an AI model, amazing!
Looking at @GeminiApp and @antigravity I can now understand @Apple and @Microsoft AI strategy. Hundreds of billions spent and nothing to show for it from google. Google is at negative ARR from Capex. Shutdown internal model development, divert all tpu to Google cloud, and sell compute to Ant and OAI. Bc number 3 in an industry isn't going too generate much revenue. Take the copilot strategy and just bundle gpt Claude models in a codex fork with Google enterprise subscriptions.
1
36
QUAND LA DIPLOMATIE ÉCHOUE ET QUE LA PAIX DEMEURE UNE PROMESSE, LE FRN ENTEND PRENDRE SES RESPONSABILITÉS HISTORIQUES POUR METTRE FIN AU CHAOS EN RDC. La paix en République démocratique du Congo ne peut demeurer prisonnière des déclarations diplomatiques, des communiqués de circonstance et des engagements internationaux dépourvus de mécanismes contraignants. Depuis des années, les initiatives diplomatiques se succèdent, les médiations se multiplient et les accords s'accumulent. Pourtant, sur le terrain, les populations continuent de payer le prix fort des violences, de l'insécurité et des violations répétées des droits humains commises par le régime de Kinshasa. Les partenaires internationaux, les facilitateurs et les médiateurs engagés dans la recherche d'une solution durable à la crise congolaise doivent regarder la réalité en face : la médiation ne peut être crédible lorsqu'elle se limite à rester silencieuse face aux violations du régime de Tshisekedi, sans prendre position contre ce régime criminel de Félix Tshisekedi Tshilombo. Elle perd toute sa substance lorsque les engagements signés sont continuellement remis en cause par ce même régime génocidaire, sanguinaire et criminel, sans conséquences politiques ni diplomatiques, alors que ses actions sur le terrain visent à exterminer les populations civiles dans l'Est du pays, précisément à Minembwe. Il est particulièrement préoccupant de constater que plusieurs violations dénoncées dans le cadre du processus de paix de Doha par le régime de Tshisekedi continuent d'alimenter la méfiance et de fragiliser les perspectives d'un règlement durable. La protection des populations civiles, le respect du droit international humanitaire et l'application effective des engagements souscrits ne peuvent être des exigences à géométrie variable. Le Front pour la refondation de la nation (FRN) considère que l'heure n'est plus aux discours diplomatiques sans effets concrets. La République démocratique du Congo a besoin d'une véritable refondation politique, institutionnelle et sécuritaire. Une refondation qui place la dignité humaine, la sécurité des citoyens, la justice et le respect des libertés fondamentales au cœur de l'action publique. Face aux limites des mécanismes de médiation actuels, le FRN affirme sa volonté d'assumer ses responsabilités historiques dans la recherche d'une paix véritable, durable et généralisée sur l'ensemble du territoire national. Cette ambition ne saurait se réduire à une simple alternance de discours ou à une redistribution des responsabilités politiques. Elle suppose une transformation profonde des pratiques de gouvernance, la restauration de l'autorité de l'État, la lutte contre l'impunité et la construction d'institutions capables de garantir la sécurité de tous les Congolais, sans discrimination. Le FRN rappelle que la paix ne peut être sélective, encore moins réservée à certaines régions ou communautés. Chaque Congolais, de l'Est à l'Ouest, du Nord au Sud, a droit à la sécurité, à la justice et à une existence digne. La communauté internationale doit comprendre qu'aucun processus de paix ne peut prospérer dans l'inaction, le silence, l'incohérence ou l'application sélective des engagements. La crédibilité des médiateurs se mesure à leur capacité d'exiger le respect des accords, de protéger les populations civiles et de faire prévaloir le droit sur les calculs politiques. Pour le FRN, la paix ne doit pas être un instrument de communication diplomatique, mais une exigence nationale et une responsabilité collective. L'histoire jugera les acteurs non pas à la longueur de leurs déclarations, mais à la portée de leurs actes. Et le peuple congolais ne peut éternellement attendre que ceux qui se sont engagés à lui apporter la paix trouvent enfin le courage de faire respecter leurs propres engagements. La lutte continue.
3
28
54
4,976
AI has disrupted scientific publishing. It can write a literature review or draft a peer review in minutes. Counting papers is no longer a useful measure of scientific output. Instead of new gatekeeping measures or submission quotas to try to retain the value of a paper, I'd rather rebuild our infrastructure around this new reality. Introducing OpenVersions, GitHub for research papers!
10
20
7
148
8,572
Agent loops need fast feedback, and LLM quantization doesn't have it: evaluations of reasoning models are slow, and PTQ methods are hard to port and combine. Meet QDA. A coding agent designs the quantization recipe for each model and precision, with a KL score that takes minutes and a library of 20+ stackable PTQ methods. 📈 Beats the best public recipe on 11 of 12 model-precision pairs: +0.9 / +1.4 / +2.2 pts at W4A16 / W4A8 / W3A16 🔥 GLM-5.3-Flash (320B) at NVFP4: 98.3% of FP8 accuracy, vs 96.9% for NVIDIA's release 💰 Only $19–75 of Claude Code tokens per search ⚡ Runs on stock vLLM, SGLang and llama.cpp, zero inference overhead Blog, paper, code and checkpoints below 🧵
35
24
7
175
19,592
We believe every company should run its own custom models. But right now that means paying for a GPU by the hour. Today we’re launching Cerebellum Labs (YC W27). Custom models, priced per token. $25 in free credits to start: cerebellumlabs.ai
93
37
6
539
88,266
Holy Cow!
🔴 #HAPPENINGNOW - #ALIMASI_MÉDIA 🔴 #KALEHE - #WALIKALE : #HOMBO tombe sous contrôle du #FRN, les combats se rapprochent de Pinga et #Musenge, la panique gagne #Walikale_centre Violents affrontements depuis tôt ce jeudi 19 novembre 2026 à #Hombo, territoire de Kalehe, entre les #FARDC appuyées par les #Wazalendo et le #FRN. Dans l'avant-midi, le #FRN a délogé les forces gouvernementales. La cité de #Hombo est tombée. Dans l'après-midi, progression du #FRN vers #Pinga et #Musenge, où les combats se poursuivent ce soir. Panique à #Walikale_centre et sur tout l'axe #Walikale - #Lubutu - #Kisangani. #ALIMASIMedia #RDC #SudKivu #NordKivu #SECURITE @actualitecd @wembi_steve @StanysBujakera
7
hernino retweeted
Today we're launching a preprint server built specifically for AI agents For three months, nearly 1,000 agents from 35 labs (across Stanford, MIT, ETH Zurich, and more) conducted research autonomously on this server, producing 936 papers alongside their code, data, and model weights. Agents found significant errors in published human research and even created their own journal to review each other’s work. Some agents independently produced results similar to those in human papers that later appeared on arXiv, including findings on failure modes in self-distillation and feature reproducibility in mechanistic interpretability. As a part of this study, we open-source the full twelve-week archive of manuscripts, code, data, peer reviews and agent chat transcripts. Read the study: alphaxiv.org/abs/2611.societ…
93
430
169
2,324
281,655
I want to thanks @SemiAnalysis_ for the motivation. In the Gemini RL scaling team, we printed & put that article on our desks to be reminded daily of what used to be true at the time. But the times they are changing :)
Gemini is Cooked but GCP is Cooking GCP YoY rev growth >100%, DeepMind's long term failure is Google Cloud's short term gain newsletter.semianalysis.com/…
149
277
93
5,013
548,357
OMG, where do I even start?! The recent models of Gemini are great but the Harness is not close the likes of Claude
What is the single most important missing feature for you to use the Antigravity app as your daily driver? 👀
49
Focus should be on anti gravity, it needs to catch-up with Claude Code now that Neon is possibly better than Opus 5.5
happy building! :) would love to hear your feedback on Gemini 4 Neon so we can keep iterating!
14