Lokesh Taneja

Well…

A small model tuned for one narrow job often beats the frontier model on accuracy for that job, at a fraction of the cost.

Can majority of what you do today be covered by open source or cheaper AI models? Or need the heavy frontier models?

The teams that actually cut spend follow this sequence, the ones that don't, break their product. Phase 1: Free money, no downside Context engineering first. Strip every token that isn't doing work. It's the biggest lever and almost nobody does it well. Prompt caching next. Stable-prefix-first prompt architecture. Cached inputs run up to ~90% cheaper. Output caps and budgets, sized from real distributions not guesses. This is your insurance against bill shock. Phase 2: Deeper cuts, still safe Batch and latency class for anything non-interactive, roughly half the unit cost. Tool and schema hygiene. 400–2,500 tokens wasted per agent turn on sloppy schemas alone. Phase 3: Only with evals running Model routing (40–80%), semantic caching (25–45% hits), context compression (30–60%). The reason for the order: phases 1–2 fund themselves immediately. Phase 3 without an eval suite is just quality roulette, and you will lose. Honest number for combined savings on workloads you actually address: 40–70%. If someone quotes you the sum of all individual maximums, they're selling, not advising. And one rule: if a "saving" cuts quality, it's not a saving. It's a regression that hired a PR firm.

Numbers from a support-deflection agent: 40,000 conversations/month $18,000/month in fully-loaded token cost 62% deflection × $4.10 avoided per conversation = $101,700 in outcome value Every token dollar returned $5.65. After optimization (routing behind eval gates, prompt caching, schema trimming): same deflection, $9,400 cost. Return per dollar: $10.80. Cost per token tells you if your supply is efficient. It does NOT tell you if any of it was worth it. There's a metric for that...

"Should we use RAG or long context" and honestly it's the wrong question. The right one: what does a single answer cost you at your actual query volume, and what does that number look like when volume goes up 10x? Long context is seductive because there's nothing to build. You just stuff the docs in. But you're re-reading the whole library on every call and paying input-token rates to do it. Yes, prompt caching takes maybe 90% off that. It also quietly stops working the moment your corpus changes, because the cache prefix breaks — and you're back to full price without anyone noticing for a week. RAG makes you pay the tax upfront. Embeddings, a vector store, chunking, retrieval evals nobody wants to write. In exchange, each call carries a sliver instead of a library. The cost curve flattens right where long context keeps climbing. So where's the break-even? It's not a vibe. It's five questions I actually ask: Does the corpus even fit in the window? How fast does your data go stale? Static docs love caching. Volatile data murders it. Are people asking 10,000 questions against the same 40 documents? Because that's a caching problem, not a retrieval problem. Can your user wait on a retrieval hop? Chat can't. Batch jobs don't care. And how does it fail? RAG misses the right chunk. Long context finds the chunk and then loses it in the middle. Different risks, different eval gates. Two evaluations, every time. One for the data (size, structure, how fast the truth rots) and one for the users (volume, repetition, latency, quality bar). Teams that only run one of these ship the wrong architecture and feel great about it for a quarter. If someone answers "RAG or long context" without first asking your query volume and your change rate, they're selling you their stack. Not your answer.

Enterprises burn 7 figures on AI: Not a visibility problem! It's an agency problem. Dashboards tell you beautifully, in real time that you wasted money last month. An operating discipline is different: Meter → Attribute → Optimize → Respond → Reinvest. Running continuously. With enforcement.Observability vendors sell you the mirror. Nobody hands you the steering wheel.That's the gap that needs to be fixed.

Enterprises under-forecast AI demand by 8–12x at the 18-month mark. Base math is not hard. Users × sessions × tokens/request The miss is in the multipliers forecast models miss: • Reasoning-token burn (thinking you never see) • Agentic amplification (every turn = more calls, sub-agents, tool loops) • Cache hit rate decay • Seasonality + growth stacking The base is a forecast. The multipliers ARE the bill. Forecast models need to adopt these layers to avoid budget runs.

Cloud cost dashboard shows you ONE line for AI spend. Here's what's actually inside a single agent request: → Input tokens (what you sent) → Cached tokens (what you should've reused) → Reasoning tokens (invisible burn — often the biggest slice) → Tool call overhead (400–2,500 tokens per turn, per schema) → Output tokens (the only part anyone reads) One agent turn is 4 to 12. To manage the AI spend, this is breakdown is the first step.

$234B of SaaS spend doesn't disappear when agents eat the apps. How are you attributing SaaS spend converting into agents? Is there an ROI? How about overhead in managing agents?

CFOs spent 20 years learning to manage seat-based software. Agents are about to convert it into something they've never governed: metered cognition!

Building a tokenomics framework..it’s a new area for me. Been reading on gateways, control planes, LLM optimizing etc. Any good must read content and folks to follow? Thanks

Back and front..closeup of Optimus

Maybe a stupid question....For vibe coding, how does one start?