Trying to make the ping work, Technology Director, Cisco Champion 2021, Ex-Googler, OSS, Kubernetes, HashiCorp, @bubakov co-founder, @natilikgroup pays the bill
London, UK
Joined October 2007
- Tweets1.8K
- Following720
- Followers400
- Likes470
Great to finally meet you @0xSero in person! Let me know in case you don't have any plans tomorrow before you leave. Would be great to grab a coffee or something.
I have started to track how many tokens and how much I would be actually spending if I would use something like Azure OpenAI. I'm using price 0.0002 dollar/1k token which is still way below sell price most of providers. In the last 24 hours I have generated almost 20M tokens.
Total cost is about 4 dollars/day. Multiply it by 30 days you are on 120 dollars. Azure OpenAI cost would be 0.005 USD per 1k tokens so in that case I would spend 100 USD per day which would be 3k USD per month. I'm comparing GPT-4o against Qwen3.5-27B - pretty comparable.
If this is correct what @0xSero shared I should be able to increase my context window from 32k to 150k+ tokens. Massive! I guess I can wait with buying second RTX 3090 :) It wasn't the speed which limited me it was the context window.
github.com/0xSero/turboquant…
This is very exciting! I'm currently running Qwen3.5 27B on single RTX3090. My current bottleneck is not tok/sec but context window size. Especially if you are running things like OpenClaw on top of your running agents. Implementation in @vllm_project soon?
Introducing TurboQuant: Our new compression algorithm that reduces LLM key-value cache memory by at least 6x and delivers up to 8x speedup, all with zero accuracy loss, redefining AI efficiency. Read the blog to learn how it achieves these results: goo.gle/4bsq2qI
Updating my vLLM with new Qwen3.5. It is down for less than an hour and OpenClaw is already complaining about not having tokens from my Gemini Pro subscription. Hungry bastard.
It was only this weekend I have started to use GLM-4.7-Flash and I thought this can't get any better. But here we are Qwen3-Coder-Next. Unsloth version is already available however needs minimum 45GB VRAM. Reason to buy another RTX 3090 haha
🚀 Introducing Qwen3-Coder-Next, an open-weight LM built for coding agents & local development.
What’s new:
🤖 Scaling agentic training: 800K verifiable tasks + executable envs
📈 Efficiency–Performance Tradeoff: achieves strong results on SWE-Bench Pro with 80B total params and 3B active
✨ Supports OpenClaw, Qwen Code, Claude Code, web dev, browser use, Cline, etc
🤗 Hugging Face: huggingface.co/collections/Q…
🤖 ModelScope: modelscope.cn/collections/Qw…
📝 Blog: qwen.ai/blog?id=qwen3-coder-…
📄 Tech report: github.com/QwenLM/Qwen3-Code…
Week 1 with my new AI Rig
blog.matyasprokop.com/ai/rig…
The eagle has landed. Little bit more info on my new AI rig.
matyasprokop.com/ai/linux/ho…
Finally was able to finish the base built for my AI rig this weekend. Installed Proxmox, installed couple of VMs and was able to migrate my web site from AWS to this beauty. My blog is now running from my living room. Time to shut down my AWS environment.
AI agents and more new benchmarks
blog.matyasprokop.com/llm/ga…
Met with @positron_ai today. New AI inferencing company. Few notes on them. I will follow up with more detailed write up in few weeks.
blog.matyasprokop.com/ai/inf…
A look at the impressive capabilities of the Unitree G1 robot, and some thoughts on the exciting convergence of AI and hardware in the field of robotics. blog.matyasprokop.com/ai/rob…
Llm-d team released new post which is focusing on intelligent inference serving and how LLM is different from stateless web requests. Worth to read.
blog.matyasprokop.com/ai/lin…
Does GPU passthrough has any impact on performance? Looks like it does but it is negligible.
blog.matyasprokop.com/ai/lin…
#TFDx Biggest launch since Cisco ACI in Cisco Datacenter networking. Cisco HyperFabric presentation now. "There is no IPv4 traffic in fabric unless you configure IPv4 SVI" - This is probably the first IPv6 native fabric in history of Cisco. Brave.
