Marcel retweeted
GPT-6 Astra makes significant gains in the Artificial Analysis Coding Agent Index, scoring equal to Fable 5 at lower cost. In the Intelligence Index, it uses fewer tokens than GPT-5.6 Sol for similar performance, but this is outweighed by higher prices
Pricing is 2.5x GPT-5.6 Sol’s current prices across the board, up from $4/$20 to $10/$50 per million input/output tokens, with the same 90% discount for cache reads and 25% premium for cache writes.
We see distinct stories across our two flagship Indices. In the Artificial Analysis Coding Agent Index, GPT-6 Astra equals Fable 5 at less than half the cost, driven by significant token efficiency gains. In the Artificial Analysis Intelligence Index, GPT-6 Astra is more token efficient than its predecessor for similar performance, but this is offset by the price increase.
Artificial Analysis Coding Agent Index - key takeaways:
➤ Rivals top models: In Codex, GPT-6 Astra scores 67 in the Index - approximately equal to Claude Opus 5 and Fable 5 in Claude Code, and Muse Spark 1.3 in Muse Code. Fable 5.1 in Claude Code leads the Index with a score of 70.
➤ 70% more token efficient than GPT-5.6 Sol: GPT-6 Astra sees a substantial improvement in token efficiency, using one third of the tokens compared to GPT-5.6 Sol (max) in the Codex harness, and one fifth of the tokens of Claude Opus 5 (xhigh). Various effort levels of the model occupy the Pareto frontier of token efficiency.
➤ Leads Coding Agent Index cost efficiency frontier: At max effort, GPT-6 Astra costs about the same as GPT-5.6 Sol (max) while scoring 2 points higher on the Index. Per task, the model is less than half the cost of Claude Fable 5, for the same score.
Artificial Analysis Intelligence Index - key takeaways:
➤ Sits beside GPT-5.6 Sol in Intelligence: GPT-6 Astra scores equal to GPT-5.6 Sol in the Index at 61. This is 5 points lower than Claude Fable 5.1 (max with fallback). The model also trails Meta’s newly released Muse Spark 1.3 (max).
➤ ~10% fewer output tokens, offset by price increase: GPT-6 Astra defines a new Pareto frontier for Intelligence Index vs Output Tokens per Task - with a ~10% reduction in token use at max effort compared to GPT-5.6 Sol. However, due to the 2.5x increase in price, the model is 75% more expensive per task than its predecessor at max effort.
➤ Hallucinates half as much as GPT-5.6 Sol: GPT-6 Astra sees a large jump in AA-Omniscience, our knowledge and hallucination benchmark. This is driven by a significant decrease in hallucination rate from 92% to 51% at max effort. Unlike some models, this improvement does not come at the cost of accuracy - Astra increased accuracy by 4 points at the same time.
➤ ~80 point gain in AA-Briefcase Elo: GPT-6 Astra improves ~80 points in AA-Briefcase, our frontier long-horizon knowledge work evaluation. Models are tested on multi-week projects, with many linked tasks and thousands of source files. Astra sees a significant increase in both rubric scores and Analytical Quality Elo in AA-Briefcase compared to its predecessor. In the other direction, we observe a reduction in Presentation Quality Elo, where GPT-5.6 Sol (max) still leads all models.
➤ Mixed progress on other evaluations: The model sees a 6 point gain in Humanity’s Last Exam, a long-standing evaluation with emphasis on mathematics, science, and humanities. This is offset by a drop of ~80 Elo points in GDPval-AA v2 - a benchmark we adapted from OpenAI’s dataset measuring economically valuable tasks across 44 occupations. We also observe 2-3 point regressions on other evaluations across a mix of capabilities, including reductions in τ³-Banking (customer support), SciCode (Python problems in a scientific domain), and AA-LCR (long context reasoning over large documents).
Congratulations @OpenAI and @sama on the launch!
Marcel retweeted
🇵🇹 On the run up to fire season Ive built a website to help people in Portugal track fires in real time (rural and urban). Also tracks how many resources are deployed.
Last year the fires caused complete mayhem and wanted to see if I could build something useful.
fogosagora.pt
Marcel retweeted
Is your generative AI delivering real business value? This article by @onlyMarcelM provides a framework for evaluating the quality and suitability of AI applications in enterprise settings.
towardsdatascience.com/why-g…
Marcel retweeted
Der AI-Hype-Train rollt – aber ohne radikale Entscheidungen isch over für Europa. Ein radikaler Fahrplan für Wettbewerbsfähigkeit:
1️⃣ AI Act beerdigen
2️⃣ Datenschutz anpassen für KI-Training
3️⃣ Energiepolitik überdenken
4️⃣ Kapital für KI entfesseln
5️⃣ Rechenzentren ohne Bürokratie bauen
6️⃣ Top-Talente sofort reinlassen
7️⃣ Gesellschaft über KI aufklären
8️⃣ Öffentliche Datenbanken
9️⃣ Steuerliche Anreize für KI Startups
Passiert das nicht? Dann müssen wir uns nicht der Illusion hingeben, dass wir einen Podiumsplatz bekommen.
Im Thread mehr Infos zu den einzelnen Punkten 👇🏻
Und ganz am Ende mein Plädoyer ☺️
Marcel retweeted
⭐ WEEKLY ROUNDUP ⭐ This lineup of articles tackle the nuts and bolts of building AI apps and leveraging their potential for well-defined goals, while avoiding common pain points.
🖊️ by @onlyMarcelM, @jsbaan, @ChengzhiZhao, @carolinabento, @murtazaali_5253 + more
towardsdatascience.com/build…
Marcel retweeted
We've just released a VC search engine 🙃
4,400+ VC firms. Type in "Enterprise SaaS 👨💼", "Fintech 💵", "Black founder 👨🏿🦱", "Woman-led 🙋🏻♀️"... And voilà!
It's 100% free and collaborative: openvc.app
Also, RT to help a founder out! 👏👏👏
Marcel retweeted
Making LLMs Work For Your Use-Case - Fine-Tuning vs. RAG vs. Long Context Length vs. Prompting
Literally every company is busy trying to understand how LLMs can impact their business positively.
Several have identified multiple use-cases - Some of them LLMs can solve and some need newer generations of GPT-4 - maybe a GPT-5 or 6 😃
There are different ways to implement a custom LLM use-case and here are some approaches we automate as part of the Abacus AI platform.
Retrieval Augmented Generation (RAG) + LLMs - If your use-case involves the LLM knowing about you custom knowledge base or custom data. Then we almost always recommend using a retrieval index in conjunction with an LLM.
The retrieval index is first called when a user queries the customLLM and returns the relevant chunks that contain the response, from the knowledgeBase. Web search, for example is powered by a "retrieval index", that returns the response to your search query, in the form of 10 relevant documents.
These chunks are then sent to the LLM, that extracts and crafts the response. The whole request-response and on-going conversation with the custom ChatLLM needs to be co-ordinated with an orchestration layer.
Of course this is all an oversimplification - Depending on the complexity of the use-case there are multiple steps like OCR, chunking, generating embeddings, creating pipelines that are involved.
In addition, there are complexities around the response sometimes spanning multiple documents, ranking of these documents by the information retriever and understanding the nuance and/or context in the search query.
In summary, more often than not, this is no vanilla DIY set-up where you can simply plug and play a open-source library. For example, our platform makes a number of intelligent decisions based on the type of data, type of common questions and the specifics of the use-cases
Context Length - There is a lot of discussion on X and elsewhere on the context length of the LLMs. This is the max. number of tokens you can send to the LLM in one request.
Of course, the longer the context length, the more chunks of the knowledgebase you can send to the LLM and have the LLMs parse through these chunks and respond appropriately.
So can long context lengths replace RAG? - That is unlikely. Experiments show RAG combined with 32K context lengths are better for SOTA accuracy.
While RAG boosts smaller context (2K and 4K) context length models significantly, it also improves 16K and 32K context length models
It is important to note that LLMs have the "lost in the middle" problem. LLMs perform poorly on the words / information in the middle of the context that is passed to them.
This phenomenon manifests as a U-shaped performance curve where models excel at using relevant information situated at the extremes of the input but falter when this information is nestled in the middle
In specific tasks like multi-document question answering and key-value retrieval, where identifying relevant information within long contexts is crucial, this lost in the middle issue becomes a significant bottleneck
Another angle to this is that as the input context grows longer, the performance of the models, even those designed for long-context, substantially decreases
All this makes a strong case for RAG + LLM vs. just long-context LLMs. Unfortunately the devil is in the details and designing performant custom ChatLLMs requires a combination of skill and trial and error. End-to-end LLMOps platforms like Abacus AI make it easy to test, iterate and create performant systems
Fine-tuning LLMs - This has been a hot topic on X as well with a lot of research on how to efficiently fine-tune LLMs. Fine-tuning LLMs make sense when you have several supervised examples or labelled data. With fine-tuning, you are basically teaching it the type of response you expect for a particular type of question
You can generate the labelled data either form human labelling, asking another LLM to generate labelled data (clever ;)) or from using human preferences/feedback on a existing LLM responses.
We also recommend fine-tuning when prompt engineering is NOT sufficient. For example, if you want to LLM to generate cease and desist letters, you might be able to fine-tune an LLM with a few hundred examples of those types of letters
Alternatively, you may able to simply pass an example of the letter in the the prompt and see if the LLM is performant without having to do fine-tuning
Fine-tuning also makes sense of domain-specific tasks like python and code generation.
In summary, different techniques make sense based on your use-case and creating these custom ChatLLMs will vary widely in performance based on different settings, the details of the orchestration and and prompt engineering cheats
The good news is that you can build some very interesting and useful AI agents and ChatLLMs that are capable of smart decision making as long as you have access to a smart LLMOps platform
Marcel retweeted
Marketing teams must balance AI and human-created content for distinctive digital campaigns. AI can generate content quickly, but human Oversight ensures accuracy to avoid misinterpretation and capitalize on creativity. #AI #marketing #contentcreation
metia.com/blog/is-ai-generat…
Marcel retweeted
Explore the strategic implications of #GenerativeAI on companies' #valuecreation processes at our upcoming webinar. Learn how to leverage this rapidly-evolving field to automate & complement manual work & revolutionize software products. #AI #Automation
eventbrite.de/e/chat-gpt-mid…
Marcel retweeted
Our next free Webinar for Innovation Managers is coming on Wednesday, 10.05.2023
#webinar #innovation
eventbrite.de/e/chat-gpt-mid…
Marcel retweeted
Recent reports predict that AI could automate up to 300 million full-time jobs worldwide. But a distinction must be made between jobs that can be automated and jobs that will be supplemented by #AI. #automation #jobs #Employment
ctvnews.ca/business/300-mill…
Marcel retweeted
Check out the latest post on “WHO’S @NGI4eu" BLOG, with @onlyMarcelM 🤩
✅How KnowledgeX analyses & processes #data-driven decisions in a trusted, traceable & transparent environment
✅Contribute to the ONTOCHAIN software ecosystem
✅ONTOCHAIN support 👇
ngi.eu/blog/2023/03/14/trust…
Marcel retweeted
Hängt Deutschland am VC-Tropf? Wir haben in Sachen Wachstumskapital Nachholbedarf und drohen in eine Abhängigkeit von außereuropäischem Kapital zu rutschen.
Warum ist das problematisch?
🌎 Geopolitische Abhängigkeit
👋 Risiko der Abwanderung
📉 Verpasste Rendite
Marcel retweeted
In our latest article we show how Trust Trail is using #blockchain technology to provide trust, integrity and auditability for multi-participant processes in carbon capture supply chains. Learn how it can ensure trust in the #carboncapture process! deeptechcenter.org/post/intr…
Marcel retweeted
iExec offers grants for projects using the iExec $RLC protocol.
Here's an interview with @onlyMarcelM, whose project KnowledgeX was funded through the iExec Developer Program.
KnowledgeX is a marketplace for data scientists, powered and secured by iExec: youtu.be/G4mRSx2HoKQ
Marcel retweeted
Data scientists: Are you freelancing, or can you imagine freelancing at one point?
23%Already freelancing
44%Might consider freelancin
17%Always corporate
15%Tried and didn't work
124 votes • Final results Marcel retweeted
Have you seen KnowledgeX demo?
This platform connects data owners with #datascientists, enabling #trust -aware knowledge extraction from #data, without giving away control of it. 🤩
Check it out! 👇
https://xitter.gq/t.co/pd2bYlvgKh




















