Marcel (@onlyMarcelM) profile picture

I work with VCs to scale their pipeline. See https://xitter.gq/t.co/vT2Y1rcrB1

Lisbon, Portugal
Joined October 2016
Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)Photo by Marcel (@onlyMarcelM)
Jensen Huang has a message for Germany
Jensen Huang has a message for the UK
121
615
207
4,117
1,155,957
GPT-6 Astra makes significant gains in the Artificial Analysis Coding Agent Index, scoring equal to Fable 5 at lower cost. In the Intelligence Index, it uses fewer tokens than GPT-5.6 Sol for similar performance, but this is outweighed by higher prices Pricing is 2.5x GPT-5.6 Sol’s current prices across the board, up from $4/$20 to $10/$50 per million input/output tokens, with the same 90% discount for cache reads and 25% premium for cache writes. We see distinct stories across our two flagship Indices. In the Artificial Analysis Coding Agent Index, GPT-6 Astra equals Fable 5 at less than half the cost, driven by significant token efficiency gains. In the Artificial Analysis Intelligence Index, GPT-6 Astra is more token efficient than its predecessor for similar performance, but this is offset by the price increase. Artificial Analysis Coding Agent Index - key takeaways: ➤ Rivals top models: In Codex, GPT-6 Astra scores 67 in the Index - approximately equal to Claude Opus 5 and Fable 5 in Claude Code, and Muse Spark 1.3 in Muse Code. Fable 5.1 in Claude Code leads the Index with a score of 70. ➤ 70% more token efficient than GPT-5.6 Sol: GPT-6 Astra sees a substantial improvement in token efficiency, using one third of the tokens compared to GPT-5.6 Sol (max) in the Codex harness, and one fifth of the tokens of Claude Opus 5 (xhigh). Various effort levels of the model occupy the Pareto frontier of token efficiency. ➤ Leads Coding Agent Index cost efficiency frontier: At max effort, GPT-6 Astra costs about the same as GPT-5.6 Sol (max) while scoring 2 points higher on the Index. Per task, the model is less than half the cost of Claude Fable 5, for the same score. Artificial Analysis Intelligence Index - key takeaways: ➤ Sits beside GPT-5.6 Sol in Intelligence: GPT-6 Astra scores equal to GPT-5.6 Sol in the Index at 61. This is 5 points lower than Claude Fable 5.1 (max with fallback). The model also trails Meta’s newly released Muse Spark 1.3 (max). ➤ ~10% fewer output tokens, offset by price increase: GPT-6 Astra defines a new Pareto frontier for Intelligence Index vs Output Tokens per Task - with a ~10% reduction in token use at max effort compared to GPT-5.6 Sol. However, due to the 2.5x increase in price, the model is 75% more expensive per task than its predecessor at max effort. ➤ Hallucinates half as much as GPT-5.6 Sol: GPT-6 Astra sees a large jump in AA-Omniscience, our knowledge and hallucination benchmark. This is driven by a significant decrease in hallucination rate from 92% to 51% at max effort. Unlike some models, this improvement does not come at the cost of accuracy - Astra increased accuracy by 4 points at the same time. ➤ ~80 point gain in AA-Briefcase Elo: GPT-6 Astra improves ~80 points in AA-Briefcase, our frontier long-horizon knowledge work evaluation. Models are tested on multi-week projects, with many linked tasks and thousands of source files. Astra sees a significant increase in both rubric scores and Analytical Quality Elo in AA-Briefcase compared to its predecessor. In the other direction, we observe a reduction in Presentation Quality Elo, where GPT-5.6 Sol (max) still leads all models. ➤ Mixed progress on other evaluations: The model sees a 6 point gain in Humanity’s Last Exam, a long-standing evaluation with emphasis on mathematics, science, and humanities. This is offset by a drop of ~80 Elo points in GDPval-AA v2 - a benchmark we adapted from OpenAI’s dataset measuring economically valuable tasks across 44 occupations. We also observe 2-3 point regressions on other evaluations across a mix of capabilities, including reductions in τ³-Banking (customer support), SciCode (Python problems in a scientific domain), and AA-LCR (long context reasoning over large documents). Congratulations @OpenAI and @sama on the launch!
385
340
352
2,925
1,681,412
🇵🇹 On the run up to fire season Ive built a website to help people in Portugal track fires in real time (rural and urban). Also tracks how many resources are deployed. Last year the fires caused complete mayhem and wanted to see if I could build something useful. fogosagora.pt
65
66
9
577
71,966
Marcel retweeted
89
193
15
4,363
98,633
Is your generative AI delivering real business value? This article by @onlyMarcelM provides a framework for evaluating the quality and suitability of AI applications in enterprise settings. towardsdatascience.com/why-g…
4
7
2,534
Der AI-Hype-Train rollt – aber ohne radikale Entscheidungen isch over für Europa. Ein radikaler Fahrplan für Wettbewerbsfähigkeit: 1️⃣ AI Act beerdigen 2️⃣ Datenschutz anpassen für KI-Training 3️⃣ Energiepolitik überdenken 4️⃣ Kapital für KI entfesseln 5️⃣ Rechenzentren ohne Bürokratie bauen 6️⃣ Top-Talente sofort reinlassen 7️⃣ Gesellschaft über KI aufklären 8️⃣ Öffentliche Datenbanken 9️⃣ Steuerliche Anreize für KI Startups Passiert das nicht? Dann müssen wir uns nicht der Illusion hingeben, dass wir einen Podiumsplatz bekommen. Im Thread mehr Infos zu den einzelnen Punkten 👇🏻 Und ganz am Ende mein Plädoyer ☺️
39
33
5
240
12,203
⭐ WEEKLY ROUNDUP ⭐ This lineup of articles tackle the nuts and bolts of building AI apps and leveraging their potential for well-defined goals, while avoiding common pain points. 🖊️ by @onlyMarcelM, @jsbaan, @ChengzhiZhao, @carolinabento, @murtazaali_5253 + more towardsdatascience.com/build…
2
4
2,362
We've just released a VC search engine 🙃 4,400+ VC firms. Type in "Enterprise SaaS 👨‍💼", "Fintech 💵", "Black founder 👨🏿‍🦱", "Woman-led 🙋🏻‍♀️"... And voilà! It's 100% free and collaborative: openvc.app Also, RT to help a founder out! 👏👏👏
110
633
95
1,501
Making LLMs Work For Your Use-Case - Fine-Tuning vs. RAG vs. Long Context Length vs. Prompting Literally every company is busy trying to understand how LLMs can impact their business positively. Several have identified multiple use-cases - Some of them LLMs can solve and some need newer generations of GPT-4 - maybe a GPT-5 or 6 😃 There are different ways to implement a custom LLM use-case and here are some approaches we automate as part of the Abacus AI platform. Retrieval Augmented Generation (RAG) + LLMs - If your use-case involves the LLM knowing about you custom knowledge base or custom data. Then we almost always recommend using a retrieval index in conjunction with an LLM. The retrieval index is first called when a user queries the customLLM and returns the relevant chunks that contain the response, from the knowledgeBase. Web search, for example is powered by a "retrieval index", that returns the response to your search query, in the form of 10 relevant documents. These chunks are then sent to the LLM, that extracts and crafts the response. The whole request-response and on-going conversation with the custom ChatLLM needs to be co-ordinated with an orchestration layer. Of course this is all an oversimplification - Depending on the complexity of the use-case there are multiple steps like OCR, chunking, generating embeddings, creating pipelines that are involved. In addition, there are complexities around the response sometimes spanning multiple documents, ranking of these documents by the information retriever and understanding the nuance and/or context in the search query. In summary, more often than not, this is no vanilla DIY set-up where you can simply plug and play a open-source library. For example, our platform makes a number of intelligent decisions based on the type of data, type of common questions and the specifics of the use-cases Context Length - There is a lot of discussion on X and elsewhere on the context length of the LLMs. This is the max. number of tokens you can send to the LLM in one request. Of course, the longer the context length, the more chunks of the knowledgebase you can send to the LLM and have the LLMs parse through these chunks and respond appropriately. So can long context lengths replace RAG? - That is unlikely. Experiments show RAG combined with 32K context lengths are better for SOTA accuracy. While RAG boosts smaller context (2K and 4K) context length models significantly, it also improves 16K and 32K context length models It is important to note that LLMs have the "lost in the middle" problem. LLMs perform poorly on the words / information in the middle of the context that is passed to them. This phenomenon manifests as a U-shaped performance curve where models excel at using relevant information situated at the extremes of the input but falter when this information is nestled in the middle In specific tasks like multi-document question answering and key-value retrieval, where identifying relevant information within long contexts is crucial, this lost in the middle issue becomes a significant bottleneck Another angle to this is that as the input context grows longer, the performance of the models, even those designed for long-context, substantially decreases All this makes a strong case for RAG + LLM vs. just long-context LLMs. Unfortunately the devil is in the details and designing performant custom ChatLLMs requires a combination of skill and trial and error. End-to-end LLMOps platforms like Abacus AI make it easy to test, iterate and create performant systems Fine-tuning LLMs - This has been a hot topic on X as well with a lot of research on how to efficiently fine-tune LLMs. Fine-tuning LLMs make sense when you have several supervised examples or labelled data. With fine-tuning, you are basically teaching it the type of response you expect for a particular type of question You can generate the labelled data either form human labelling, asking another LLM to generate labelled data (clever ;)) or from using human preferences/feedback on a existing LLM responses. We also recommend fine-tuning when prompt engineering is NOT sufficient. For example, if you want to LLM to generate cease and desist letters, you might be able to fine-tune an LLM with a few hundred examples of those types of letters Alternatively, you may able to simply pass an example of the letter in the the prompt and see if the LLM is performant without having to do fine-tuning Fine-tuning also makes sense of domain-specific tasks like python and code generation. In summary, different techniques make sense based on your use-case and creating these custom ChatLLMs will vary widely in performance based on different settings, the details of the orchestration and and prompt engineering cheats The good news is that you can build some very interesting and useful AI agents and ChatLLMs that are capable of smart decision making as long as you have access to a smart LLMOps platform
27
146
8
757
225,250
Explore the strategic implications of #GenerativeAI on companies' #valuecreation processes at our upcoming webinar. Learn how to leverage this rapidly-evolving field to automate & complement manual work & revolutionize software products. #AI #Automation eventbrite.de/e/chat-gpt-mid…
1
1
64
Check out the latest post on “WHO’S @NGI4eu" BLOG, with @onlyMarcelM 🤩 ✅How KnowledgeX analyses & processes #data-driven decisions in a trusted, traceable & transparent environment ✅Contribute to the ONTOCHAIN software ecosystem ✅ONTOCHAIN support 👇 ngi.eu/blog/2023/03/14/trust…
1
19
1
36
1,018
Hängt Deutschland am VC-Tropf? Wir haben in Sachen Wachstumskapital Nachholbedarf und drohen in eine Abhängigkeit von außereuropäischem Kapital zu rutschen. Warum ist das problematisch? 🌎 Geopolitische Abhängigkeit 👋 Risiko der Abwanderung 📉 Verpasste Rendite
4
11
1,393
iExec offers grants for projects using the iExec $RLC protocol. Here's an interview with @onlyMarcelM, whose project KnowledgeX was funded through the iExec Developer Program. KnowledgeX is a marketplace for data scientists, powered and secured by iExec: youtu.be/G4mRSx2HoKQ
19
23
5
49
Data scientists: Are you freelancing, or can you imagine freelancing at one point?
23%Already freelancing
44%Might consider freelancin
17%Always corporate
15%Tried and didn't work
124 votes • Final results
Have you seen KnowledgeX demo? This platform connects data owners with #datascientists, enabling #trust -aware knowledge extraction from #data, without giving away control of it. 🤩 Check it out! 👇 https://xitter.gq/t.co/pd2bYlvgKh
3
2
1
14