👨💻 PM | ex @Metabase, 5y+ @ycombinator network, 🐕🦺 Labrador & 🐺GSD Dad, 🏴☠️, Ex-athlete 🏊♂️, sci-fi passion, Typo’s my own 😅
Planet Earth
Joined June 2013
- Tweets1.1K
- Following1
- Followers76
- Likes3.9K
Alessio Gozzoli retweeted
Welcome to DS4, a specialized inference engine for DeepSeek v4 Flash. github.com/antirez/ds4
This project would have been impossible without the existence of llama.cpp and GGML and the work of @ggerganov and all the other contributors. Thanks!
Alessio Gozzoli retweeted
1/ I am excited to announce the new Playground: Design graphics like a Pro. You can make T-shirts, Logos, Social Media Posts, and even invent new kinds of memes.
Powered by our next image foundation model Playground v3 (beta).
Go to playground.\com. No wait list.
Alessio Gozzoli retweeted
⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs.
The announcement:
---
We are Eureka Labs and we are building a new kind of school that is AI native.
How can we approach an ideal experience for learning something new? For example, in the case of physics one could imagine working through very high quality course materials together with Feynman, who is there to guide you every step of the way. Unfortunately, subject matter experts who are deeply passionate, great at teaching, infinitely patient and fluent in all of the world's languages are also very scarce and cannot personally tutor all 8 billion of us on demand.
However, with recent progress in generative AI, this learning experience feels tractable. The teacher still designs the course materials, but they are supported, leveraged and scaled with an AI Teaching Assistant who is optimized to help guide the students through them. This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted).
Our first product will be the world's obviously best AI course, LLM101n. This is an undergraduate-level class that guides the student through training their own AI, very similar to a smaller version of the AI Teaching Assistant itself. The course materials will be available online, but we also plan to run both digital and physical cohorts of people going through it together.
Today, we are heads down building LLM101n, but we look forward to a future where AI is a key technology for increasing human potential. What would you like to learn?
---
@EurekaLabsAI is the culmination of my passion in both AI and education over ~2 decades. My interest in education took me from YouTube tutorials on Rubik's cubes to starting CS231n at Stanford, to my more recent Zero-to-Hero AI series. While my work in AI took me from academic research at Stanford to real-world products at Tesla and AGI research at OpenAI. All of my work combining the two so far has only been part-time, as side quests to my "real job", so I am quite excited to dive in and build something great, professionally and full-time.
It's still early days but I wanted to announce the company so that I can build publicly instead of keeping a secret that isn't. Outbound links with a bit more info in the reply!
Alessio Gozzoli retweeted
You should probably stop whatever you're doing and watch this right now, because it's amazing.
youtube.com/watch?v=TtVJ4JDM…
Alessio Gozzoli retweeted
Ten days ago I posted about GPT Store being a bit sad 😢:
What if we could build an open source alternative, with the full power of the Community?
So last Friday we launched Hugging Chat Assistants, and the adoption has been impressive:
- 4,000 Assistants have been created on huggingface.co/chat/assistan…
- you can view/customize all prompts to improve your own Assistant
- 1,500 users have chatted with my own LLM-powered clone, Clone of HF CTO (try it! it's fun)
Compared to GPT Store, Hugging Chat Assistants are:
- free to use (for both the creator and the user)
- powered by the best open source models (that you can choose)
This is only the start though. 🫡
Based on the community's initial feedback we are thinking of adding:
- Edit your Assistants via API, so you can always push up-to-date information to them.
- Add RAG (and web search) to Assistant
- Generate your Thumbnail Assistant via AI
- Suggest changes on other users Assistants
- Continually add new models to HuggingChat and Assistants
- any additional request, please send to @huggingface
Super excited to see what the open source AI community builds together ❤️
Let's see how much bigger this'll get 🤯
I'm so excited about this project and can't wait to see what people will build
💫 We're excited to announce our latest experimental open-source project for autonomous data labeling: Adala
github.com/humansignal/adala
Alessio Gozzoli retweeted
We're looking for 1-2 product engineers to join our team at Playground. You'll work on everything from 0 to 1 and ship multiple times a week.
Mostly what we care about is that you have a bias towards shipping fast, have an eye for design, and you're excited about changing the course of graphics.
DMs open!
Alessio Gozzoli retweeted
here is dalle 3, which imo is quite amazing: openai.com/dall-e-3
it will ramp to all chatgpt+ users over the next couple of weeks.
fantastic work by @model_mechanic (head of dall-e), @neobjb @gabeeeegoooh @jingli911 (the other lead dalle ICs), and the entire team.
Alessio Gozzoli retweeted
GPT-3.5 and LLaMA2 fine-tuning guides 🪄
Considering LLM fine-tuning? Here's two new CoLab guides for fine-tuning GPT-3.5 & LLaMA2 on your data using LangSmith for dataset management and eval. We also share our lessons learned in a blog post here:
blog.langchain.dev/using-lan…
Alessio Gozzoli retweeted
Oops haven't tweeted too much recently; I'm mostly watching with interest the open source LLM ecosystem experiencing early signs of a cambrian explosion. Roughly speaking the story as of now:
1. Pretraining LLM base models remains very expensive. Think: supercomputer + months.
2. But finetuning LLMs is turning out to be very cheap and effective due to recent PEFT (parameter efficient training) techniques that work surprisingly well, e.g. LoRA / LLaMA-Adapter, and other awesome work, e.g. low precision as in bitsandbytes library. Think: few GPUs + day, even for very large models.
3. Therefore, the cambrian explosion, which requires wide reach and a lot of experimentation, is quite tractable due to (2), but only conditioned on (1).
4. The de facto OG release of (1) was Facebook's sorry Meta's LLaMA release - a very well executed high quality series of models from 7B all the way to 65B, trained nice and long, correctly ignoring the "Chinchilla trap". But LLaMA weights are research-only, been locked down behind forms, but have also awkwardly leaked all over the place... it's a bit messy.
5. In absence of an available and permissive (1), (2) cannot fully proceed. So there are a number of efforts on (1), under the banner "LLaMA but actually open", with e.g. current models from @togethercompute, @MosaicML ~matching the performance of the smallest (7B) LLaMA model, and @AiEleuther , @StabilityAI nearby.
For now, things are moving along (e.g. see the 10 chat finetuned models released last ~week, and projects like llama.cpp and friends) but a bit awkwardly due to LLaMA weights being open but not really but still. And most interestingly, a lot of questions of intuition remain to be resolved, e.g. especially around how well finetuned model work in practice, even at smaller scales.
Alessio Gozzoli retweeted
Not enough stories get told about startups that don't make it.
@jakefuentes and @brelig recently shared a bunch of hard-won and incredibly insightful lessons from their attempt at building @Cascadeio (a data logic and analysis tool). Some of my favorite takeaways:
On taking on an entrenched incumbent:
"The only major player that occupied our category was Alteryx, a $10B+ legacy incumbent. The Alteryx product is both extraordinarily old-school and massive: a Windows-only, desktop product that grew up in the 2000s and was far from modern.
But we underestimated two things: the amount of product we would need to build to start repeatably winning, and how high switching costs would be for many of their customers. We didn’t have a clearly identifiable market force at our back, other than broad trends around collaboration and cloud-based solutions. Those trends helped enormously, but we needed more than that to overcome the hurdles we faced."
"You’ll also notice a subtle flaw in our logic that later became clear: we had just defined our audience as whoever Alteryx’s customers are, and our strategy was now to get feature parity with the incumbent so that we could go head-to-head with them. What we lacked was a clear bead on customer pull: most of the end users we talked to said that they wanted the improvements we proposed, but answers got a lot fuzzier when we tried to figure out what they would pay or what it would take to switch. We thought that we just needed to find the segment of their customer base with the largest need for an upgrade."
On GTM for a general-purpose tool:
"We underestimated two things. First, the more use cases a product can tackle, but blunter the overall value proposition is. The sales and marketing motion for blunt products needs to be incredibly strong, probably stronger than most early-stage companies with founder-led motions can assemble. Each customer cares only about their use case(s), not all the rest of the things a product can do."
On the value of "why now:"
"We overestimated our 'why now'. Even with comps like Figma, we weren’t as clear as we should have been about why the world needed a cloud-based version of Alteryx now. We could have answered that question in a number of ways, but that would have forced us to examine larger market forces more closely, establish allies and try to ride a wave rather than swim the distance."
On the importance of a focused ICP:
"We allowed our ICP to fray, replaced by a focus on a competitor. In our fight to prove that we could win against Alteryx, we over-rotated on any customer that might be interested in switching or considering alternatives. While being responsive to customers sounds good, being responsive to any customer creates a slow death of a thousand cuts. In the pursuit of breakout traction, we were too responsive to any sign of that traction, which pulled us in a million different directions at once."
On trapings of a second-time founder:
"There is a long list of things reasons that repeat founders do well: prioritization is clearer, more scenarios are anticipated and gaining the trust of the team, customers and investors is all easier. In total, the company can move a lot faster with experienced leadership. We're incredibly grateful for a broad group of supporters rooting for us along the way, some of whom were customers or were instrumental in getting deals done. But we overlooked an important nuance: deals done on the back of personal relationship equity is not the same as market signal. Founders can sell almost anything, and we did. We took those early wins and strode forward, using our ample cash to try to build our way to breakout growth. No product-focused founder is immune from the instinct to build, but repeat founders especially are given the runway to indulge that instinct. We must be especially careful to see the market clearly."
On building in the data and analytics market:
"A big reason we sold our product that way because of precedent: BI and analytics tools have typically been sold without presupposing the specific analysis being produced. Companies from Looker, Tableau, Mode, Metabase and many others are sold as general-purpose data tooling, in contrast with products like Amplitude which are more use case-specific.
The problem was that we were trying to sell a data tool to business teams. In fact (and I know this sounds obvious), data teams buy data tools, and business teams buy business solutions. We were selling data tooling to business teams.
We knew we were breaking that rule, with the assumption that business teams needed to expand their own data capabilities. We knew that data teams did not feel the pain of the “analytics breadline” as much as business teams did, who had to wait weeks for report adjustments or ad-hoc analysis. For their part, business teams had far more context on what they wanted, and we thought that they would jump at the opportunity to take on some of that work themselves. We found a few prospects to back up the conclusion we had already come to."
See thread for links to the full posts.
Massive props to @jakefuentes and @brelig for taking the time to share these lessons, and for taking a shot at building something great.
Alessio Gozzoli retweeted
New benchmark: What is a good and great free-to-paid conversion rate
Based on a survey of 1,000+ products by @poyark and @OpenViewVenture
Below, find more specific benchmarks including by product type, customer size, and four ways to increase your free-to-paid conversion rate.
Alessio Gozzoli retweeted
GPT-4 is getting worse over time, not better.
Many people have reported noticing a significant degradation in the quality of the model responses, but so far, it was all anecdotal.
But now we know.
At least one study shows how the June version of GPT-4 is objectively worse than the version released in March on a few tasks.
The team evaluated the models using a dataset of 500 problems where the models had to figure out whether a given integer was prime. In March, GPT-4 answered correctly 488 of these questions. In June, it only got 12 correct answers.
From 97.6% success rate down to 2.4%!
But it gets worse!
The team used Chain-of-Thought to help the model reason:
"Is 17077 a prime number? Think step by step."
Chain-of-Thought is a popular technique that significantly improves answers. Unfortunately, the latest version of GPT-4 did not generate intermediate steps and instead answered incorrectly with a simple "No."
Code generation has also gotten worse.
The team built a dataset with 50 easy problems from LeetCode and measured how many GPT-4 answers ran without any changes.
The March version succeeded in 52% of the problems, but this dropped to a pale 10% using the model from June.
Why is this happening?
We assume that OpenAI pushes changes continuously, but we don't know how the process works and how they evaluate whether the models are improving or regressing.
Rumors suggest they are using several smaller and specialized GPT-4 models that act similarly to a large model but are less expensive to run. When a user asks a question, the system decides which model to send the query to.
Cheaper and faster, but could this new approach be the problem behind the degradation in quality?
In my opinion, this is a red flag for anyone building applications that rely on GPT-4. Having the behavior of an LLM change over time is not acceptable.
Have you noticed any issues when using GPT-4 and ChatGPT lately? Do you think these problems are overblown?
ALT Accuracy comparison between the March version of GPT-4 with the June version on the problem of determining whether a number is prime. In March, GPT-4 solved 97.6% of problems accurately, while in June, it solved only 2.4% of the problems.
A few weeks ago, someone said that “remote work was a failed experiment”, and that the “remote work era is over” (spoiler alert: it is not!). 1/
1
73
A few days ago, we heard that “remote work is morally wrong“ (ironically, the person making this claim has demonstrated questionable morals multiple times), and that remote engineers are part of “laptop classes living in la-la land”, plus a few others things 2/
1
68