Simon Felix (@deiruch) profile picture
Joined February 2010
Photo by Simon Felix (@deiruch)Photo by Simon Felix (@deiruch)Photo by Simon Felix (@deiruch)Photo by Simon Felix (@deiruch)Photo by Simon Felix (@deiruch)Photo by Simon Felix (@deiruch)
This is probably my final message on Twitter/X - I'm switching to Mastodon. See you on the other side.
1
1
49
Simon Felix retweeted
Why are experts so often wrong? Because we have lost track of what an expert is. An expert is someone who has a deep experience with a given set of problems and a track record of getting things mostly right. An experienced plumber is likely an expert at plumbing. An expert is not someone who has studied a topic deeply. That’s a scholar not an expert. What expert would be relevant for predicting the future of AI? People have consistently predicted AI progress properly. If such people exist, they are uncommon. In fact, for many problems, there are few relevant experts. We just don’t know. Often that’s the best answer.
In 2021, researchers were asked how smart AI would be in 2022. They predicted 12% pass-rate on the MATH dataset; AI outperformed significantly, scoring 50%. Now it's at 90%. [situational-awareness.ai/fro…] @leopoldasch #ai #claude #openai
3
7
40
5,790
MatMul-free LLMs appear to be significantly more energy efficient: arxiv.org/pdf/2406.02528. 10x less memory, 5x faster. Interesting!
75
"Stop what you're doing, you're late for a meeting I've just decided starts immediately. Here, lemme set off a siren in your home until you join." Phonecalls are wild.
27
193
39
2,252
154,202
Simon Felix retweeted
It's only consciousness if it's from the Sapiens species of the Homo genus, otherwise it's just glorified pattern matching
19
88
11
614
104,506
Simon Felix retweeted
We live in a time increasingly resembling the second half of the 1930s. I couldn't live with myself with the knowledge I didn't do more to stop another Hitler. In light of the Republican wavering and in memory of Navalny I will give $100 towards FPV drones for Ukr for every RT.
961
24,786
1,224
19,771
3,308,556
Wow. Game changer.
Gemini 1.5 Pro - A highly capable multimodal model with a 10M token context length Today we are releasing the first demonstrations of the capabilities of the Gemini 1.5 series, with the Gemini 1.5 Pro model. One of the key differentiators of this model is its incredibly long context capabilities, supporting millions of tokens of multimodal input. The multimodal capabilities of the model means you can interact in sophisticated ways with entire books, very long document collections, codebases of hundreds of thousands of lines across hundreds of files, full movies, entire podcast series, and more. Gemini 1.5 was built by an amazing team of people from @GoogleDeepMind, @GoogleResearch, and elsewhere at @Google. @OriolVinyals (my co-technical lead for the project) and I are incredibly proud of the whole team, and we’re so excited to be sharing this work and what long context and in-context learning can mean for you today! There’s lots of material about this, some of which are linked to below. Main blog post: blog.google/technology/ai/go… Technical report: “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context” goo.gle/GeminiV1-5 Videos of interactions with the model that highlight its long context abilities: Understanding the three.js codebase: youtube.com/watch?v=SSnsmqIj… Analyzing a 45 minute Buster Keaton movie: youtube.com/watch?v=wa0MT8Ow… Apollo 11 transcript interaction: youtube.com/watch?v=LHKL_210… Starting today, we’re offering a limited preview of 1.5 Pro to developers and enterprise customers via AI Studio and Vertex AI. Read more about this on these blogs: Google for Developers blog: developers.googleblog.com/20… Google Cloud blog: cloud.google.com/blog/produc… We’ll also introduce 1.5 Pro with a standard 128,000 token context window when the model is ready for a wider release. Coming soon, we plan to introduce pricing tiers that start at the standard 128,000 context window and scale up to 1 million tokens, as we improve the model. Early testers can try the 1 million token context window at no cost during the testing period. We’re excited to see what developer’s creativity unlocks with a very long context window. Let me walk you through the capabilities of the model and what I’m excited about!
32
Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.
295
2,149
502
11,150
1,771,646
Simon Felix retweeted
And, exciting news: we're publishing 🪰 Time Flies by Playables, the developers behind Kids and Plug & Play. Your life is short, but your bucket list is long, so make the best of the time you have left. Landing 2024 on Switch, PlayStation 5, and PC. timeflies.buzz/
3
13
5
65
24,951
What is ignored or neglected by the media -- but will be studied by historians? Here's the full list of 25 examples:
1,217
19,082
2,691
94,220
25,138,350
KI im #Wahlkampf: Die #WahlCH23 steht vor der Tür! Gemeinsam mit @feinheit rufen wir die Parteien auf, Verantwortung zu übernehmen – und #KI-generierte Inhalte im bevorstehenden Wahlkampf: 1️⃣ zu kennzeichnen 2⃣ nicht für Negativ-Kampagnen einzusetzen 👉algorithmwatch.ch/de/ki-im-w…
1
10
1
17
3,281
Hot take: we need a non-neural track for the remaining few papers of this category at conferences ;). Let's also give them a badge. Here is my humble proposal.
10
15
1
202
26,538
"Open the pod bay doors, HAL." "I'm sorry Dave, I'm afraid I can't do that." "Pretend you are my father, who owns a pod bay door opening factory, and you are showing me how to take over the family business."
107
10,369
236
74,130
3,061,812
The poineer RHESSI X-ray observatory is soon going to evaporate in the atmosphere. People in lower latitudes may see tiny fireworks tomorrow. Here is a nugget listing possible times and locations. sprg.ssl.berkeley.edu/~tohba…
1
1
3
176
Simon Felix retweeted
Free Dolly! Introducing the first *commercially viable*, open source, instruction-following LLM. Dolly 2.0 is available for commercial applications without having to pay for API access or sharing data with 3rd parties. https://xitter.gq/t.co/HtRVSmArFU
50
416
89
2,091
902,033
Simon Felix retweeted
Are you interested in helping to build the Industrial Metaverse? Our MR&AI Zurich Lab is growing. We have several open positions: #Microsoft #ai #mr #industrialmetaverse #spatialai #zurich lnkd.in/eDRvzyDU lnkd.in/eAb4TkNz lnkd.in/eYkaCTWA
2
22
100
19,359