bohemian, private eye, hacker, designer, teacher, data scientist.
Chicago
Joined October 2008
- Tweets882
- Following517
- Followers474
- Likes187
Irmak Sirer retweeted
Martin Scorsese talking to his parents about sauce is canon
Italianamerican (1974)
Scorsese
Irmak Sirer retweeted
# On the "hallucination problem"
I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.
It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.
At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.
All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
</rant> Okay I feel much better now :)
Irmak Sirer retweeted
ChatGPT can now see, hear, and speak. Rolling out over next two weeks, Plus users will be able to have voice conversations with ChatGPT (iOS & Android) and to include images in conversations (all platforms).
openai.com/blog/chatgpt-can-…
Irmak Sirer retweeted
Yann LeCun is obviously a legend but I found this tweet to be quite misinformed.
The whole point of "emergent abilities" such as few-shot prompting and chain-of-thought prompting, is that we clearly *did not* explicitly train or fine-tune them into the model. These abilities require scaling up to a lot of parameters, which is the reason we call them "emergent"---they are quantitive behavior shifts that come from qualitative changes. At least this is how researchers use the term.
All we did was train a language model with a lot of parameters on a big dataset. These abilities were not intentionally added into the model during training---rather, people found them via evaluations after the model was trained. A great pointer to this is Jacob Steinhardt's post: bounded-regret.ghost.io/emer…
What's also significant is that we often aren't able to predict these emergent phenomena ahead of time, and this has safety implications since risks can also be emergent.
Also, I don't know why finetuning is mentioned, because these abilities are clearly in the base model, which is should be apparent from papers before RLHF became popular... (e.g., PaLM-1, GPT-3 paper)
But even if you insist to take a finetuning as an example, ChatGPT RLHF was originally done in English, and it extended to many other languages. This multilingual ability was not "finetuned" into the model, which is a pretty good example of zero-shot generalization.
Please correct me if I'm wrong, but the technical use of the term definitely still holds.
Mustafa's tweet is also a bit misinformed, in my circles people don't use "emergence" to mean autonomy or agency. Again, what AI researchers usually mean is that it is a qualitative change that arises from quantitative changes.
Irmak Sirer retweeted
National Lab (LBNL) results support LK-99 as a room-temperature ambient-pressure superconductor.
Simulations published 1 hour ago on arxiv support LK-99 as the holy grail of modern material science and applied physics.
(arxiv.org/abs/2307.16892)
Here's the plain-english explanation:
- The simulations modeled what the original Korean authors proposed was happening to their material - where copper atoms were percolating into a crystal structure and replacing lead atoms, causing the crystal to strain slightly and contract by 0.5%. This unique structure was proposed to allow this amazing property.
- @sineatrix from Lawrence Berkeley National Lab simulated this using heavy-duty compute power from the Department of Energy, and looked to see what would happen to the 'electronic structure' of this material, meaning, what are the available conduction pathways in the material.
- It turns out that there are conduction pathways for electrons that are in just the right conditions and places that would enable them to 'superconduct'. More specifically, they were close to the 'Fermi Surface' which is like the sea-level of electrical energy, as in '0 ft above sea-level.' It's believed currently that the more conduction pathways close to the Fermi surface, the higher the temperature you can superconduct at (An analogy might be how its easier for planes to fly close to the surface of the ocean due to the 'ground effect' that gives them more lift.)
This plot in particular shows the 'bands', or electron pathways, crossing above and below the Fermi surface.
- Lastly, these interesting conduction pathways only form when the copper atom percolates into the less likely location in the crystal lattice, or the 'higher energy' binding site. This means the material would be difficult to synthesize since only a small fraction of crystal gets its copper in just the right location.
This is insanely bullish for humanity.
Irmak Sirer retweeted
As a former materials physicist, I am genuinely unsure of what to make of this superconductivity paper.
🧵 👇
Irmak Sirer retweeted
You can tell the authors of the new room temperature superconductor paper believe in their results because they published two papers about it concurrently: one with six authors, and one with only three.
The Nobel prize can be shared by at most three people.
Irmak Sirer retweeted
Today might have seen the biggest physics discovery of my lifetime. I don't think people fully grasp the implications of an ambient temperature / pressure superconductor. Here's how it could totally change our lives.
Irmak Sirer retweeted
This is a strangler fig tree, Inside this tree is a hollow space where a different tree stood
The strangler fig's seeds have made their way into the canopy of a host tree and germinated. As the fig's roots grow, they cascade down the trunk,
Once they are in the ground it competes for nutrients and water with the host. Then It gradually tightens its grip around the host tree
This process hinders the flow of water and nutrients, causing the host tree to weaken and eventually die. Over time, the strangler fig takes over completely, once the process is complete, the original tree decomposes, a hollow centre is all that remains to mark its existence
Irmak Sirer retweeted
GPT-4 "discovered" the same sorting algorithm as AlphaDev by removing "mov S P".
No RL needed. Can I publish this on nature?
here are the prompts I used chat.openai.com/share/95693d…
(excuse my idiotic typos, but gpt4 doesn't mind anyways)
Sorting algorithm underpins all critical softwares. DeepMind's AlphaDev speeds up sorting small sequences (3-5 items) by 70%.
Key takeaways:
* The main RL algorithm is based on AlphaZero that originally played Go, Chess & Shogi. Same idea applies to searching programs!
* Instead of optimizing over C code, they optimize assembly code instead. It's a deliberate choice to go low-level to squeeze out every instruction saving.
* The assembly code is then reverse-engineered by hand to C, and open-sourced in LLVM.
* Even though the representation network uses transformer, it is NOT a foundation model. The whole pipeline only works on sorting, and has to be re-trained for other tasks like hashing.
Irmak Sirer retweeted
Elevator To The Gallows (1958) score was composed and performed by Miles Davis in a one session recording while he watched a screening of the film. He took notes while watching the rough cut and then invited 4 fellow musicians without preparation to record it in one go. Genius.
Irmak Sirer retweeted
ALIEN was released 44 years ago today. One of the definitive science fiction/horror movies, its influence is still seen to this day. The behind-the-scenes story of how the film was made is a fascinating one...
A THREAD
1/38
Irmak Sirer retweeted
I know it’s getting banned and all, but there’s an account on tiktok that just strips the laugh track out of Seinfeld and sets the show to Laura Palmer’s theme and it’s the singles greatest contribution to comedy in 30 years.
