It doesn't have neither the context length nor can it focus on every line in a huge codebase. Give it a tight scope and it'll do wonders.
We're still good (meanwhile), because there's place for us to scope and organize and design, but not because we misuse a tool and blame it.
im shocked at how poorly the general public is receiving ai minecraft. ive never seen a work of sci-fi that predicted that people would react to powerful new technology as though it was unimpressive+ unimaginative + idiotic
We applied GPT-4 to interpretability — automatically proposing explanations for GPT-2's 300k neurons — and found neurons responding to concepts like similes, “things done correctly,” or expressions of certainty. We aim to use Al to help us understand Al: openai.com/research/language…
I do believe that sometimes it's worth it to incur technical debt, if you make it pay off later. Same as you'd take a loan to expand your business because the profit is higher than the interest. And maybe even never pay it back, because there's better cost savings to prioritize.