I’ve finally understood why hallucinations in LLMs are a feature not a bug. Deep neural networks are function approximators, which means they map input -> output. They learn this function via data. The usefulness of LLMs is that they can give an *approximate* answer to inputs never before seen in training data (like write a Haiku on Punjabis). If we wanted an exact answer, we could have used a database and it would have returned a null result. A database never hallucinates. But LLMs never return a null result as they approximate output for any input and that’s what makes them useful most of the times (but when we’re not happy, we call such results hallucinations)