Replying to @humanloop
@humanloop is a company that provides tools to help businesses customize and fine-tune their GPT-3 models for higher performance and lower cost.
Replying to @llama_index
@gpt_index is a library that makes it easier to use import data from external sources such as Notion, Airtable, and others and use them with your LLMs.
Find out more about tools like @gpt_index at tuningfork.ai/resources/llam…
Replying to @LangChain
@langchain is a library that helps developers create powerful applications using large language models. It provides six modules for creating applications such as chatbots, data augmented generation, question answering, and more.
Users can get started by checking out the Quickstart Guide. python.langchain.com/en/late…
The library has support for Python and Typescript! 🎉
Find out more at: tuningfork.ai/resources/lang…
If you're looking for a low-code/no-code approach to using Langchain, consider using Langflow.
Langflow provides a way for users to prototype and iterate on your chains, agents, etc without using code.
Find out more at tuningfork.ai/resources/lang…
For teams building AI-powered apps, one question comes up over and over:
How do I know if the results my users are getting are any good?
When using LLMs, the problem is uniquely tricky. Here are the best tools for feedback collection and model evaluation that we've seen 👇
3. @langchain
The popular orchestration library has some relatively new but *very* useful evaluation tools. For example, it can automatically tell you if two generated answers are close enough to be "correct", even if they're articulated differently.
Find more resources for evaluation and testing at
tuningfork.ai/resources?filt…
A way to get around the prompt size limitations is to use embeddings and vector indices. Embeddings help us encode the semantic meaning of text. We can use embeddings to compare text to determine how similar they are in meaning, even if they don’t necessarily share any keywords.
Vector Indices can then be stored locally or on a vector database (such as @pinecone, @trychroma, etc) to avoid making redundant requests to create embeddings.
For developers diving into the LLM world, prompt management quickly becomes a big issue. You've done all that prompt engineering, how do you track which templates are doing better than others?
Here are the best prompt management tools and libraries we've found 👇
@PromptableAI (tuningfork.ai/resources/prom…) has a set of prompt recording and comparison tools, as well as the ability to manage prompt deployments. Super nice for managing prompt engineering in production.
Finally, Baseplate (tuningfork.ai/resources/base…) has a set of prompt comparison tools, paired with analytics that include other variables like temperature. Probably the nicest UI we've seen for that so far.
As you begin communicating with LLMs, you'll find that you need to keep track of all the prompts you've made, and it can quickly become messy.
Here are a few tools that can help you organize, track, and add versioning to your prompts: 🧵👇
PromptSource is a platform that enables users to create and share natural language prompts, which are stored in Jinja templating language files. It also provides an API to access thousands of existing and newly created prompts.
