WizardLM

🫡 We are sorry for that. It’s been a while since we’ve released a model months ago😅, so we’re unfamiliar with the new release process now: We accidentally missed an item required in the model release process - toxicity testing. We are currently completing this test quickly and then will re-release our model as soon as possible. 🏇 ❤️Do not worry, thanks for your kindly caring and understanding.

🧙‍♀️We not only opensource the models, but also share you how we reach that! 🚀So now let's verify step-by-step to review the whole training method of WizardLM-2 together: -------- 🧵 -------- 👉Motivation First: As the natural world's human-generated data becomes increasingly exhausted ❄️ through LLM training, we believe that: the data carefully created by AI and the model step-by-step supervised by AI will be the sole path towards more powerful AI. Thus, In the past one year, we built a 💻fully AI powered synthetic training system💻 to improve WizardLM-2: Let's see🧐 1. Data Pre-Processing: 1.1 Data Analysis: We use this pipline to get the distribution of different attributes for new source data. This helps us to have a preliminary understanding of the data. 1.2 Weighted Sampling: The distribution of the best training data is always not consistent with the natural distribution of human chat corpus, thus we need adjust the weights of various attributes in the training data based on experimental experience. 2. Progressive Learning: Unlike the common practice of using all data for one-time training, we found that using different data partitions and progressively training stage-by-stage can achieve better results with less data. In each stage, we firstly feed the data slice to following Evol Lab to get more diverse and complex [instruction, response] pairs. Immediately, we leverage a new framework named "AI Align AI" (AAA) which can group multi state-of-the-art LLMs to teach and improve each other. Finally, we successively apply the Supervised Learning, Stage-DPO, and RLEIF to optimize each variant. 2.1 Evol Lab: Evol-Instruct: Recently, we have dedicated significant effort to reassessing the various issues within the original Evol-Instruct method and have initiated preliminary modifications. The new method enables various agents to automatically generate high quality instructions. Evol-Answer: Guiding the model to generate and rewrite responses multiple times can improve its logic, correctness, and affinity. 2.2 AI Align AI (AAA): Co-Teaching: We collect WizardLMs, and various licensed opensource and proprietary state-of-the-art models, then let them co-teach and improve each other, the teaching contains simulated chat, quality judging, improvement suggestions and closing skill gap, etc. Self-Teaching: WizardLM can generate new evolution training data for supervised learning and preference data for reinforcement learning via activate learning from itself. 2.3 Learning: Supervised Learning. Stage-DPO: For more effective offline reinforcement learning, we also split the preference data to different slices, and progressively improve the model stage by stage. RLEIF: We employ instruction quality reward model (IRM) combined with the process supervision reward model (PRM) to achieve more precise correctness in the online reinforcement learning. 🏇More associated details and rethinking will be presented in our upcoming📰paper!

As the natural world's human data becomes increasingly exhausted through LLM training, we believe that: the data carefully created by AI and the model step-by-step supervised by AI will be the sole path towards more powerful AI. Thus, we built a Fully AI powered Synthetic Training System to improve WizardLM-2:

🧙‍♀️ WizardLM-2 8x22B is our most advanced model, and just slightly falling behind GPT-4-1106-preview. 🧙 WizardLM-2 70B reaches top-tier capabilities in the same size. 🧙‍♀️ WizardLM-2 7B even achieves comparable performance with existing 10x larger opensource leading models. The model weights of WizardLM-2 8x22B and WizardLM-2 7B are shared on Huggingface, and WizardLM-2 70B and the demo of all the models will be available in the coming days. huggingface.co/collections/m…

🔥Today we are announcing WizardLM-2, our next generation state-of-the-art LLM. New family includes three cutting-edge models: WizardLM-2 8x22B, 70B, and 7B - demonstrates highly competitive performance compared to leading proprietary LLMs. 📙Release Blog: wizardlm.github.io/WizardLM2 ✅Model Weights: huggingface.co/collections/m…

Excited to see the Evol-Instruct has become a fundamental method to improve instruction-following for state-of-the-art LLMs. Congratulations @01AI_Yi ! This is a series of solid work and driving the open-source community forward.

🎉 Awesome job! Find an impressive LLM Course with more than 13k Github stars⭐️! 🧙‍♂️ Evol-Instruct, as an essential and basic node of the LLM Scientist RoadMap, you could refer to our open-source project for its detailed code and papers. 👉github.com/nlpxucan/WizardLM

The🏆 EvalPlus Leaderboard just updated! New WizardCoder is the best open-source model, and just behind the GPT series. evalplus.github.io/leaderboa…

Details.

🔥 Excited to release WizardCoder-33B-V1.1, the SOTA OSS Code LLM. 🥇79.9% pass@1 on HumanEval, surpasses GPT3.5-Turbo, DeepSeek-Coder-33B-instruct, and Gemini Pro. 🥇78.9% pass@1 on MBPP, comparable with GPT3.5-Turbo, surpasses DeepSeek-Coder-33B-instruct and Gemini Pro. Model: huggingface.co/WizardLM/Wiza… Github: github.com/nlpxucan/WizardLM…

🔥 Excited to release the SOTA WizardMath-7B-V1.1 model, the best 7B math LLM powered by RLEIF. 🥇 83.2% pass@1 on GSM8k, surpasses ChatGPT 3.5, Gemini Pro, Mixtral, and Claude Instant. 🥇 33.0% pass@1 on MATH, surpasses Mixtral, and comparable with ChatGPT 3.5, Gemini Pro. Model: huggingface.co/WizardLM/Wiza… Github: github.com/nlpxucan/WizardLM…

🔥Find and excited to share a new professional survey to understand Math LLMs: 📜Mathematical Language Models: A Survey 👉Paper Link: arxiv.org/abs/2312.07622 🔉"This paper conducts a comprehensive survey of mathematical LMs, systematically categorizing pivotal research endeavors from two distinct perspectives: tasks and methodologies. In addition, our survey entails the compilation of over 60 mathematical datasets, including training datasets, benchmark datasets, and augmented datasets. Addressing the primary challenges and delineating future trajectories within the field of mathematical LMs, this survey is positioned as a valuable resource, poised to facilitate and inspire future innovation among researchers invested in advancing this domain."

🔥🔥🔥Incredible work! OSS-Instruct + 🧙‍♂️Evol-Instruct = 🎩Magicoder Paper: arxiv.org/abs/2312.02120 Code: github.com/ise-uiuc/magicode…

🎉🎉🎉 Excited to introduce the 🏠Home Page (wizardlm.github.io) of our WizardLM Team. ✅WizardLM is an AI research team, our long-term mission is to build AGI for humanity and the world. 🏇We now focus on large language model pre-training and post-alignment.