Pinned Tweet
💥 We're shipping a new feature to the 🤗 Hub's datasets - statistics over numbers, labels and texts. Get insights about the data without downloading it:
🧮 View the distribution of numerical data
📊 Check for class imbalance
🔎 Look if you have outliers in texts
Polina Kazakova retweeted
🤖 Open LLM Leaderboard Tech Fact!
When you submit a model, you probably know that the proper way to check its status is to go to the Requests dataset, find your request file, and look at the "status" field. But guess what? We've got a cool new feature for you! 😎
🔎 Starting today, you can use the SQL console in Requests! You can find your model by its name and check its status, or you can filter models by "sender" and see the status for all models you have submitted. It's totally up to you how to use these new feature!
🔔 Reminder: Need us to check logs or resubmit your model? Please, open a discussion on the Open LLM Leaderboard page and send us your request file. We've got you covered!
🚀 Oh, and by the way, we have 1.83k requests now, and we're reaching towards 2k!
👏 Huge shoutout to @polinaeterna for suggesting Requests dataset improvements and @calebfahlgren for integrating the SQL search!
🔗 Requests dataset: huggingface.co/datasets/open…
🔗 Open LLM Leaderboard: huggingface.co/spaces/open-l…
Polina Kazakova retweeted
Check out the new Structured #Wikipedia dataset by @Wikimedia Enterprise: abstract, infobox, structured sections, main image,...
Currently in early beta (English & French). Explore it and give feedback: huggingface.co/datasets/wiki…
More info: enterprise.wikimedia.com/blo…
#AI #NLP
Now you can run SQL queries over 200k datasets directly on the 🤗 Hub
For example, this gives you insights on the most popular base models from Open LLM Leaderboard 📊 and their scores right in your browser:
NEW SQL Console on @huggingface Datasets Viewer🤗
🔸Run SQL on any public dataset
🔸Powered by @duckdb WASM running entirely in the browser
🔸 Share your SQL Queries via URL with others!
More coming soon!
And then you can share your query!
huggingface.co/datasets/open…
5
140
Polina Kazakova retweeted
In PyCharm 2024.2, you can inspect @huggingface Datasets library data as an interactive dataframe! 📊 Enjoy features like chart view, pagination, sorting, and exporting tables, just like you do with #pandas, #Polars, #PyTorch, and #TensorFlow!
Learn more: jb.gg/hf-interactive-datafra…
Polina Kazakova retweeted
For our StreamingDataset users: We're thrilled to announce support for storing MDS datasets in @huggingface. S/O to @orionweller for the contribution!
Check out the docs here: docs.mosaicml.com/projects/s…
Polina Kazakova retweeted
Masader now supports inspecting datasets's contents using the @huggingface dataset viewer. Thanks to @severo_dev for the support.
Check the viewer at Masader: arbml.github.io/masader/
Polina Kazakova retweeted
One fundamental @huggingface is to try to help shape good practices in AI development. Thinking through data (huggingface.co/datasets):
What data-related features would you like to see?
What would you be interested in for data discoverability? Exploration? Quality analysis?
Polina Kazakova retweeted
We just integrated dataset viewer in AutoTrain 💥 So, now you can look into your dataset, identify correct splits and columns before training the model, without leaving the page 🚀
Polina Kazakova retweeted
Just added the @huggingface embed to my EV Charge Finder app. It's pretty neat to view a dataset inside an app.
Polina Kazakova retweeted
Using the new viewer iframe support, I built a quick space that allows you to explore all the datasets inside a collection easily. You can duplicate this Space to create a Space to explore datasets in any HF collection!
huggingface.co/spaces/davans…
🗃️This also means that you can embed Dataset Viewer for any! dataset into your 🤗 space
Cool weekend update from the @huggingface datasets team,
You can now embed our viewer on any webpage 😎
(Look for the "Embed" button.)
Let us know what you think!
It might be hard to follow the numerous dataset releases that happen every day. That’s why we are working on improving datasets’ discoverability 🧐🔍 on the 🤗 Hub so that you can find **the data you need**.
Meet new interface to search for specific:
Polina Kazakova retweeted
Run SQL on any @huggingface Dataset in the Browser
• Powered by @duckdb WASM 🦆
• Open Source Chrome Extension
• Query different splits and configs
Polina Kazakova retweeted
My lecture "Behind the scenes of LLM pretraining" at @Stanford's CS25: Transformers United class is now available on Youtube: youtube.com/watch?v=jm2hyJLF…
It covers the following aspects:
- The scaling laws and how much data is needed for pretraining
- Where to find this data (web, code and synthetic)
- How to filter it
The slides are also available here:
docs.google.com/presentation…
Polina Kazakova retweeted
Community-centric and awesome: @huggingface and @Wikimedia 🤗 I wrote an article on how we can advance ML with diverse datasets from @Wikipedia, why and how to create more Wikimedia datasets on Hugging Face, and community consent. Have a look! huggingface.co/blog/frimelle…
Polina Kazakova retweeted
We are (finally) releasing the 🍷 FineWeb technical report!
In it, we detail and explain every processing decision we took, and we also introduce our newest dataset: 📚 FineWeb-Edu, a (web only) subset of FW filtered for high educational content.
Link: hf.co/spaces/HuggingFaceFW/b…






















