vaex (@vaex_io) header
vaex (@vaex_io) profile picture

vaex

@vaex_io

Out of core dataframes for Python, visualize and explore big tabular data at a billion rows per second. ML ready. https://xitter.gq/t.co/GmNIpbuNlY @maartenbreddels @JovanVaex

github.com/vaexio/vaex
Joined June 2018
  • Tweets747
  • Following173
  • Followers1.7K
Vaex version 4.0 is out! 🚀 with core @ApacheArrow integration Blog: A Hybrid Apache Arrow/Numpy DataFrame with Vaex version 4.0 vaex.io/blog/a-hybrid-apache… #Python #vaex #datascience
A Hybrid Apache Arrow/Numpy DataFrame with Vaex version 4.0

A Hybrid Apache Arrow/Numpy DataFrame with Vaex version 4.0

Integrating Apache Arrow and NumPy seamlessly in a single dataframe library

vaex.io
1
11
2
49
Introducing Reacton 🥳 - a pure Python port of React for ipywidgets! With the proven success of ReactJS, we've implemented a similar API in Python to help create larger applications using ipywidgets. Check out my article at: maartenbreddels.medium.com/a… #React #Python #Jupyter
9
29
5
130
If you are in a hurry, read this thread, which summarizes @JovanVaex 's latest Vaex article. If you want to read the full article: medium.com/@jovan.veljanoski…
8 powerful Vaex DataFrame features you might have not known about

8 powerful Vaex DataFrame features you might have not known about

Vaex is an incredibly powerful DataFrame library in Python capable of processing hundreds of millions or even billions of rows per second…

medium.com
Maarten A. Breddels (@maartenbreddels) profile pictureMaarten A. Breddels@maartenbreddels17 Nov 2022
Learn 8 powerful Vaex DataFrame features you might not have known about. Read @JovanVaex 's article medium.com/@jovan.veljanoski… Or get the short version in this thread 👇🏻
8 powerful Vaex DataFrame features you might have not known about

8 powerful Vaex DataFrame features you might have not known about

Vaex is an incredibly powerful DataFrame library in Python capable of processing hundreds of millions or even billions of rows per second…

medium.com
2
5
Replying to @NickCerny @duckdb @vaex_io
I know duckdb but haven't seen Vaex before. Looks pretty interesting!
1
1
2
Did you know that @vaex_io 's documentation contains a section with guides? Guides can help one better understand certain functionalities of Vaex, especially those that are not common in other DataFrame libraries. vaex.io/docs/guides.html 1/4
1
3
5
Vaex: "One of the 13 best libraries for machine learning and data analysis."
PyQuant News 🐍 (@pyquantnews) profile picturePyQuant News 🐍
@pyquantnews14 Jul 2022
Replying to @pyquantnews
Vaex Vaex is a high-performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets. github.com/vaexio/vaex
GitHub - vaexio/vaex: Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization...

GitHub - vaexio/vaex: Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization...

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀 - vaexio/vaex

github.com
8
College completely failed to teach me data analysis. So I spent over 10,000 hours learning Python. Then, I picked the 13 best libraries for machine learning and data analysis. But unlike college, these won't cost you $120,000. Here they are for free:
134
2,251
65
9,676
Wondering how to make simple and fast @streamlit apps using large datasets? In our latest article we use combine @vaex_io and @streamlit to do just that. We cover: - caching for efficient computations - optimized delayed evaluations - progress bars vaex.io/blog/vaex-plus-strea…
6
15
Awesome writeup :)
Orchest (@orchestofficial) profile pictureOrchest@orchestofficial2 May 2022
📣 New blog post: "Out-of-core processing with Vaex", the second part of our series "The great Python dataframe showdown" orchest.io/blog/the-great-py… Ever heard of @vaex_io but haven't tried it yet? Read on 👇
Image from Orchest (@orchestofficial)
1
1
5
RADIS 0.12.1 was released today ; - more accurate for very thin lineshapes. - @vaex_io is the new default HDF5 library for all platforms (faster!) - use arbitrary @astropy units when rescaling spectra - and many docs improvements But mainly : we welcome 5 new contributors !
2
4
1
12
Jovan and Maarten will showcase @vaex_io, an open-source DataFrame library in Python, tailor-made to allow fast, interactive workflows with datasets that are too large to fit in RAM on a single node. Join us live here go.mlops.community/register on Wednesday at 9am PST/ 5pm UK!

Events | Agentic AI Foundation

The Agentic AI Foundation (AAIF) is a Linux Foundation hosted, vendor-neutral community creating open standards for transparent, interoperable AI agents across ecosystems.

home.mlops.community
3
3
Is your data in the Cloud? No problem: Vaex can open and stream your data directly from your favorite cloud storage provider. Even better - it will only download the parts of the data that you view or use! Learn more about reading data from the cloud at: vaex.io/docs/example_io.html
Image from vaex (@vaex_io)
4
9
It would be interesting to add support for the SIMD instruction sets of various POWER architectures in #xsimd, which underlies SIMD acceleration in Scipy (through Pythran), Apache Arrow, Xtensor, and more. Would folks at @ibm be interested in funding such work?
9
15
Vaex 4.8.0 is out! It includes various performance improvements, improved asyncio support, fancy progressbars and more! See the full changelog at github.com/vaexio/vaex/blob/…
4
17
Turn caching on to speed up computations in Vaex! Especially useful for dashboard backends and interactive EDA. In this example we see that the groupby operation of 1 billion rows is sped up significantly by reusing a previous result from the unique calculation.
2
1
13
Did you know that Vaex can do multiple computations with a single pass over the data? Set “delay=True” to schedule operations to be executed in parallel. The Rich progress bars nicely indicate this with [1] and [2], showing how we aggregate the 1.1 billion rows on a single node!
9
27
See our latest talk on Vaex with @maartenbreddels and @JovanVaex
GOTO (@GOTOcon) profile pictureGOTO@GOTOcon19 Jan 2022
Watch this new #GOTOcph talk where Jovan and @maartenbreddels show how you can access and stream your data directly from the #Cloud — perfect for building cloud services. All of that using just a single machine! youtu.be/ykxhj9xJdWY?list=PL…
3
7
Wordify 2.0 is out! 🚀 Now faster, more interactive, and has more languages. These improvements have been possible thanks to great libraries in the ecosystem, respectively: @vaex_io, @streamlit, and @spacy_io. Thanks to @federicobianchy, @dirk_hovy, and the @MilaNLProc lab!
This Post is from an account that no longer exists. Learn more
3
6
Wow. Neato. Seeing a lot of clever uses of Rich today!
vaex (@vaex_io) profile picturevaex@vaex_io10 Jan 2022
Together with the #Rich library, Vaex can show detailed information about how it executes a computation. Next to the time spent on each step, we also see how many passes over the data it takes (in square brackets). Great for understanding and optimizing your processing pipelines!
1
9
66
Together with the #Rich library, Vaex can show detailed information about how it executes a computation. Next to the time spent on each step, we also see how many passes over the data it takes (in square brackets). Great for understanding and optimizing your processing pipelines!
3
26
4
130
How to analyse 100s of GBs of data on your laptop with Python vaex.io/blog/how-to-analyse-… #Python #vaex #datascience via @vaex_io
How to analyse 100s of GBs of data on your laptop with Python

How to analyse 100s of GBs of data on your laptop with Python

Your laptop is way more powerful than you think.

vaex.io
4
4
Load more