Pinned Tweet
I designed a pipeline that models cross-chain transactions , reconciling the txn from source chain to destination chain.
Agents & analysts can then know exactly what a txn is.
github.com/Freemandaily/crosā¦
Access to blockchain data has hit the rock bottom . Most blockchain analyst finds it hard to access data to work with .
Want to hear from analyst . Which data would you want to work with ?
Week 2 of Data Engineering with Analytic Sages: done.
We went deep on extracting raw blockchain data via RPC:
ā JSON-RPC calls by hand with the request method
ā web3.py for fetching blocks & logs
ā Decoding raw EVM hex blobs into human-readable transfers
š A quick look into our Blockchain Data Engineering class!
From exploring Docker to getting hands-on with the tools and commands that power modern development, our learning journey continues.
At Analytic Sages, we believe the best way to grow in tech is to go beyond theory and learn by doing.
Every session brings us one step closer to building practical skills for the future of data and blockchain.
Want to be part of the next learning experience? š
š Join the next cohort waitlist today: analyticsages.io/programs/blā¦
Take your first step towards building your Blockchain Data Engineering skills.
#AnalyticSages #BlockchainDataEngineering #Docker #DataEngineering #BlockchainTechnology #TechEducation #Web3
Freemanš¼ retweeted
Just built my first end-to-end data pipeline š
DefiLlama API ā pandas ā PostgreSQL, fully containerized with Docker. Hereās how it came together:
1ļøā£ Extract: pulled @monarq_mgmt's daily TVL history from the DefiLlama API with Pythonās requests
2ļøā£ Transform: used pandas to keep date and totalLiquidityUSD and convert Unix timestamps to UTC datetimes
3ļøā£ Load: wrote the DataFrame to a Postgres table with SQLAlchemy
4ļøā£ Containerize: packaged the pipeline in a Dockerfile, with dependencies managed by uv
5ļøā£ Verify: queried the table with pgcli to confirm the data landed
Big thanks to @AnalyticSages and @Freemandayly for the structure and guidance.
Repo: github.com/tobeydev/assignmeā¦
Freemanš¼ retweeted
If you are a data analyst, the best thing you can do, as you work with an AI, is make a skill out of Kimball's Dimensional modelling book.
I think AI is actually quite bad at making data models. IMO, there is something very human in making and accessing data models that AI can't capture.
If you read (well, the first few chapters) of the book, Kimball is really, really particular about designing the table to be intuitive to the user and never prioritizing hard data modelling rules.
My personal rule to the AI when making tables is that it has to be so intuitive that the user don't even need docs and just get cracking and exploring the tables
Likewise, in this new weird paradigm, also consider designing tables for the AI for the easiest possible access.
Idk what I've cooked with my Cursor, but it keeps creating new data marts out of my gold tables instead of writing queries to get some dataš
I get what she is saying. Historically, you have
- Human slop cannon
- People who think all code should be Good Code, who are very useful because of talent but requiring management because of the constant desire to boil the ocean / move at the speed of a snail
- People who think important (seams, data modeling, rules engines, stuff like that) code should be Good Code, and its fine if other code is slop
The most valuable persona is the third. The first will destroy a business due to overrunning tech debt, the second will tank a business due to neverending refinement. The third understands those forces need to be kept in balance.
I have a huge amount of respect for Lauren, I know folks who have worked with her and say she is absolute top tier. I know she spent a huge amount of time and effort working through Dune for example. There was probably a non trivial amount of code involved there, and it probably is the kind of thing that needed to be Good Code.
"Slop is a slur" I think is cope. "Most code in the before-times was slop and that was ok" I think is reasonable and defensible. "All code is slop" I think is an overreach, and dismisses a very important part of this conversation.
Week 1 of Data Engineering with AnalyticSages ā
We introduced Docker containers, essential in modern software development. Students learned how they work and built a Dockerfile and pipeline setup.
#DataEngineering #Docker
Registration is Now Open š
Build Your Blockchain Data Engineering Stack.
Learn how modern data systems workāand explore the tools used to extract, store, transform and deliver blockchain data.
Over 10 weeks, you'll explore technologies across:
⢠Programming & Data
⢠Blockchain Data Extraction
⢠Data Storage
⢠Data Transformation
⢠Workflow Orchestration
⢠Real-Time Data
⢠Data Products & Deployment
This isn't just about learning tools.
It's about understanding how they work together to power real data infrastructure.
š
Registration starts: 4th September, 2026
ā³ Registration closes: 3rd October, 2026
Build practical skills. Explore modern infrastructure. Start building.
š Register now : analyticsages.io/programs/blā¦
When i started data engineering , i used postgres so much, now i cant design data infrastructure without thinking of cloud infra
When I started Coinbase, getting a bank to work with us at all was nearly impossible.
Now Citi is partnering with us to bring stablecoins to institutions.
Weāve come a long way from 2012. Thank you @Citi.
Freemanš¼ retweeted
Blockchain data is becoming more important.
But working with it takes more than just understanding blockchain.
You need to know how to collect, process, store and analyze on-chain data.
Thatās exactly what weāre exploring in our Blockchain Data Engineering Program.
Registration ends October 3rd.
Secure your spot now!!
analyticsages.io/programs/blā¦
See you in class.
#BlockchainData #DataEngineering #Web3 #Blockchain #AnalyticSages
Developing a tech skill should scare you more than the impression you get when you see a tech guyās workstation setup.
Now that weāve closed shop.
Hereās everything you need to know.š
web3bridge.substack.com/p/isā¦
Freemanš¼ retweeted
08 days to go.
Blockchain Data Engineering kicks off 6th October 2026.
If youāre a data engineer thinking, āOkay, but how do I actually work with on-chain data?ā
this is for you.
Weāre getting hands-on with:
ā Python + Web3.py
ā PostgreSQL + ClickHouse
ā dbt
ā Apache Airflow
ā Docker + Git
Real tools, real workflows, and the skills to build reliable blockchain data pipelines.
Register now:
analyticsages.io/programs/blā¦
Freemanš¼ retweeted
14 days until we kick off Blockchain Data Engineering on 6th October 2026.
This isnāt another theory-heavy course.
Itās a practical journey into building production-ready data pipelines for the blockchain space powered by the tools that actually matter:
Python | PostgreSQL | Web3.py | dbt
Apache Airflow | ClickHouse | Docker | Git
If youāre serious about becoming a Blockchain Data Engineer, this is your moment.
Spots are limited.
Register now š
analyticsages.io/programs/blā¦
#Blockchain #Web3 #DataEngineering
This is such a great project . I found it helpful when i was experimenting with S3
AWS en local, sin cuenta ni tokens y sin suscripciones.
TambiƩn disponible para Azure y Google Cloud.
ā S3, Lambda, DynamoDB, RDSā¦
ā Para practicar y para CI
ā Docker compose y listo
ā github.com/floci-io/floci
What's your experience with BigQuery?
What your take on lakehouse compared to BigQuery data warehouse
What is you experience with data lakehouse and data warehouse ?
#Dataengineering
It Was a pleasure discussing blockchain data infrastructure/pipeline with @AnalyticSages community .
#DataEngineering #Blockchain
Join us tomorrow as we discuss data infrastructure for blockchain data.
Who builds it? What does it take to build data infrastructure for blockchain?
Come join us
Everyone wants to build on blockchain.
But hereās the question:
Who builds the data infrastructure that makes it all possible?
Blockchain is more than smart contracts, protocols, and dApps. Behind every meaningful on-chain insight is data infrastructure that helps us collect, process, analyze, and make sense of blockchain data.
Join us tomorrow as we unpack the people, systems, and skills powering the data layer of Web3.
TOPIC:
Everyone Wants to Build on Blockchain. Who Builds the Data Infrastructure?
Date: September 4, 2026
Time: 7:00 PM WAT
Space: xitter.cf/i/spaces/1rGmqpbā¦
Speakers
⢠@apostleoffin ā Quant/Data Engineer
⢠@oladeeayo ā Data Analyst
⢠@Freemandayly ā Blockchain Data Engineer.
Whether you're a data analyst, engineer, researcher, developer, or simply curious about blockchain data, this conversation is for you.
Come learn, Come ask questions, Come discover what happens behind the blockchain data.
Set your reminder, you don't want to miss this.
Freemanš¼ retweeted
Everyone wants to build on blockchain.
But hereās the question:
Who builds the data infrastructure that makes it all possible?
Blockchain is more than smart contracts, protocols, and dApps. Behind every meaningful on-chain insight is data infrastructure that helps us collect, process, analyze, and make sense of blockchain data.
Join us tomorrow as we unpack the people, systems, and skills powering the data layer of Web3.
TOPIC:
Everyone Wants to Build on Blockchain. Who Builds the Data Infrastructure?
Date: September 4, 2026
Time: 7:00 PM WAT
Space: xitter.cf/i/spaces/1rGmqpbā¦
Speakers
⢠@apostleoffin ā Quant/Data Engineer
⢠@oladeeayo ā Data Analyst
⢠@Freemandayly ā Blockchain Data Engineer.
Whether you're a data analyst, engineer, researcher, developer, or simply curious about blockchain data, this conversation is for you.
Come learn, Come ask questions, Come discover what happens behind the blockchain data.
Set your reminder, you don't want to miss this.
I designed lakehouse locally using Iceberg + Trino + dbt
No cloud needed. Everything runs on one machine with LocalStack standing in for AWS S3.
This demonstrates how lakehouse manages data
github.com/Freemandaily/crypā¦
Iāve been reading a lot about Apache Iceberg, and itās been a wonderful experience learning how data can be stored and efficiently retrieved because of the structure and organisation Iceberg provides.
#iceberg
The easy path in data analytic is dumping your data into BigQuery and query it directly.
BigQuery charges you every time you READ and WRITE. At scale, that costs a fortune.
To cut the cost, store raw data in S3/gcs as Parquet, then use Spark + Apache Iceberg for transformation
The real power of Iceberg is that it brings warehouse-like capabilities to cheap object storage , reliable ACID writes, schema and partition evolution, and time travel to query historical versions of your data.
All while your data stays in S3/GCS as Parquet.



















