Risto | Tracardi (@tracardi) header
Risto | Tracardi (@tracardi) profile picture

Indie Hacker | ⭐️ hacking TRACARDI - Open Source Customer Data Platform. #nocode #ecommerce #opensource #customerexperience #CX #headless #cdp

Poland
Joined April 2021
Welcome to Tracardi.
1
1
424
Building LLMs that remember? I’ve launched a subreddit for developers and researchers working on long-term memory architectures for LLMs — retrieval, embeddings, and beyond. Join here → reddit.com/r/llm_memory/
31
Welcome to the wilds of 2025, where the landscape is constantly shifting, AI is running the show… link.medium.com/pRWXJroZVPb
71
The end of the year is approaching, and it’s time for reflections and summaries. Tracardi has achieved another successful year with an impressive organic growth of over 40%. This growth was accomplished without the involvement of investors or marketing efforts.
1
61
I like it. Show me only open-source Customer Data Platforms. Only #tracardi on the list.
43
Risto | Tracardi retweeted
Most people think the cloud saves them money. ​ Not with Kafka. ❌ ​ Storage costs alone are 32 times more expensive than what they should be. ​ Even a miniscule cluster costs hundreds of thousands of dollars! ​ Let’s run the numbers. 🤓 ​ Assume a small Kafka cluster consisting of: ​ • 6 brokers • 35 MB/s of produce traffic • a basic 7-day retention on the data (the default setting) ​ With this setup: ​ 1. 35MB/s of produce traffic will result in 35MB of fresh data produced. 👌 ​ 2. Kafka then replicates this to two other brokers, so a total of 105MB of data is stored each second - 35MB of fresh data and 70MB of copies 🤔 ​ 3. a day’s worth of data is therefore 9.07TB (there are 86400 seconds in a day, times 105MB) ✅ ​ 4. we then accumulate 7 days worth of this data, which is 63.5TB of cluster-wide storage that's needed 😳 ​ Now, it’s prudent to keep extra free space on the disks to give humans time to react during incident scenarios, so we will keep 50% of the disks free. ​ ☝️Trust me, you don't want to run out of disk space over a long weekend. ​ 63.5TB times two is 127TB - let’s just round it to 130TB for simplicity. ​ That would have each broker have 21.6TB of disk. 💡 ​ On to Pricing 🤑 ​ We will use AWS’s EBS HDDs - the throughput-optimized st1s. ​ Note st1s are 3x more expensive than sc1s, but speaking from experience... we need the extra IO throughput. ​ Keep in mind this is the cloud where hardware is shared, so despite a drive allowing you to do up to 500 IOPS, it's very uncertain how much you will actually get. 💡 ​ Further, the other cloud providers offer just one tier of HDDs with comparable (even better) performance - so it keeps the comparison consistent even if you may in theory get away with lower costs in AWS. 👌 ​ st1s cost 0.045$ per GB of provisioned (not used) storage each month. That’s $45 per TB per month. 💡 ​ We will need to provision 130TB. ​ That’s: 🛑 $188 a day 😕 🛑 $5850 a month 😳 🛑 $70,200 a year 🤯 ​ btw, this is the cheapest AWS region - us-east. ​ Europe Frankfurt is $54 per month which is $84,240 a year. 😭 ​ But is storage that expensive? ​ Hetzner will rent out a 22TB drive to you for… $30 a month. ​ 6 of those give us 132TB, so our total cost is: 🟩 $5.8 a day 🟩 $180 a month 🟩 $2160 a year ​ Hosted in Germany too. ​ AWS is 32.5x more expensive! ❌ ​ 39x times more expensive for the Germans who want to store locally. 🇩🇪😥 ​ Let me go through some potential rebuttals now. 🙅‍♂️ ​ 🙅‍♂️ What about Tiered Storage? ​ It’s much, much better with tiered storage. You have to use it. ​ It'd cost you around $21,660 a year in AWS, which is "just" 10x more expensive. But it comes with a lot of other benefits, so it's a trade-off worth considering. ​ I won't go into detail how I arrived at $21,660 since it's a unnecessary. ​ Regardless of how you play around with the assumptions, the majority of the cost comes from the very predictable S3 storage pricing. The cost is bound between around $19,344 as a hard minimum and $25,500 as an unlikely cap. 👌 ​ That being said, the Tiered Storage feature is not yet GA after 6 years... most Apache Kafka users do not have it. ​ 🙅‍♂️ What about other clouds? ​ In GCP, we'd use pd-standard. It is the cheapest and can sustain the IOs necessary as its performance scales with the size of the disk. ​ It’s priced at 0.048 per GiB (gibibytes), which is 1.07GB. ​ That’s 934 GiB for a TB, or $44.8 a month. ​ AWS st1s were $45 per TB a month, so we can say these are basically identical. 👍 ​ In Azure, disks are charged per “tier” and have worse performance - Azure themselves recommend these for development/testing and workloads that are less sensitive to perf variability. ​ We need 21.6TB disks which are just in the middle between the 16TB and 32TB tier, so we are sort of non-optimal here for our choice. ​ A cheaper option may be to run 9 brokers with 16TB disks so we get smaller disks per broker. 💡 ​ With 6 brokers though, it would cost us $953 a month per drive just for the storage alone - $68,616 a year for the cluster. (AWS was $70k) ​ Note that Azure also charges you $0.0005 per 10k operations on a disk. ​ If we assume an operation a second for each partition (1000), that’s 60k operations a minute, or $0.003 a minute. ​ An extra $133.92 a month or $1,596 a year. Not that much in the grand scheme of things. ​ If we try to be more optimal, we could go with 9 brokers and get away with just $4,419 a month. ​ That’s $54,624 a year - significantly cheaper than AWS and GCP's ~$70K options. But still more expensive than AWS's sc1 HDD option - $23,400 a year. ​ All in all, we can see that the cloud prices can vary a lot - with the cheapest possible costs being: ​ • $23,400 in AWS • $54,624 in Azure • $69,888 in GCP ​ Averaging around $49,304 in the cloud. ​ Compared to Hetzner's $2,160... 💀 ​ 🙅‍♂️ Can Hetzner’s HDD give you the same IOPS? ​ This is a very good question. ​ The truth is - I don’t know. 🤷‍♂️ ​ They don't mention what the HDD specs are. ​ And it is with this argument where we could really get lost arguing in the weeds. There's a ton of variables: ​ • IO block size • sequential vs. random • Hetzner's HDD specs • Each cloud provider's average IOPS, and worst case scenario. ​ Without any clear performance test, most theories (including this one) are false anyway. 🤷‍♂️ ​ But I think there's a good argument to be made for Hetzner here. ​ A regular drive can sustain the amount of IOs in this very simple example. Keep in mind Kafka was made for pushing many gigabytes per second... not some measly 35MB/s. ​ And even then, the price difference is so egregious that you could afford to rent 5x the amount of HDDs from Hetzner (for a total of 650GB of storage) and still be cheaper. ​ Worse off - you can just rent SSDs from Hetzner! They offer 7.68TB NVMe SSDs for $71.5 a month! ​ 17 drives would do it, so for $14,586 a year you’d be able to run this Kafka cluster with full on SSDs!!! ​ That'd be $14,586 of Hetzner SSD vs $70,200 of AWS HDD st1, but the performance difference would be staggering for the SSDs. While still 5x cheaper. ​ 🙌 Pro-buttal: Increase the Scale! ​ Kafka was meant for gigabytes of workloads... not some measly 35MB/s that my laptop can do. ​ What if we 10x this small example? 60 brokers, 350MB/s of writes, still a 7 day retention window? ​ You suddenly balloon up to: ​ • $21,600 a year in Hetzner • $546,240 in Azure (cheap) • $698,880 in GCP • $702,120 in Azure (non-optimal) • $700,200 a year in AWS st1 us-east • $842,400 a year in AWS st1 Frankfurt ​ At this size, the absolute costs begin to mean a lot. ​ Now 10x this to a 3.5GB/s workload - what would be recommended for a system like Kafka... and you see the millions wasted. ​ And I haven't even begun to mention the network costs, which can cost an extra $103,000 a year just in this miniscule 35MB/s example. ​ (or an extra $1,030,000 a year in the 10x example) ​ More on that in a follow-up. ​ In the end? ​ It's still at least 39x more expensive. ​ If you think this problem deserves more attention - consider pressing the heart button and sharing to your network. 🙏
12
107
9
609
64,314
Risto | Tracardi retweeted
ChatGPT Prompt Framework: Follow @heyDhavall for more!
73
411
4
2,074
406,644
🌟 Exciting news! We're thrilled to introduce the Tracardi blog! 🎉 Dive into insights on customer data, personalization, and automation strategies that can transform your marketing game. 🔑💡 #Tracardi #CustomerExperience #DataDrivenMarketing blog.tracardi.com/
1
1
33
Risto | Tracardi retweeted
A Kafka in the cloud doing 30MB/s costs more than $110,000 a year. A $1,000 laptop can do 10x that. Where did we go wrong? 👇 The Cloud. Namely - its absurd networking charges 👎 Let’s break it down simply: • AWS charges you $0.01/GB for data crossing AZs (but in the same region). • They charge you on each GB in and out. Meaning each time a GB passes, you pay twice - for the one who sends it (outgoing) and the one who receives it (incoming) • For a normal Kafka cluster with replication factor of 3 and a read fanout of 3x, you are going to be charged: • 2x for 2/3rd of the produce throughput • 4x for 100% of the produce from replicating it • 6x of 2/3rd of the produce throughput for consumption. (but it can get a lot worse - read until the end to see) Simple example: • 3-broker cluster, each in a separate AZ • 3 producers, each in a separate AZ • 3 consumer groups with 3 consumers each, each group with consumers in a separate AZ The producers are producing 30MB/s in total to the same leader. 2/3 producers are in a different AZ, so 20MB/s of produce traffic is being charged at cross-zone rates. 👌 It’s charged both on the OUT (producer’s side) and IN (broker’s side). The leader is replicating the full 30MB/s to both of its replicas. This is again being charged both on the OUT (leader’s side) and IN (follower’s side), for both replication links. (60MB/s) Then, each of the 3 consumer groups has 3 consumers. All consumers read from the leader, with 2/3 in a different zone. This results in 20MB/s of consume traffic charged at cross-zone rates PER GROUP. (60MB/s total) Again charged both on the OUT (broker’s side) and IN (consumer’s side). The total amounts to 140MB/s worth of cross-AZ traffic. Charged both ways. When one MB is $0.00001/s, this means we’re paying $0.0028/s. 🤔 That’s: • $241 a day 😕 • $7500 a month 😥 • $88,300 a year 🤯 It all goes down the drain on network traffic ALONE. 🔥 What about the hardware? Quick napkin math assuming: • 7 day retention • all of the data is on EBS (not using tiered storage since it's not GA yet) • keeping 50% of the disk free for operational purpose (don't ask me what happens if we run out of disk) • the 3 brokers are running modest r4.xlarge instances (kinda overkill but hey, why not) We'd pay: • $19,440/yr for the EBS storage • $6,990/yr for the EC2 instances That’s right - you’re paying just $26.4k/yr for the hardware and 88.3k for the network (3.3x the hardware) For a total of $115k/yr. 💸 I’m not even counting load balancer costs, which could be $12k by some quick napkin math too. How ridiculous is that? 😂 Want it to get more ridiculous? This calculation assumes you’re hosting your own Kafka cluster in the same AWS account. 💡If you use a managed Kafka provider that’s not AWS, or otherwise just another AWS account, you’re typically connecting to them through a public endpoint. AWS then charges all traffic at the cross-AZ $0.01/GB rate internet traffic rate, even if it's in the same AZ. The end result? $113,000 a year for network costs. 💀 For 30MB/s. (!!!) btw - 30 MB/s is absolutely nothing for Kafka... 🤡 It is most often network/disk bounded. Doing 3GB/s is not hard. 👌 The higher throughput you go, the more absurdly large this discrepancy between network and hardware cost becomes. For example - this exact setup could probably do 3x the traffic (90MB/s), assuming storage space isn't a concern. Then you'd have: • $264,000 a year for the cross-AZ rate. 🥲 • $339,000 a year for the internet rate. 💀 Why is this cost (more than 300k a year) and complexity (this calculation) the case when three laptops can run this practically for free? Where did we go wrong? Worth Noting: There are a few optimizations that can be done here: • consumers can use fetch from follower, which results in free read traffic (no cross-AZ charges) in the first example. But the second example would still be charged internet costs. 🤝 • you can avoid internet costs by VPC-peering or Private Link-ing the two AWS accounts. This is largely what most cloud providers do, otherwise it becomes prohibitively expensive. It can be super complex to do. 🔧 • AWS can give you large discounts (up to 90%+ afaict) on the quoted prices, depending on your usage. It’s unclear what customer gets what discount. 💰 And perhaps the best example - you can use an ingeniously-designed product like WarpStream that eliminates all of this complexity and cost. ⭐️ It's no wonder they got acquired after just 13 months of operation.
52
290
33
1,816
387,296