Dylan Ander | CRO & SplitTesting

We used to prioritize A/B tests with ICE, and we stopped. Here’s why. ICE asks you to score Impact, Confidence, and Ease on a scale of 1 to 10. But what does a “7 impact” mean? No guideline defines it, so every score is a guess. Confidence is worse because two people on the same team will give the same test two different scores, depending on their experience and their biases. And Ease breaks the whole thing, since a low-effort test gets treated as a high priority just for being easy. The big tests that could move revenue get pushed down the list, and teams end up testing button colors. So we built IMPACT, where every factor is scored against criteria you can verify, rather than a feeling. I - Importance How much does this matter to the business if it wins? → Is it in a critical part of the funnel? → Does the page get enough traffic to matter? → Does it move the focus metric? M - Motivation Is it backed by real evidence about your customers? → Supported by qualitative findings → Supported by quantitative findings → Based on previous test learnings If there’s no research behind it, it scores zero. P - Potential How visible is the change? → Above the fold → Noticeable within 5 seconds → Adds or removes an element A - Alignment Does it match what the brand is focused on right now? → Impacts the focus metric or current business goal Those four score the value of a test, and the last two score its cost. C - Creative How much design and copy work does it take? Design ranges from “no design, or swapping an image” to “entire page redesign,” and copy from “2-3 words” to “entire page from scratch.” T - Technicality How much dev work does it take? The range runs from “no development, like changing a headline” to “entire page build.” Here’s what changes: effort can’t push a test to the top on its own anymore. A big build with strong research behind it, sitting in a critical part of the funnel, still ranks first, while a quick tweak with nothing behind it doesn’t. And when two people score the same idea, they land on roughly the same number, because they’re checking boxes instead of guessing. Your testing roadmap is only as good as the way you rank it.

I didn't think CRO could get better

Hot Take: I think AEO/GEO optimization is a bit of scam.

This might be a stupid question … But what can Grok Bot do that Claude Code or Codex can’t? Is it just way better at completing tasks on the internet?

3 founders. 1 room. No script. @DylanAnder (Heatmap) @ezrafirestone (Zipify) @sterlingmsnow (Redo) 60 minutes on what changes (and what doesn't) when AI does the shopping. Thu, Oct 15 · 2 PM ET Seats are limited. Grab yours before they're gone. tinyurl.com/hm-oct15-webinar

We’ve started making AI animation ads at Zagged. The feedback speaks for itself. These ads are ripping for clients spending >$200k/mo. If you run a brand and haven’t tested this format yet, let’s chat.

Never met a vibe coder using MySQL. Never met a vibe coder using MongoDB. Never met a vibe coder using SQLite. Never met a vibe coder using Oracle. Never met a vibe coder using DynamoDB. Never met a vibe coder using Cassandra. ​Vibe coding is literally just: Next.js + Tailwind + Postgres + Claude

question for my app people. if you have a pixel on your site how do i disable it? please explain in comments

Distribution will be the only moat left

Ya’ll are not prepared for the @ecomfounders Q4 Summit today. No expense was spared. So wild to see it grow and upgrade every year. Can’t wait to see you guys soon!