developer/programmer/ai nerd/🍁
Canada
Joined October 2023
- Tweets6.3K
- Following2.4K
- Followers551
- Likes10.6K
g023 retweeted
PROMPT
Build a single self-contained HTML file (three.js r160 from cdn.jsdelivr via importmap) that renders a detailed voxel diorama: "Liberty Island & New York, autumn 1886 — dedication day of the Statue of Liberty", as a floating island in a warm hazy sky.
STYLE
- Chunky voxels with slightly beveled (chamfered) edges, vertex colors with per-voxel color jitter and baked ambient occlusion.
- Golden-hour light: low warm sun (ACES tone mapping, soft PCF shadows), sky-dome gradient with a sun glow, warm haze fog, subtle bloom.
- Floating island: grassy rim, layered rock strata on the sides, a jagged rocky underside with hanging stalactites. Small floating islets with autumn trees drift around it, plus a few voxel clouds.
LAYOUT
- Center: Bedloe's Island with star-shaped Fort Wood (11 points, granite + red brick walls), autumn trees, a seawall, small piers.
- On it: the granite pedestal with its loggia openings, and THE STATUE (must be instantly recognizable): green patina, flowing robe with vertical folds and a diagonal sash, raised right arm holding a torch with a balcony and a glowing gold flame, a tablet in the left arm with "JULY IV MDCCLXXVI", a crown with 7 rays and windows, a broad face.
- Around it: a harbor ring of animated water with waves, a paddle-wheel ferry, a tugboat with smoke, and a three-masted sailing ship.
- Behind it: a crescent of 1886 Manhattan on a street grid. Brownstones, red-brick tenements with black iron fire escapes, cast-iron store buildings with striped awnings, waterfront warehouses, wooden water tanks on the rooftops, chimneys with smoke, gas lamps with warm glow, a few lit windows, a gothic Trinity-style church with a tall spire, a white New England steeple church, a Central-Park-style green with a pond and a bandstand, US flags on rooftops.
- A river channel cuts through the city to the island edge, with the Brooklyn Bridge over it: granite towers with twin gothic arches, 4 main cables, vertical suspenders and the diagonal stay web, carriages and pedestrians crossing.
- An elevated railway (the El) on iron columns along one avenue, with a small steam engine pulling 3 cars and puffing smoke.
- Waterfalls pour off the island rim where the water reaches the edge, with mist at the bottom.
LIFE (all deterministic, driven by time t)
- Hundreds of tiny voxel people: pedestrians on sidewalks, crowds on the waterfront and a dedication grandstand with red-white-blue bunting, some waving and some holding parasols.
- Horse-drawn cabs on the streets, waving flags, seagulls circling the statue, smoke and steam particles.
CAMERA & OUTPUT
- A 10-second slow orbit of about 100° around the whole island with a slight zoom-in, the island always fully in frame (like a turntable showcase shot).
- Expose window.renderAt(t) for frame-by-frame capture at 30 or 60 fps, canvas sized from URL params ?w=&h= (default 1920x1080).
- Performance: merge static voxels into chunked meshes, cull hidden faces, use instancing for people and particles, and render the static shadow map once.
g023 retweeted
I wanted to test Fable 5.2's quality with one-shot prompts without human feedback.
Everything in this game is from scratch: models, sounds, textures. ~$550 at API pricing, ~2.7M output and 445M input tokens, heavily cached. Took 8 hours!
Play it here: claude.ai/artifact/BGY1SztrA…
g023 retweeted
this is kind of insane... i didn't realize there was this much efficiency gains to be had just from making the kv-cache block boundaries aware of agent turns so you dont throw away mid block tokens🤯
we are soo early...
g023 retweeted
Every time we have a new model, I run the same simple prompt on all my projects:
"Read this entire project and find everything that’s broken, inconsistent, inefficient, or no longer needed. Fix what you’re sure about and list the rest for me to review."
highly recommend trying
g023 retweeted
Zeroing an LLM's weights can leave its computation unchanged!
Unstructured pruning sets individual weights to zero, but keeps the matrix dimensions. A dense matrix multiply still processes those zeros. To save memory and work, we need to pack the surviving values with their locations and use a sparse kernel, a routine that can skip the zero terms.
g023 retweeted
A dude on Reddit tried to delete his 23andMe account.
Then learned the company would still keep his genetic data.
Best comment:
"Your DNA data is the real value, not the fee you paid."
That is a terrifyingly good business model.
Your customer can cancel the subscription.
They cannot get new DNA.
g023 retweeted
Paste this into Claude Code
when your project gets messy:
"Map this codebase for me.
Show the folder structure,
what each main file does,
how data flows from UI to database,
and the 5 riskiest files to touch.
Write it all to ARCHITECTURE.md."
You'll understand your own
vibe-coded app for the first time.
g023 retweeted
Yes, plenty more in the same family of latent defects:
5. Idempotency gaps: high. Retries or duplicates produce side effects that should have been no-ops.
6. Resource/lifetime mismatches: high. Acquired handles, locks, or connections never released on every path.
7. Hidden shared mutable state: critical. Globals, singletons, or caches mutated from multiple call sites without coordination.
8. Protocol/contract drift: high. Callers and callees diverge on assumed sequences or error semantics that types never capture.
Those plus the earlier four cover most of the “works until it doesn’t” class. An agent skill that systematically enumerates and ranks them would be useful.
g023 retweeted
Yes. Related concepts that surface the same class of bugs:
1. Implicit invariants
"Find all implicit invariants in this codebase: properties that must always hold but are only assumed via scattered checks or comments. List violations reachable via normal paths, retries, or concurrency."
2. Race conditions / shared mutable state
"Find all potential race conditions: shared mutable state accessed without synchronization, check-then-act sequences, and async interleavings that can produce inconsistent state."
3. Temporal coupling
"Find temporal couplings: sequences of calls or state changes that must happen in a fixed order but are not enforced by types or explicit machines. Identify what happens if the order is violated."
4. Incomplete failure paths
"Find incomplete error and cancellation paths: places where success is handled but retries, partial failures, or mid-flow aborts leave the system in invalid states."
Run them read-only first, rank by risk, then make the implicit explicit.
g023 retweeted
在只有 64GB 内存、0 张显卡的普通台式机上，硬跑 284.33B 参数的 DeepSeek-V4-Flash（还开满 256K 上下文），以前唯一的结局就是触发 Linux OOM Killer 当场暴毙，或者掉进 NVMe 硬盘 Swap 慢成幻灯片。
挂在 Hacker News 前排的开源项目 llama-modchip (v2.4.0)，硬是把这个原本占 76.87 GiB 的庞然大物，生生塞进了 57.61 GiB 的峰值内存（VmRSS） 里，纯 CPU 预填充飙到 44.36 tok/s，纯 CPU 解码跑出 11.5–14.7 tok/s；如果换跑 30B 的 MoE（Nemotron-3.5-30B-A3B），在 16 核 Zen 5 上纯 CPU 能跑出 60.4 tok/s，Threadripper 上直接干到 111.6 tok/s。
顺着它的 ARCHITECTURE.md 扒了一下，作者为了突破 64GB 内存墙，在底层干了三件极其暴力的系统工程活：
1. 二阶海森矩阵解析压缩 + 自定义 Fractal-64 格雷码量化，硬削 20.31 GiB
它没有用常规的粗暴低比特截断，而是针对 SwiGLU-H2 做了闭式二阶海森压缩，配合自定义的 GGML_TYPE_Q2_0 = 40（3.75 bpw 的 Fractal-64 格雷码张量重编码）和 L2 正交 MoE 裁剪，把模型体积从 76.87 GiB 压到了磁盘上的 56.56 GiB。这意味着整个 284B 权重加上 256K 上下文的 KV Cache 可以 100% 锁死在 64GB DDR5 内存里，零磁盘交换。
2. 把国际象棋引擎 Stockfish 的主变例搜索（PVS）搬来做 1.64 MB 的 L3 缓存投机采样器
纯 CPU 跑 MoE 最大的瓶颈是每次从 DDR5 内存拉冷门专家权重时，会把 CPU 宝贵的 L3 Cache 冲刷干净。
作者干脆写了一个只有 1.64 MB 的 Stockfish-PVS v2 + RouteCrack 投机草稿器（接受率高达 96.4%），并用 _MM_HINT_NTA（非临时缓存指令，HARPCache） 把冷专家权重的内存流隔离出去——让流式读取 MoE 专家时绝不污染常驻在 L2/L3 缓存里的热草稿器。
3. /dev/shm 共享内存热恢复，把本地跑 Claude Code 的首字等待从 45 秒砍到 13.1 毫秒
平时用本地 CPU 模型接 Claude Code，挂满 40 个工具的系统提示词每次重算都要卡上 45 秒。./modchip claude 直接把 Slot-0 的 KV 状态持久化在 /dev/shm 内存盘里，热恢复只要 13.1 ms。
g023 retweeted
Day 「3/28」 🔭 NextTransformer Auto Research
At step 512, Claude’s eventual winner was in second place. At step 768 it took the lead. After 49.152B tokens, AttnRes-Canon ranked first on CORE among our 36 formal runs.
How did it get there? In our first Arena study, Claude Opus 5.5 wrote the plugins and tested 12 candidates in NextT-Gym. Human approved budgets and stopping points. Claude picked candidates using dev bits per byte (BPB), with CORE-22 held out of selection.
Two edits kept showing up in the better runs. One let each token weight every earlier Transformer block separately. The other, Canon, mixed each channel across four nearby token positions. In round 3, Claude put them together: 0.79651 dev BPB vs 0.80031 for Block AttnRes at 2,000 steps. Each edit also helped on its own.
The gain held up in formal training: 35.99 CORE vs 34.00 for Block AttnRes and 33.10 for Dense, on the same protocol (seed 42).
You can try architecture search with our public Gym. Its 22 MCP tools let an agent submit code, check FLOPs, run experiments and freeze its pick.
Gym: github.com/Interactive-Activ…
Model, results and research trace: github.com/Interactive-Activ…
ALT Claude tested 12 candidates across three search rounds. At 2,000 updates, the best dev BPB fell from 0.79951 in round 1 to 0.79819 in round 2 and 0.79651 in round 3. The selected model, AttnRes-Canon, reached 35.99 CORE ×100 after 49.152 billion tokens, ranking first among 36 formal runs at seed 42.
ALT Four token positions each mix their own embedding and earlier block outputs; Canon then combines the resulting features at the current and previous three positions. At 2,000 updates, dev BPB for eight depth groups was 0.80031 without Canon and 0.79819 with Canon A–D. For 23 individually readable blocks it was 0.79951 without Canon and 0.79651 with Canon A–D. Lower is better.
g023 retweeted
you can do this for absolutely any scene
i usually like gatekeeping but just because i'm nice here's the skill: github.com/dotslashsimran/si…
tried recreating @garments__ 's vid from scratch in three.js with claude code to learn perspective illusion
telephoto lens squashes the street flat, then it widens and the depth appears. ink lines are a glsl shader. side by side below
and after some iterations i've now turned this into a claude skill 😺
g023 retweeted
Weight Space Programming can play Parameter Golf
Gave Opus 5.5 and GPT-6 Astra one looped transformer block with LayerNorm and every weight set to zero. Together with a Python sandbox and 200 forward passes of feedback on practice data from classical algos. The goal: 100% correct on hidden test set, then use as few nonzero weights as possible. Four highlights:
🔢 4-digit multiplication: Opus needed 273 weights, below the 355-weight network that inspired this (@deepaksinghml). Astra got to 657.
⚖️ Parity of up to 60 bits, a classic task trained transformers can fail at: 18 weights (Opus) vs 24 (Astra). It works on bit strings longer than any it practised on. The trick is attention as a mean-pool that averages the bits.
🧮 Bracket matching (Dyck-2): 55 weights (Opus) vs 262 (Astra). The attention map shows each bracket looking back at its partner, and the network decides in 3 loop rounds.
🌀 Rule 110 cellular automaton: 323 weights (Opus) vs 630 (Astra).
All four networks are exact on fresh sequences they never saw. Astra had correct networks within minutes but stopped improving early. Opus thinks much longer per step, keeps finding sparser constructions, and won all four.
g023 retweeted
🚨“It works, but why is training so much slower?”
We kept hearing this from people replacing standard residual connections with Hyper-Connections, and ran into it ourselves. 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸 𝗺𝗛𝗖 𝗮𝗱𝗱𝘀 𝗲𝘅𝘁𝗿𝗮 𝟱𝟵–𝟭𝟬𝟬% 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗼𝘃𝗲𝗿𝗵𝗲𝗮𝗱 𝘄𝗶𝘁𝗵 𝗮 𝗣𝘆𝗧𝗼𝗿𝗰𝗵 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻.
That’s why we’re introducing 𝗦𝗶𝗺𝗽𝗹𝗲𝗛𝗖, one of the fastest Hyper-Connections methods to date.
It adds just 2.4% training overhead over vanilla residual at the largest tested width, with competitive performance compared to mHC and demonstrated scalability.
📄 arxiv.org/abs/2611.05984
💻 github.com/aHapBean/SimpleHC
g023 retweeted
Weight Space Programming: a glimpse into continual learning.
Agent edited Qwen 0.8B's weights to inject facts about my X profile directly into the model: my handle, bio, GitHub, and what i build. It edited 2% of the network (5 of 488 tensors). No training, no gradients.
Result:
Before, it had never heard of me ("I don't have access to private information…").
After, it tells you who i am, my bio and GitHub, answer project questions it was never trained on. Everything else it knew is untouched.
Reproduce : Link to reproduction repo in comments. Point your agent to the repo and teach model your own facts.
g023 retweeted
40,000 small objects in a city of grey blocks. Left draws everything inside the view: 12,364. Right tests chunks, then objects, against a Hi-Z depth pyramid: 1,953.
Watch the split 👇
▸ The problem
Frustum culling keeps everything inside the view cone, including everything behind the next wall. Walking down a street, most of that is invisible, but it is still sent to the GPU and still costs vertices.
▸ How it works
The source (a Unity URP renderer feature) does it in three steps:
1. Copy the depth buffer into a 512 texture and build mips. Each texel of the next mip keeps the farthest of the 2x2 below it, so one texel answers "what is the farthest thing covering this area?"
2. A compute shader projects every instance, picks a mip, and reads the Hi-Z there. If the instance is farther than the farthest occluder, it is hidden. Off-screen instances go too.
3. Survivors are appended to a buffer and drawn with one indirect instanced draw.
I added a chunk level: the map is cut into 1,024 cells of 10 m. Chunk boxes are tested first, and objects in a rejected chunk are never tested at all.
▸ What I hit
WebGL2 has no compute shaders and no indirect draw. My tests run in fragment passes, one pixel per chunk or object. I read the results back and pack the survivors into instanced meshes on the CPU.
The source tests one centre point per instance with a distance-based mip. A centre point can hide an object that still peeks out, so I test the whole bounding box and pick the mip from its screen size.
My Hi-Z is 512 x 256 from a 960 x 1080 view. The first level takes the farthest depth over each texel's whole footprint, so it never claims more cover than the full-res depth shows.
▸ My build
three.js (r187), WebGL2. 254 buildings are the only occluders in the Hi-Z. 40,000 objects of 3 kinds: 80, 32 and 14 triangles.
At 3 s: left draws 12,364 objects, 600,872 triangles. Right keeps 132 of 1,024 chunks (216 rejected by Hi-Z), tests 5,474 objects and draws 1,953, 93,480 triangles.
▸ Trade-offs
+ Big cuts wherever walls hide most of the view
+ The chunk test skips most per-object tests
- Occluders must be in depth before the test
- Reading results back stalls the GPU; the source avoids that with indirect draws
- Small objects never hide anything
- I rendered on a software GPU, so counts only, no timings
▸ When to use it
Cities, interiors and dense props behind big occluders. In open fields frustum culling already does most of the work.
