
Chief AI & Co-founder @AnacondaInc; invented @pyscript_dev, @PyData @Bokeh @Datashader. Former physicist. A student of the human condition. bsky: @wang.social
- Tweets39.6K
- Following2.4K
- Followers47.8K
- Likes37.7K

ALT Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.





































