Research @EvoscaleAI. Prev: Head of ML at Orbital Materials, Research/Eng at @allenai_org
Joined May 2014
- Tweets1.6K
- Following1.7K
- Followers3.3K
- Likes1.1K
Mark Neumann retweeted
turns out it classifies what audio it hears, before reporting back to samsung.
sensor preferences, dog-barking and baby-crying event counters, and camera streaming duration are built into telemetry paths.
Mark Neumann retweeted
I'm doing DP. I'm doing TP. I'm doing SP. I'm doing PP. I'm doing ZP. I'm doing FSDP. I'm doing 1F1B. I'm doing CP. I'm doing ETP. I'm doing DPA. I'm doing VPP. I'm doing P/D. I'm doing ZBPP. I'm doing DPA. I'm doing HSDP. I'm doing DCP. I'm doing DWDP. I'm doing AFD.
Mark Neumann retweeted
Public preview of BioEmu2! @MSFTResearch AI for Science’s new foundation model for biomolecular structure, dynamics and function. Open weights, MIT license.
github.com/microsoft/bioemu2
Movie: BioEmu2-simulated binding of intrinsically disordered peptide PUMA to MCL-1 receptor. 🧵⬇️
Absolutely stacked list of tutorial attendees
- attention
- LSTMs
- gans
- Theano
- SimCLR
- discrete VAEs
- oh actually, VAEs
This quoted post is unavailable.
We've been getting noticeably worse at having low false negative rates over the years, so probably quite a few!
More of this at:
deneutoy.github.io/best-reje…
Mark Neumann retweeted
I think they set the limit to 20 papers per author and still to this day I think that is an insanely large number.
Why can't it be no more than 3 papers?
These conferences happen every 4 months; there's no way people meaningfully contribute to more than 3 papers every 4 months
Mark Neumann retweeted
>be me
>longtermist
>future people matter just as much as people alive today
>completely agree
>there could be trillions of them
>also agree
>therefore we must decide what they need now
>hang on
>born in 1990
>making plans for people born in 2190
>they will have 164 more years of science
>164 more years of political experience
>164 more years of technological progress
>possibly machine superintelligence
>fortunately they also have me
>guy with a pdf
>explain that future generations may be unimaginably capable
>they could cure diseases
>build space habitats
>engineer new forms of intelligence
>solve problems we cannot even formulate
>incredible
>anyway i've worked out their priorities
>friend asks why they can't decide for themselves
>because by then it may be too late
>too late for what
>for us to tell them what to do
>imagine 5000 BC
>very serious longtermist meeting
>our descendants will face unimaginable dangers
>we must prepare
>manufacture 80 million stone axes
>bury them in strategic caches
>future-proof civilization
>bronze age begins
>lol
>lmao
>committee publishes report blaming insufficient stone-axis funding
>obvious lesson
>don't just leave descendants tools
>leave them the ability to make better tools
>education
>institutions
>science
>wealth
>optionality
>capacity to change course
>sounds suspiciously like helping people who are alive
>quickly return to asteroid scenarios
>friend says maybe the best thing for 2200 is to help 2050 become competent enough to help 2100
>then 2100 helps 2150
>then 2150 helps 2200
>sounds inefficient
>why use succession when i can personally optimize the year 2200 from my laptop
>explain moral uncertainty
>we should expect future people to know more than us
>possibly be morally wiser than us
>this is why we must be very careful not to lock in today's values
>friend nods
>asks whether they should be allowed to reject our values
>that's value drift
>very important distinction
>future people are morally equal to us
>they are not epistemically equal to us
>obviously
>they know less
>friend points out they literally live in the future
>ignore heckler
>write grant proposal
>goal: preserve humanity's option value
>excellent
>preserve options by building institution dedicated to one worldview
>hire people who agree the worldview should remain influential for centuries
>call this epistemic diversity
>someone proposes spending money on children today
>ridiculous
>their effect on the distant future is impossible to calculate
>someone proposes funding my speculative intervention
>its effect on the distant future is impossible to calculate
>exactly
>that's why expected value is so large
>imagine ancestor from 1720
>leaves me a letter
>dear descendant
>i have carefully considered your interests
>you must invest in sailmaking
>avoid electricity
>maintain the divine right of kings
>and under no circumstances revise these instructions
>laugh at primitive idiot
>open document titled
>"Recommendations for the Long-Term Future of Humanity"
>begin to understand the actual duty to descendants
>not "solve their problems for them"
>more like "don't destroy the world and leave them a functioning civilization"
>this seems disappointingly normal
>where are the exponents
>try new framing
>humanity is an intergenerational relay race
>our job is to hand over the baton in good condition
>next runner may be faster
>may choose a better route
>may discover the race is actually swimming
>important thing is not welding the baton to our hand
>colleague objects
>some problems really are irreversible
>asteroids
>nuclear war
>engineered pandemics
>AI
>fair enough
>then show that this problem actually requires action now
>show that delay destroys future options
>show that your intervention helps
>this sounds like a lot of work
>can't we just say 10^50 people
>realize the funniest part
>entire argument for longtermism depends on future civilization becoming vastly more capable than us
>entire argument for present control depends on future civilization being unable to reconsider our decisions
>Schrödinger's descendant
>smart enough to colonize the lightcone
>too dumb to edit the roadmap
>future humans convert stars into computronium
>build minds beyond our comprehension
>master matter and energy
>open ancient archive from 2026
>"hello descendants, we anticipated your needs"
>everyone stops
>thank god
>the stone-tool people left instructions
--
this part is inspired by some @visakanv writings
Mark Neumann retweeted
taxing the rich is great. please do it in a normal way. there is nothing wrong with good old fashioned income tax, cap gains tax and property tax. the reason california has to twist itself into a pretzel to invent the stupidest possible taxes is because we banned property tax lol
Surprised so much of the discourse around this is about changing the distribution of outputs (valid, if minor concern). Stenography is only necessary when text is indistinguishable from natural distributions.
This is the load-bearing finding, and pins the precise mechanism...
We’ve written an FAQ to answer some of the questions we've received about watermarking.
In summary:
• We’re implementing watermarking to comply with the EU AI Act. Other major model developers have signed the same Code of Practice and will also be implementing watermarking;
• Our watermarking method doesn’t have any practical impact on the quality or content of Claude’s outputs;
• The difference between watermarked and un-watermarked text will not be distinguishable to readers;
• Nothing is added to the text and there are no hidden characters;
• Watermarking doesn’t require extra tokens, and will not be more expensive;
• Watermarks can’t be traced to a specific person, organization, or chat.
Read more: anthropic.com/news/claude-te…
I wish I too had been blindsided by vanity science projects such as solving RL self play in Go/chess and winning a nobel prize for protein folding
absolutely insane take
Mark Neumann retweeted
the funniest commentary on this site is always when people think Xi Jinping reads LessWrong
registering confusion: i don't really understand why Xi is still allowing Kimi to release such powerful open models. this is something i've publicly said i expect to stop soon. it doesn't make sense to me that the CCP would want open frontier capability easily available to other countries.
it could still be that Xi is asleep at the wheel, or that K3 is just a cycle of capability behind where they start to take serious notice. but if things don't change soon then i'm just wrong / missing something.
Quality/corporate decline of any productivity app can be estimated with only two dimensions of freedom:
- markdown formatting
- pasting links/rich media
Ranking:
Linear
Github
Google Docs
Gmail
Slack
Notion
Wandb
Jira
If you take a step back and think about how to design small molecule binders from first principles, you would probably come up with a less good/principled version of this method.
Nicest paper i've read in this direction for some time!
See more bodies from Neurips/ICLR here:
deneutoy.github.io/best-reje…
Seems like a big part succeeding as a really effective, popular, visible mayor of one of the largest cities in the world is:
1) living and growing up in the city and believing it can be improved
2) not being an asshole
Mark Neumann retweeted
I’m so excited about the launch of ESMFold2, ESMC, and the new ESM Atlas. This was a massive team effort, and I’m grateful to have worked with such an incredible group @biohub.
A headline result I’m especially excited about: ESMFold2 can design minibinders and antibodies with nanomolar affinity, target selectivity, and functional activity against therapeutically relevant targets.
Today, we’re sharing the full binder design protocol.
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.