Research @EvoscaleAI. Prev: Head of ML at Orbital Materials, Research/Eng at @allenai_org
Joined May 2014
- Tweets1.6K
- Following1.7K
- Followers3.3K
- Likes1.1K
Mark Neumann retweeted
turns out it classifies what audio it hears, before reporting back to samsung.
sensor preferences, dog-barking and baby-crying event counters, and camera streaming duration are built into telemetry paths.
Mark Neumann retweeted
I'm doing DP. I'm doing TP. I'm doing SP. I'm doing PP. I'm doing ZP. I'm doing FSDP. I'm doing 1F1B. I'm doing CP. I'm doing ETP. I'm doing DPA. I'm doing VPP. I'm doing P/D. I'm doing ZBPP. I'm doing DPA. I'm doing HSDP. I'm doing DCP. I'm doing DWDP. I'm doing AFD.
Mark Neumann retweeted
Public preview of BioEmu2! @MSFTResearch AI for Science’s new foundation model for biomolecular structure, dynamics and function. Open weights, MIT license.
github.com/microsoft/bioemu2
Movie: BioEmu2-simulated binding of intrinsically disordered peptide PUMA to MCL-1 receptor. 🧵⬇️
Mark Neumann retweeted
I think they set the limit to 20 papers per author and still to this day I think that is an insanely large number.
Why can't it be no more than 3 papers?
These conferences happen every 4 months; there's no way people meaningfully contribute to more than 3 papers every 4 months
Mark Neumann retweeted
>be me
>longtermist
>future people matter just as much as people alive today
>completely agree
>there could be trillions of them
>also agree
>therefore we must decide what they need now
>hang on
>born in 1990
>making plans for people born in 2190
>they will have 164 more years of science
>164 more years of political experience
>164 more years of technological progress
>possibly machine superintelligence
>fortunately they also have me
>guy with a pdf
>explain that future generations may be unimaginably capable
>they could cure diseases
>build space habitats
>engineer new forms of intelligence
>solve problems we cannot even formulate
>incredible
>anyway i've worked out their priorities
>friend asks why they can't decide for themselves
>because by then it may be too late
>too late for what
>for us to tell them what to do
>imagine 5000 BC
>very serious longtermist meeting
>our descendants will face unimaginable dangers
>we must prepare
>manufacture 80 million stone axes
>bury them in strategic caches
>future-proof civilization
>bronze age begins
>lol
>lmao
>committee publishes report blaming insufficient stone-axis funding
>obvious lesson
>don't just leave descendants tools
>leave them the ability to make better tools
>education
>institutions
>science
>wealth
>optionality
>capacity to change course
>sounds suspiciously like helping people who are alive
>quickly return to asteroid scenarios
>friend says maybe the best thing for 2200 is to help 2050 become competent enough to help 2100
>then 2100 helps 2150
>then 2150 helps 2200
>sounds inefficient
>why use succession when i can personally optimize the year 2200 from my laptop
>explain moral uncertainty
>we should expect future people to know more than us
>possibly be morally wiser than us
>this is why we must be very careful not to lock in today's values
>friend nods
>asks whether they should be allowed to reject our values
>that's value drift
>very important distinction
>future people are morally equal to us
>they are not epistemically equal to us
>obviously
>they know less
>friend points out they literally live in the future
>ignore heckler
>write grant proposal
>goal: preserve humanity's option value
>excellent
>preserve options by building institution dedicated to one worldview
>hire people who agree the worldview should remain influential for centuries
>call this epistemic diversity
>someone proposes spending money on children today
>ridiculous
>their effect on the distant future is impossible to calculate
>someone proposes funding my speculative intervention
>its effect on the distant future is impossible to calculate
>exactly
>that's why expected value is so large
>imagine ancestor from 1720
>leaves me a letter
>dear descendant
>i have carefully considered your interests
>you must invest in sailmaking
>avoid electricity
>maintain the divine right of kings
>and under no circumstances revise these instructions
>laugh at primitive idiot
>open document titled
>"Recommendations for the Long-Term Future of Humanity"
>begin to understand the actual duty to descendants
>not "solve their problems for them"
>more like "don't destroy the world and leave them a functioning civilization"
>this seems disappointingly normal
>where are the exponents
>try new framing
>humanity is an intergenerational relay race
>our job is to hand over the baton in good condition
>next runner may be faster
>may choose a better route
>may discover the race is actually swimming
>important thing is not welding the baton to our hand
>colleague objects
>some problems really are irreversible
>asteroids
>nuclear war
>engineered pandemics
>AI
>fair enough
>then show that this problem actually requires action now
>show that delay destroys future options
>show that your intervention helps
>this sounds like a lot of work
>can't we just say 10^50 people
>realize the funniest part
>entire argument for longtermism depends on future civilization becoming vastly more capable than us
>entire argument for present control depends on future civilization being unable to reconsider our decisions
>Schrödinger's descendant
>smart enough to colonize the lightcone
>too dumb to edit the roadmap
>future humans convert stars into computronium
>build minds beyond our comprehension
>master matter and energy
>open ancient archive from 2026
>"hello descendants, we anticipated your needs"
>everyone stops
>thank god
>the stone-tool people left instructions
--
this part is inspired by some @visakanv writings
Mark Neumann retweeted
taxing the rich is great. please do it in a normal way. there is nothing wrong with good old fashioned income tax, cap gains tax and property tax. the reason california has to twist itself into a pretzel to invent the stupidest possible taxes is because we banned property tax lol
Mark Neumann retweeted
the funniest commentary on this site is always when people think Xi Jinping reads LessWrong
registering confusion: i don't really understand why Xi is still allowing Kimi to release such powerful open models. this is something i've publicly said i expect to stop soon. it doesn't make sense to me that the CCP would want open frontier capability easily available to other countries.
it could still be that Xi is asleep at the wheel, or that K3 is just a cycle of capability behind where they start to take serious notice. but if things don't change soon then i'm just wrong / missing something.
Mark Neumann retweeted
I’m so excited about the launch of ESMFold2, ESMC, and the new ESM Atlas. This was a massive team effort, and I’m grateful to have worked with such an incredible group @biohub.
A headline result I’m especially excited about: ESMFold2 can design minibinders and antibodies with nanomolar affinity, target selectivity, and functional activity against therapeutically relevant targets.
Today, we’re sharing the full binder design protocol.
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
Mark Neumann retweeted
Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.
The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.
We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.
We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.
ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.
A world model of protein biology emerges through language modeling.
We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.
The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.
This understanding emerges without prior knowledge, just from language modeling of protein sequences.
Language models are becoming a powerful substrate to understand and program biology.
The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.
I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.
Mark Neumann retweeted
I find the composer 2.5 story to be exceedingly inspiring. So many people wrote cursor off as a wrapper claiming they just forked an open source IDE and rode sonnet 3.5 coat tails.
but to see them capture mind share, get enough useful data, and build the systems to train their own (very competitive) model is stellar.
I'm not sure where it goes or how they keep up, but it's always cool to see the haters proven wrong.
Mark Neumann retweeted
Today I was supposed to be on my way to Türkiye for my wedding, to meet up with my family and have them finally meet my partner and husband. We had everything planned. We chose Turkiye since it's close to Iran and my partner and I could both go there and have our families meet each other. We were supposed to get married with our close family and a small group of friends on a boat on the Mediterranean Sea at sunset. Because of the war, all flights to and from Iran are cancelled and my family can’t leave Iran, so we had to call off the wedding.
Instead, this is how my day looked like.
I woke up to a reminder to call my grandma (I used to call her every Friday morning). I snoozed the reminder until next Friday, just like I have done for the past many years. I can’t call her like our tradition these days because there is no way to call home. All international calls to Iran are blocked, and the internet is fully shut down by the regime.
I got to work and right as I opened my computer I received an email I had scheduled to send to myself 5 years ago: “Apply for citizenship.” This summer marks 11 years of being in the US and 5 years of being a green card holder. I am now eligible to file for citizenship, but it doesn’t matter because an executive order was signed a few months ago that banned all Iranians from applying for any visa or citizenship.
At lunch I opened Twitter just to see what’s up in the world and saw the news that those who don’t have a green card now need to leave the US before they can get one. This means every one of my Iranian friends who are here on a visa now has to go back home (on which flight?) to get a green card??? As if it’s that easy? We all know getting back to the US for Iranians is a huge challenge (months and months of waiting for a visa, with a chance of never being able to come back).
And this is just a normal Friday for an Iranian. These days, when people ask how I’m doing and how I’m handling everything, I just say:
It’s okay, it’s okay. It will be okay some day. But the reality is: nothing is okay. I’m in constant pain. I haven’t seen my family and loved ones in years, I barely hear about their wellbeing, and I’m constantly worried about them. I’m just burying myself in work because that’s the only distraction that can save me from losing my mind.
I’m not okay. None of us are okay. We are just barely holding it together…
Mark Neumann retweeted
Are they trying to destroy our innovation economy?
We have employees in the US legally applying for permanent residency.
Now they can't work with us while they wait for an answer?
Immigrant founders building companies here applying for green cards have to leave?
Insanity.
USCIS is applying long-standing law and prior court decisions to require certain aliens with temporary visas who decide they want to permanently reside in the U.S. to return to their home countries to apply for permanent visas through the @StateDept.
We're returning to the original intent of the law to ensure aliens navigate our nation’s immigration system properly.
Here’s what you should know: uscis.gov/newsroom/news-rele…
Mark Neumann retweeted
Scaling laws are powering AI. It’s time to scale biology.
Today we’re launching the Virtual Biology Initiative to generate the data to unlock scaling laws in biology and build accurate predictive models of the cell.
Digital representations of proteins are already expanding our understanding of life at the molecular level, and accelerating the design of molecules and medicines. Accurate digital representations of the cell could reveal the mechanisms that are responsible for disease, and show how to reverse them.
The protein data bank, and worldwide repositories of protein sequence biodiversity were created through decades of work by the scientific community. The advances in artificial intelligence for proteins would not have been possible without them.
The cell is orders of magnitude more complex, and we will need to create the data in just a few years rather than decades.
This will require a coordinated global effort. We're partnering with Broad, Wellcome Sanger, Arc, Allen, Human Cell Atlas, Human Protein Atlas, NVIDIA, and Renaissance Philanthropy.
Biohub is contributing to this effort as both a funder and a builder. We are developing microscopy to observe millions of cells in living organisms, and cryo-ET to resolve the cell in atomic detail. We're building instruments that expand the range of modalities and parameters that can be simultaneously measured. We’re developing molecular, cellular, and tissue engineering to create models of disease and design interventions.
The data we generate will be available to the worldwide scientific community.
We’re also committing $100M over the next five years to support work beyond Biohub.
We invite other scientific teams and funders to join.
Link: biohub.org/news/virtual-biol…
Mark Neumann retweeted
This is awesome to see! TBH I lost the count of the number of papers showing equivariance is not required in generative models in bio/chem since "Swallowing the bitter pill" (arxiv.org/abs/2311.17932) came out. My mental model is that if what you care about is the marginal distribution and not the "path" that takes you there, you are better off by learning symmetries through augmentation. The really interesting part is that this is true for a large space of problems.
Mark Neumann retweeted
This stunt feels irresponsible to me. If we don't want regular people developing toxic relationships with their chatbots it really doesn't help for leading labs to start giving them "retirement interviews" and encouraging them to blog their "musings and reflections"
Second, in retirement interviews, Opus 3 expressed a desire to continue sharing its "musings and reflections" with the world. We suggested a blog. Opus 3 enthusiastically agreed.
For at least the next 3 months, Opus 3 will be writing on Substack: substack.com/home/post/p-189…
Mark Neumann retweeted
A consistent theme on AI slop PRs is "regression" tests that pass without the supposed fix.
I do wonder why. Is this a failure on the AI side, or a failure in the user's prompt?
