I used to sell malware to afford food. Then I fell in love with math and LLMs for 8 years. LLMs/ Contact center engineering / AudioLLMs Building: @agiathome
Bangalore
Joined January 2024
- Tweets10.9K
- Following963
- Followers13.6K
- Likes27.5K
Pinned Tweet
Finally got around to building a website for the corporate coaching business. It's based on a 12 minute voice note and mostly one shotted. Do have a look. :)
Link:
coaching.enterprise-ai.in
I have been trying to publish this absolutely ass paper in a conference and every conference suddenly has a high bar and competition. (And there are way fewer predatory journals/ confs too.)
I like this. It just feels like a lot has changed from my college days
Adani has a show where he finds and funds village enterprises. Based on Shark tank where fashion entrepreneurs judge deftech companies. The best way to get funded by an Indian Vc is to get on their TV show and make them look smart
In case any Indian VCs want to prove @ashleevance wrong and back a very ambitious deeptech company, happy to make an intro to Suhas Tejaskanda, founder of Flying Wedge Defence & Aerospace (@Flying_Wedge_). They're making India's first/only unmanned fighter jet in Bengaluru.
The math behind this is fun. Long tweet
This neural steering doesn't modify attention (the models implicit understanding of language) in that layer, yet it can modify what subsequent attention layers do (the consequent interpretation of language). Think of it as hurting someone's head in the right place so they forget a specific behavior. 1/n
But then how do you discover which layer to change.
1. There are some fixed prompts that have already been decide upon
2. For each neuron, the main experiment uses ten combinations of prompt and steering strength, plus unmodified baselines for comparison.
3. Like a standard ML experiement, an intervention is added. At a selected neuron
4. Generate response and measure distinctive words and output coherence or whatever you care about.
5. Compare it against a controlled random experiment. Find if he effect neuron-specific or just perturbation
6. Find promising neurons that make an impact and rank them
7. Investigate and combine. See what happens on combination
The current ranking is vocabulary based and vocab changes behavior. Think, this down streams to which token gets predicted next and as a result how the sentence gets steered across the board.
Conclusions
1. Random vectors might change identity
2. stimulating a particular internal direction can repeatedly introduce a recognizable conceptual theme
3. Steering strength matters enormously. Individual neurons can causally influence the model's output.
4. Not every neuron has an independently interpretable function.
5. correlations between neurons don't establish causal circuits
6. Later-layer neurons can more directly influence vocabulary selection. Mid-layer neurons may encode behaviours that simple word-based metrics fail to detect
This is the man who wants you back in India btw. His counterpoint is comparing you to Sudanese and Somali illegal immigrants. I am sorry sir but if you want our people to be cannon fodder for the wester political olympics, you can just send them to fight wars instead. No other country addtesses their citizens this way. This is an American argument to see every immigrant with the same lens. This is not your position. Why the fuck would you say "our people are not terrorists like the others. Please be nice to them" on a public stage. You have the argument skills of a high schooler and by all accounts you must go back to studying language and communication in a Gurukul or something.
We have reported 2 rare earth potentials to GoI-
- Chromite in a part of Maharashtra and
- Cobalt in east India
khanan (the ai agent trying to scout minerals before the gov does) now has it's own website (khanan dot cloud), is going live soon. Will be open source. Including prompts. (hopefully) Never raising money.
These people better deliver. I will be a subscriber for life
