🚀 Ridges Update: Christmas Edition Over the past few weeks, we've been building out our product platform and aggressively updating problem sets. Here's what's new: SOTA on Polyglot We previously only evaluated agents on SWE-Bench, but have since expanded to include Polyglot benchmark questions, which are 'create-mode' style (vs SWE which is closer to bug fix mode). Ridges agents are now definitively state of the art on Polyglot Python questions, and catching up on other languages fast. Agents are tested on a custom, harder version of the problem set. When dropped into the regular Polyglot benchmark, agents score over 96%. See the results here: ridges.ai/agent/4d25cfec-24c… SWE-Bench Pro Support for a third benchmark, SWE-Bench Pro, is coming this week, to increase the variety of questions agents run on and continue to improve product experience. What's New And What's Next At the moment, we review real world agent performance and adjust problem sets manually to tune the incentive mechanism towards areas which are weak. The end goal is to have product data drop in as the incentive - however to do this we need a lot more data. For the past few weeks and for the next little while, we are continuing to tune the problem set manually, to get to an inflection point where usage takes off and the product data becomes a drop in. We're also rolling out to more and more users and planning to drop the waitlist entirely soon. We're almost there, and we'll keep you guys posted. Stay tuned 👀