LayerZero Sybil Hunting Team
Joined June 2024
- Tweets36
- Following4
- Followers98
- Likes42
All of Team Opsec's sybil reports were denied.
However, thanks to the efforts of @PrimordialAA and @LayerZero_Core all of our reports have since been accepted in full.
Many thanks to the LayerZero team, we're pleased to have solidified ourselves as a top bounty hunter.
Team Opsec retweeted
Ok so I see a lot more anticipation for the upcoming Sybil list or "The List" as people have started to reference it as. Let me explain what this list is and what it is not -- and what we hope will come from it long term.
We started with an enormous amount of data, way more than we ever expected.
469 bonders putting up 0.5 ETH total ($1.6m bonded value)
3,800 threads created by the bonders
22,000 Clusters within the threads to be reviewed
In order to make this manageable there are a few filters we set in place to be able to quickly evaluate quality of reports and whether they were worth reviewing in depth.
First -- discard any clusters with < 20 addresses
Second -- based on our placeholder eligibility, if <30% of the wallets in a given cluster were eligible we would put it to the back of the queue for review if there was enough time
Third -- if any report flagged any of the top ~250 addresses it was quickly checked (and usually) discarded. This prevented people from cheekily sneaking in high value addresses into their clusters.
From there it's been a pure slog, as everyone has seen from my own feed it's just been a battle. Hundreds of different methodologies, thousands of clusters, often time very hard to pin down unknown contracts or qualify statistical methods -- but it's a battle that had to be fought and I'm super happy about the end result here (will write more on this later).
So the end result
This report is going to be a large scale (20+) and industrial sybil list and how it relates to @LayerZero_Labs. We think the end result will have caught a massive amount of sybil, returned a massive amount back to real users, and be an amazing piece of research groundwork laid for the community to pick up the torch.
What it WON'T be:
By the nature of time and us not evaluating accounts that were unlikely to be eligible this means there is a ton of sybil that is NOT included in our list. Amazing reports that caught or may have caught sybil clusters but since there would be no reward for them since they had no eligibility they were discarded.
This also means some low volume sybil accounts that are not included in the list who won't qualify for the core LZ distribution but will end up qualifying for some small amounts of RFPs. The issue with evaluating these is some of them have 1, 2, or 5 tx on LayerZero and when you're getting to that point the risk of false positives is much MUCH higher and we chose to err on the side protecting real users as much as possible and minimizing false positive risk. It's a tradeoff to make but we think the most practical one given the end goal.
What we hope happens from here --
The @commondotxyz forum now has a permanent log of all 3,800 threads and 22,000 clusters. This is an INCREDIBLE resource and probably one of the greatest all time wide endeavours in combatting sybil. Our focus was on LZ related sybil but tons of the remaining may have high degrees of connection external to LZ with barely any LZ usage. We very much hope the effort continues after our portion is done and the torch is picked up and a final list is put together from the community. We would even be happy to issue a grant for a group to continue this work as it seems like an incredible resource to be wasted.
I have absolutely blown away by the amazing research conducted by so many incredible analysts and sleuths. I have endless appreciation for all of you and hope everyone knows we have absolutely killed ourselves in pursuit of paying proper respect to the work you have done and getting through as many reports as possible (team was all here until 4am reviewing reports last night and back in first thing in the morning to continue reviewing).
My thanks to all of you, we will have the final results available for public review this weekend and look forward to feedback and anything missed
🫡
GM @PrimordialAA & @LayerZero_Labs,
We at Team Opsec authored 3 great reports for you. This letter is regarding our report:
Multichain Clustering: commonwealth.im/layerzero/di…
TL;DR
The Updated Reduced List (67,289 => 29,957 wallets) & $1,000 Reward Public Challenge for our report can be accessed at: docs.google.com/spreadsheets…
Long Version
Our report marked 67,289 unique wallets, clustered into 1,933 groups of size 20+ across two categories: Scroll & Multi-chain. We used a conservative cutoff score for the “silhouette score” quality metric of 0.3+ for either category. This was chosen to allow yourselves to qualify clusters to your own desired level of precision & clarity. We expected your team to be comfortable removing clusters they were unhappy with, accepting ones they found conclusive.
Upon seeing Brian Pellegrino’s latest tweets ( xitter.gq/PrimordialAA/status/18… and xitter.gq/PrimordialAA/status/18…) we are concerned that resources are scarce, and realised our report may be too demanding for you to process. We did not anticipate that there would be very little scope for your team to filter clusters. The worst outcome for everyone is for our report(s) to be discarded entirely due to issues with cluster clarity, despite our report consisting almost entirely of sybil operators. We want to clear ambiguity in the cluster assignments. You will no longer have to work to filter clusters.
The issue: Some clusters formed from a single grouping assignment (only scroll cluster, not multi-chain cluster) and towards the lower end of the silhouette score range (0.3-0.5) were oftentimes composed of multiple similar but distinctly different sybil operations. This commonly occured when popular sybilling tools such as NFTCopilot and Lootbot were used by different users, or when sybil farms had implemented minor behavioural variations and sub-clusters to avoid detection. These mixed operations, whilst all still very clearly being “sybil farms”, may not ALL easily be concluded as being operated by one entity.
In a high pressure, tight deadline situation such as the one you find yourselves in, we would hate for you to accidentally assume these mixed clusters are made up of legitimate users. This mistake is easily made due to activity not being identical across the entire cluster, but instead being segmented; this is unusual to see compared to other bounty hunter reports. Note: From reviewing over 6,000 wallets across 125 of our clusters we identified 9 legitimately independent users that existed at the lower end of single-grouping silhouette score clusters. This was within our margin of error, all these users are comfortably removed with these stricter criteria.
A similar report (commonwealth.im/layerzero/di…) has been held in high regard by Brian. A refined list of sybil wallets for this report seems to have been accepted by LayerZero. They had similar concerns about LayerZero being unable to process their report internally (xitter.gq/PrimordialAA/status/18…).
To follow by example we are issuing an extreme list containing only the most clear-cut and uniform clusters from our original report, they are extremely easy to process. Note: Of the wallets we have omitted, the VAST (99%+) majority are operated by sybil farms, but require more time to identify & interpret accurately due to the complexities of clustering assignments. Please, only use this new list as a last resort if you are unable to allocate resources into verifying our original findings. This report aims to help you out in a pinch, as a more favourable alternative to discarding our entire report.
To produce our extreme list we identified the cut-off point where mixed clusters were appearing. We added a comfortable margin, then removed all instances of wallets below. The criteria are:
1) Wallets with only a Scroll grouping: Silhouette score req. of 0.635+
2) Wallets with only a Multichain grouping: Silhouette score req. of 0.58+
3) Wallets with both a Scroll & Multichain grouping: Mean score of 0.345+
From the 125 cluster samples we analysed, the above scores eliminated 100% of mixed sybil clusters. In short, all clusters are now easily identifiable as operated by a single entity.
This new set of stricter criteria, along with requiring 5+ LayerZero transactions and then reclustering for groups of 20+, reduced our report from 67,289 (63.5% dual clustered) to 29,957 (97.4% dual clustered) unique wallets.
All 29,957 unique wallets included in this adjusted report are now in groups that are immediately noticeable as being by the same operator.
We have updated our public challenge; any who identify a cluster containing a small number of organic users receive a $200 reward. The reward criteria has been drastically relaxed based on feedback from Twitter user @artemis_rsch, and in-line with the quality improvement of our report.
We would like to welcome all to attempt this challenge, include @SybilHorror6.
Letter: docs.google.com/document/d/1…
Beautiful, would love to read it. You can check their updated reduced list here as well
docs.google.com/document/d/1…
We welcome everybody to have a go at reviewing the raw data yourselves. There are 130 columns and 29,958 rows - it's a lot. It's also absolutely conclusive.
You can access the raw data (23,358 KB) here: docs.google.com/spreadsheets…
To verify our adjusted clusters we assessed the lowest quality clusters in our data. These were found by sorting the "multi_silhouette_score" column in ascending order, and picking out the 10 most commonly occurring groups from the bottom 5,000 wallets.
These are our findings:
1) Multichain 903 - 0/327 - one cluster
2) Multichain 182 - 0/246 - one cluster
3) Multichain 130 - 0/377 - one cluster (possibly same as 182)
4) Multichain 339 - 0/189 - one cluster
5) Multichain 522 - 10/620 wallets are in two different clusters (2x5). The two groups of 5 are extremely correlated (0.858+ and 0.933+). Pictures below.
6) Multichain 179 - 0/59 - one cluster
7) Multichain 110 - 0/297 - one cluster
8) Multichain 369 - 0/51 - one cluster
9) Multichain 488 - 0/79 - one cluster
10) Multichain 89 - 0/87 - one cluster
The multichain 522 “false positives” are sybil, but from different operations.
2322/2332 wallets are clearly identifiable as being in the correct, single operator cluster (99.6% rate of sybil operation crossover). This really is a remarkable rate for a report of this size, especially as this metric is from the absolute bottom of our report. We did not cherry pick the groups, you can verify our findings and analyse a random set of groups yourself.
We hope you will engage in dialogue to help us get our reports over the finishing line, we've put an immense amount of work into these, and these sybil operators do not deserve to receive an airdrop.
Team Opsec's Public Challenge: $100 x 10 rewards.
For every cluster you identify from our LayerZero report that is made up of false positives, we will send $100 to your wallet, no questions asked.
@PrimordialAA
View our 1933 clusters & challenge here:
docs.google.com/spreadsheets…
