Hunting for parallel data for Asian Languages? ParaCrawl just added 9 new bonus corpora. More info & paper by Philipp Koehn from @jhuclsp to be presented at WMT24 (#EMNLP2024): paracrawl.eu/moredata
The 9 datasets, as Bonus Release: paracrawl.eu/
Hi there, three new Bonus ParaCrawl languages have been just released:
- English- Azerbaijani
- English-Tajik
- English-Armenian
Go to the ParaCrawl website, scroll down to Bonus Languages (Low-Resource), download your preferred version: paracrawl.eu/
ParaCrawl retweeted
Interested in Open and Community-Driven MT initiatives? CrowdMT is for you!
🎙️Invited speakers from Wikimedia Foundation and Apertium announced.
📜Accepted papers and abstracts announced.
Time to register at events.tuni.fi/eamt23/regist…
Details: hplt-project.org/events
Parallel (en-*) and monolingual new corpora from #MaCoCu just released. Included languages:
Albanian
Bosnian
Bulgarian
Croatian
Macedonian
Maltese
Montenegrin
Serbian
Slovene
Turkish
We've published new #MaCoCu web corpora for 11 under-resourced languages!
56 million documents, 17 BILLION words (monolingual corpora) and 580 million words (English-X parallel corpora) were just uploaded to the CLARIN.SI repository (clarin.si/repository/xmlui/d…) 🥳
1
7
456
A new ParaCrawl parallel corpus is available!
🌍 languages: Polish-Czech
🎒 size: 24 million sentences
🗒️ license: CC0
🎯 location: paracrawl.eu bonus section
🧐 more info: paracrawl.eu/moredata
ParaCrawl retweeted
Indeed, this is the first data release of the #Macocu effort. You will find both monolingual and bilingual (with English) corpora on ELRC-Share and CLARIN repositories and the website. Insights coming soon! Most of the code also ready for you to try it out!
Massive AND high-quality corpora for Bulgarian, Croatian, Slovene, Macedonian, Icelandic, Maltese and Turkish, collected by the #MaCoCu project, are now available in our repository! Check them out and share the word:
➡️macocu.eu/
➡️clarin.si/repository/xmlui/
Check out MultiParacawl 9, including 36 parallel corpora for Ukrainian and a total of 705 bitexts. Thanks OPUS and @TiedemannJoerg to share this great resource! paracrawl.eu/news/item/18-mu…
ParaCrawl retweeted
If you have an MT system, try bleualign (github.com/bitextor/bleualig…) from @ParaCrawl . Scales to ParaCrawl-sized data.
We're back with more language resources: English-Ukrainian parallel corpus with aprox. 13M sentence pairs has been released. More info and downloads:
paracrawl.eu/news/item/17-en…
Please, spread the word and use it!
Done! All #ParaCrawl v9 corpora are now available at paracrawl.eu, some also on Corset corset.paracrawl.eu to further inspect or filter them and a new Bitextor is also out github.com/bitextor! Thanks to #CEF and the EU for co-funding this great project!
Summer was for work! Now #ParaCrawl v9 corpora are done and again bigger than the previous ones!🤩 Extrinsic evaluation through MT almost finished and, according to old BLEU and new COMET, the quality of the MT output improves! 🥳 We will share corpora and more results soon!🕑
Very clear TODO from #ParaCrawl's last stakeholder board meeting: we need better language identification, specially for closely-related languages and for under-resourced ones. Such a basic thing! Trying here to improve current results mixing Fastext and Hunspell, take a look👇
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
A new version of ParaCrawl is being cooked! We are aiming at not only more bilingual but also monolingual data. And we are applying neural cleaning this time with bicleaner-ai (github.com/bitextor/bicleane…). Stay tuned!🧐
Milestone reached! We just published Corset, a data selection portal to get relevant data from massive amounts of parallel data such as ParaCrawl corpora. Thanks #CEFTelecom! Users welcome!
Test it here: corset.paracrawl.eu/
Code & docs here: github.com/paracrawl/corset
We almost forgot to tell you, ParaCrawl 8 is out!
First highlight: wow the size of it!
Check yourself at paracrawl.eu/releases
#ParaCrawl #crawling #parallelcorpus #CEFTelecom #MT
ParaCrawl retweeted
Bitextor 8 is out! Many improvements and features that will make it into next @ParaCrawl data release, including ones from Snakemake 6 by @johanneskoester. Check all the changes: github.com/bitextor/bitextor…
Our corpora were evaluated as part of the great effort at "Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets" (🧐 arxiv.org/pdf/2103.12028.pdf). We will keep our efforts in trying to deliver high-quality corpora out of web crawled content. ParaCrawl v8 about to come!
