Text collection for training the BERTić transformer model BERTić-data

  Authors

Ljubešić, Nikola

  Item identifier

http://hdl.handle.net/11356/1426

 Project URL

https://huggingface.co/classla/bcms-bertic

 Demo URL

https://huggingface.co/classla/bcms-bertic-ner

 Referenced by

https://arxiv.org/abs/2104.09243

 Date issued

2021-05-05

 Type

corpus, text

 Size

8387681518 words

 Language(s)

Bosnian , Croatian , Montenegrin , Serbian

 Description

The BERTić-data text collection contains more than 8 billion tokens of mostly web-crawled text written in Bosnian, Croatian, Montenegrin or Serbian. The collection was used to train the BERTić transformer model (https://huggingface.co/classla/bcms-bertic). The data consists of web crawls before 2015, i.e. bsWaC (http://hdl.handle.net/11356/1062), hrWaC (http://hdl.handle.net/11356/1064), and srWaC (http://hdl.handle.net/11356/1063); previously unpublished 2019-2020 crawls, i.e. cnrWaC, CLASSLA-bs, CLASSLA-hr, and CLASSLA-sr; the cc100-hr and cc100-sr parts of CommonCrawl (https://commoncrawl.org/); and the Riznica corpus (http://hdl.handle.net/11356/1180). All texts were transliterated to the Latin script. The format of the text collection is one-sentence-per-line, empty-line-as-document-boundary. More details, especially on the applied near-deduplication procedure, can be found in the BERTić paper (https://arxiv.org/pdf/2104.09243.pdf).

 Publisher

Jožef Stefan Institute

 Acknowledgement

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

European Union’s Rights,Equality and Citizenship Programme 875263 "IMSyPP - Innovative Monitoring Systems and PreventionPolicies of Online Hate Speech"

ARRS (Slovenian Research Agency) N6-0099 "LiLaH: Linguistic Landscape of Hate Speech"

ARRS (Slovenian Research Agency) J7-8280 "FRENK: Resources, methods, and tools for the understanding, identification, and classification of various forms of socially unacceptable discourse in the information society"

 Subject(s)

web corpus language model

 Collection(s)

CLARIN.SI data & tools

Show full item record