DSI-enriched ParaCrawl 9 en-nl corpus

  Authors

Bañón, Marta ; et al.show everyone

Bañón, Marta ; Esplà-Gomis, Miquel ; Forcada, Mikel L. ; García-Romero, Cristian ; Kuzman, Taja ; Ljubešić, Nikola ; van Noord, Rik ; Pla Sempere, Leopoldo ; Ramírez-Sánchez, Gema ; Rupnik, Peter ; Suchomel, Vít ; Toral, Antonio ; van der Werff, Tobias ; Zaragoza, Jaume

  Item identifier

http://hdl.handle.net/11356/1527

 Project URL

https://macocu.eu/

 Date issued

2022-04-28

 Type

corpus, text

 Size

89135870 sentences

 Language(s)

Dutch , English

 Description

This is a derivative work based on Paracrawl release 9 English-Dutch (https://paracrawl.eu/). This version of the corpus includes a set of probabilities corresponding to the affinity of each segment pair to a specific Digital Service Infrastructure (DSI), which includes Cybersecurity, Electronic Exchange of Social Security Information, E-health, E-justice, Europeana, Online Dispute Resolution, Open Data Portal and Safer Internet. The model that assigned the probabilities is a fine-tuned pre-trained language model (DeBERTa-v3-large), trained on a crawled corpus of English DSI-specific texts. More information is available on the corresponding GitHub page: https://github.com/RikVN/DSI. The rest of the information in the original version of the corpus remained unchanged. Notice and take down: Should you consider that our data contains material that is owned by you and should therefore not be reproduced here, please: (1) Clearly identify yourself, with detailed contact data such as an address, telephone number or email address at which you can be contacted. (2) Clearly identify the copyrighted work claimed to be infringed. (3) Clearly identify the material that is claimed to be infringing and information reasonably sufficient in order to allow us to locate the material. (4) Please write to the contact person for this resource whose email is available in the full item record. We will comply with legitimate requests by removing the affected sources from the next release of the corpus. This action has received funding from the European Union's Connecting Europe Facility 2014-2020 - CEF Telecom, under Grant Agreement No. INEA/CEF/ICT/A2020/2278341. This communication reflects only the author’s view. The Agency is not responsible for any use that may be made of the information it contains.

 Publisher

Jožef Stefan Institute

 

Prompsit

 

Rijksuniversiteit Groningen

 

Universitat d'Alacant

 Acknowledgement

Connecting Europe Facility (CEF) Telecom INEA/CEF/ICT/A2020/2278341 "MaCoCu - Massive collection and curation of monolingual and bilingual data: focus on under-resourced languages"

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

Jožef Stefan Institute CLARIN "CLARIN.SI"

 Subject(s)

parallel corpus web corpus multilingual DSI

 Collection(s)

CLARIN.SI data & tools

Show full item record