Egothor/Radixor
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Refresh multilingual benchmarks and fix overlapping gold evaluation

Browse Source 
Recompute published benchmark results for all default language models,
exclude Polish Polimorf, add Hebrew documentation, and record the current
benchmark environment. Evaluate repeated surface forms as an overlapping
gold cover and publish only applicable metrics for candidate policies.
This commit is contained in:
Leo Galambos
2026-07-23 17:06:41 +02:00
parent 1f1b03c6a8
commit b29699b763
64 changed files with 5392 additions and 4765 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/stemmer-model-catalog.md Normal file
View File

@@ -0,0 +1,25 @@
# Published Stemmer Model Catalog
| Model ID | Language | Default | Coordinates | Version | Source | Repository | Source version | Revision | Revision status | License | Attribution | SHA-256 | Bytes |
|---|---|---:|---|---:|---|---|---|---|---|---|---|---|---:|
| cs-cz-default | CS_CZ | true | org.egothor:radixor-model-cs-cz-default | 1.0.0 | UniMorph | https://github.com/unimorph/ces | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Witold Kieraś is credited for the separate MorfFlex-CZ conversion | 62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4 | 142365 |
| da-dk-default | DA_DK | true | org.egothor:radixor-model-da-dk-default | 1.0.0 | UniMorph | https://github.com/unimorph/dan | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90 | 73030 |
| de-de-default | DE_DE | true | org.egothor:radixor-model-de-de-default | 1.0.0 | UniMorph | https://github.com/unimorph/deu | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5 | 838450 |
| es-es-default | ES_ES | true | org.egothor:radixor-model-es-es-default | 1.0.0 | UniMorph | https://github.com/unimorph/spa | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | 7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721 | 2269280 |
| fa-ir-default | FA_IR | true | org.egothor:radixor-model-fa-ir-default | 1.0.0 | UniMorph | https://github.com/unimorph/fas | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1 | 8934 |
| fi-fi-default | FI_FI | true | org.egothor:radixor-model-fi-fi-default | 1.0.0 | UniMorph | https://github.com/unimorph/fin | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f | 4867450 |
| fr-fr-default | FR_FR | true | org.egothor:radixor-model-fr-fr-default | 1.0.0 | UniMorph | https://github.com/unimorph/fra | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9 | 1117956 |
| he-il-default | HE_IL | true | org.egothor:radixor-model-he-il-default | 1.0.0 | UniMorph | https://github.com/unimorph/heb | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Omer Goldman (annotator); Wiktionary contributors | 9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256 | 179776 |
| hu-hu-default | HU_HU | true | org.egothor:radixor-model-hu-hu-default | 1.0.0 | UniMorph | https://github.com/unimorph/hun | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Christo Kirov, Ryan Cotterell, and Khuyagbaatar Batsuren (conversion); Judit Ács and Gábor Bella (validation); English Wiktionary contributors | 359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e | 2346297 |
| it-it-default | IT_IT | true | org.egothor:radixor-model-it-it-default | 1.0.0 | UniMorph | https://github.com/unimorph/ita | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12 | 841105 |
| nb-no-default | NB_NO | true | org.egothor:radixor-model-nb-no-default | 1.0.0 | UniMorph | https://github.com/unimorph/nob | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9 | 205645 |
| nl-nl-default | NL_NL | true | org.egothor:radixor-model-nl-nl-default | 1.0.0 | UniMorph | https://github.com/unimorph/nld | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e | 70600 |
| nn-no-default | NN_NO | true | org.egothor:radixor-model-nn-no-default | 1.0.0 | UniMorph | https://github.com/unimorph/nno | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398 | 51244 |
| pl-pl-polimorf | PL_PL | false | org.egothor:radixor-model-pl-pl-polimorf | 1.0.0 | PoliMorf 2.1 | https://github.com/morfologik/morfologik-stemming | 2.1 | 6e63b53 | recorded | BSD-2-Clause | Copyright (c) 2016, Marcin Miłkowski | 4fe4bf5e6c22c1beea5b3d57f1ce4c9ea5aac1ed8ab24c616fb06df745e40d15 | 12624997 |
| pl-pl-unimorph | PL_PL | true | org.egothor:radixor-model-pl-pl-unimorph | 1.0.0 | UniMorph | https://github.com/unimorph/pol | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; SGJP authors Marcin Woliński, Zygmunt Saloni, Robert Wołosz, Włodzimierz Gruszczyński, Danuta Skowrońska, and Zbigniew Bronk; Witold Kieraś (conversion); Wiktionary contributors | 8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721 | 334951 |
| pt-pt-default | PT_PT | true | org.egothor:radixor-model-pt-pt-default | 1.0.0 | UniMorph | https://github.com/unimorph/por | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d | 509965 |
| ru-ru-default | RU_RU | true | org.egothor:radixor-model-ru-ru-default | 1.0.0 | UniMorph | https://github.com/unimorph/rus | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40 | 2414507 |
| sv-se-default | SV_SE | true | org.egothor:radixor-model-sv-se-default | 1.0.0 | UniMorph | https://github.com/unimorph/swe | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c | 256300 |
| uk-ua-default | UK_UA | true | org.egothor:radixor-model-uk-ua-default | 1.0.0 | UniMorph | https://github.com/unimorph/ukr | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Witold Kieraś and Maria Shvedova are credited for the separate VESUM conversion; Wiktionary contributors | cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae | 47300 |
| us-uk-default | US_UK | true | org.egothor:radixor-model-us-uk-default | 1.0.0 | UniMorph | https://github.com/unimorph/eng | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460 | 2713666 |
| yi-default | YI | true | org.egothor:radixor-model-yi-default | 1.0.0 | UniMorph | https://github.com/unimorph/yid | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph | f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3 | 12222 |