Refresh multilingual benchmarks and fix overlapping gold evaluation
Recompute published benchmark results for all default language models, exclude Polish Polimorf, add Hebrew documentation, and record the current benchmark environment. Evaluate repeated surface forms as an overlapping gold cover and publish only applicable metrics for candidate policies.
This commit is contained in:
22
README.md
22
README.md
@@ -87,16 +87,16 @@ Radixor performance is best read together with stemming quality. The English dic
|
||||
|
||||
| Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 23.113 | 7.065 | 109.8 |
|
||||
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 21.270 | 9.914 | 101.0 |
|
||||
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 19.170 | 6.609 | 91.1 |
|
||||
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 20.857 | 6.734 | 99.1 |
|
||||
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 14.975 | 1.215 | 71.1 |
|
||||
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 15.249 | 1.078 | 72.4 |
|
||||
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 15.323 | 2.340 | 72.8 |
|
||||
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 16.778 | 2.643 | 79.7 |
|
||||
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 18.929 | 3.241 | 89.9 |
|
||||
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 19.124 | 1.883 | 90.9 |
|
||||
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 20.627 | 2.117 | 98.0 |
|
||||
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 21.713 | 2.104 | 103.2 |
|
||||
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 17.408 | 1.438 | 82.7 |
|
||||
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 16.946 | 1.531 | 80.5 |
|
||||
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 15.735 | 1.278 | 74.8 |
|
||||
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 14.714 | 1.089 | 69.9 |
|
||||
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 15.090 | 1.254 | 71.7 |
|
||||
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 13.773 | 1.071 | 65.4 |
|
||||
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 15.396 | 2.497 | 73.1 |
|
||||
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 16.970 | 2.847 | 80.6 |
|
||||
|
||||
Column meanings:
|
||||
|
||||
@@ -109,7 +109,7 @@ Column meanings:
|
||||
- `Error ms` is the JMH score error converted to milliseconds.
|
||||
- `ns/token` is average nanoseconds per changed token in that operation.
|
||||
|
||||
The contracted trie result is materially stronger than the older uncontracted profile: full English coverage reaches 97.478% all-token exactness and 97.197% changed-token exactness at 109.8 ns/token, while even a 10% deterministic dictionary slice remains at 92.868% all-token exactness and 76.516% changed-token exactness at 90.9 ns/token. This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge.
|
||||
The contracted trie result is materially stronger than the older uncontracted profile: full English coverage reaches 97.478% all-token exactness and 97.197% changed-token exactness at 98.0 ns/token, while even a 10% deterministic dictionary slice remains at 92.868% all-token exactness and 76.516% changed-token exactness at 80.6 ns/token. This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge.
|
||||
|
||||
For benchmark scope, workload design, environment, commands, report locations, and interpretation guidance, see [Benchmarking](docs/benchmarking.md).
|
||||
|
||||
|
||||
55
build.gradle
55
build.gradle
@@ -715,9 +715,18 @@ def modelCatalogText = providers.provider {
|
||||
return output.toString()
|
||||
}
|
||||
|
||||
def modelCatalogDocumentationInputs = files(modelProjects().collectMany { Project modelProject ->
|
||||
[
|
||||
modelProject.file('build.gradle'),
|
||||
modelProject.file('model-version.txt'),
|
||||
modelProject.file('src/modelInput/stemmer.gz')
|
||||
]
|
||||
})
|
||||
|
||||
tasks.register('generateModelCatalogDocumentation') {
|
||||
group = 'documentation'
|
||||
description = 'Generates the deterministic model catalog in the build documentation staging tree.'
|
||||
inputs.files(modelCatalogDocumentationInputs)
|
||||
outputs.file(layout.buildDirectory.file('mkdocs-source/stemmer-model-catalog.md'))
|
||||
doLast {
|
||||
File catalog = layout.buildDirectory.file('mkdocs-source/stemmer-model-catalog.md').get().asFile
|
||||
@@ -726,10 +735,24 @@ tasks.register('generateModelCatalogDocumentation') {
|
||||
}
|
||||
}
|
||||
|
||||
tasks.register('publishModelCatalogDocumentation') {
|
||||
group = 'documentation'
|
||||
description = 'Updates the checked-in model catalog used by a direct local MkDocs invocation.'
|
||||
dependsOn(tasks.named('generateModelCatalogDocumentation'))
|
||||
inputs.file(layout.buildDirectory.file('mkdocs-source/stemmer-model-catalog.md'))
|
||||
outputs.file(layout.projectDirectory.file('docs/stemmer-model-catalog.md'))
|
||||
doLast {
|
||||
File generated = layout.buildDirectory.file('mkdocs-source/stemmer-model-catalog.md').get().asFile
|
||||
File published = layout.projectDirectory.file('docs/stemmer-model-catalog.md').asFile
|
||||
published.setText(generated.getText('UTF-8'), 'UTF-8')
|
||||
}
|
||||
}
|
||||
|
||||
tasks.register('prepareMkDocsSource', Sync) {
|
||||
group = 'documentation'
|
||||
description = 'Stages maintained documentation, generated catalog, and MkDocs configuration under build/.'
|
||||
dependsOn(modelProjects().collect { Project modelProject -> modelProject.path + ':verifyModelDescriptor' })
|
||||
inputs.files(modelCatalogDocumentationInputs)
|
||||
into(layout.buildDirectory.dir('mkdocs-source'))
|
||||
from(layout.projectDirectory.dir('docs'))
|
||||
doLast {
|
||||
@@ -746,7 +769,7 @@ tasks.register('prepareMkDocsSource', Sync) {
|
||||
|
||||
tasks.register('verifyModelCatalogDocumentation') {
|
||||
group = 'verification'
|
||||
description = 'Validates model metadata and the generated build-directory MkDocs catalog.'
|
||||
description = 'Validates model metadata and both the checked-in and staged MkDocs catalogs.'
|
||||
dependsOn(tasks.named('prepareMkDocsSource'))
|
||||
dependsOn(tasks.named('verifyAllDefaultModels'))
|
||||
doLast {
|
||||
@@ -755,6 +778,11 @@ tasks.register('verifyModelCatalogDocumentation') {
|
||||
if (!catalog.isFile() || catalog.getText('UTF-8') != expected) {
|
||||
throw new GradleException('The staged model catalog is missing or nondeterministic.')
|
||||
}
|
||||
File publishedCatalog = layout.projectDirectory.file('docs/stemmer-model-catalog.md').asFile
|
||||
if (!publishedCatalog.isFile() || publishedCatalog.getText('UTF-8') != expected) {
|
||||
throw new GradleException(
|
||||
'The checked-in model catalog is stale; run ./gradlew publishModelCatalogDocumentation.')
|
||||
}
|
||||
List<String> identifiers = modelProjects().collect { Project modelProject -> modelProject.name }
|
||||
if (identifiers != identifiers.sort()) {
|
||||
throw new GradleException('Published model projects are not in deterministic model-ID order.')
|
||||
@@ -954,6 +982,31 @@ tasks.register('stemmingQuality', JavaExec) {
|
||||
maxHeapSize = '6g'
|
||||
}
|
||||
|
||||
tasks.register('benchmarkCorpusReport', JavaExec) {
|
||||
group = 'verification'
|
||||
description = 'Reports corpus and preferred patch-command counts for every default model.'
|
||||
dependsOn(tasks.named('jmhClasses'))
|
||||
classpath = files(sourceSets.jmh.runtimeClasspath, configurations.stemmingQualityJmhRuntime)
|
||||
mainClass = 'org.egothor.stemmer.benchmark.BenchmarkCorpusReportApplication'
|
||||
args layout.buildDirectory.file('reports/jmh/benchmark-corpora.csv').get().asFile.absolutePath
|
||||
maxHeapSize = '6g'
|
||||
}
|
||||
|
||||
tasks.register('writeJmhRuntimeClasspath') {
|
||||
group = 'verification'
|
||||
description = 'Writes the complete modular JMH runtime classpath for isolated direct JMH execution.'
|
||||
dependsOn(tasks.named('jmhJar'))
|
||||
dependsOn(modelProjects().collect { Project modelProject -> modelProject.tasks.named('jar') })
|
||||
outputs.file(layout.buildDirectory.file('reports/jmh/jmh-runtime-classpath.txt'))
|
||||
doLast {
|
||||
File report = layout.buildDirectory.file('reports/jmh/jmh-runtime-classpath.txt').get().asFile
|
||||
report.parentFile.mkdirs()
|
||||
File executable = tasks.named('jmhJar', Jar).get().archiveFile.get().asFile
|
||||
report.setText(executable.absolutePath + File.pathSeparator
|
||||
+ sourceSets.jmh.runtimeClasspath.asPath + System.lineSeparator(), 'UTF-8')
|
||||
}
|
||||
}
|
||||
|
||||
tasks.register('prepareBenchmarkModelInputs', Sync) {
|
||||
group = 'verification'
|
||||
description = 'Prepares default model inputs for JMH and quality evaluation without changing source data.'
|
||||
|
||||
@@ -67,64 +67,62 @@
|
||||
padding: 0.45rem 0.7rem;
|
||||
}
|
||||
|
||||
/* Publication-quality benchmark tables retain identity columns while scrolling. */
|
||||
.quality-table {
|
||||
max-width: 100%;
|
||||
overflow-x: auto;
|
||||
/* Primary quality rankings fit the content column; raw counts remain in details. */
|
||||
.quality-summary {
|
||||
width: 100%;
|
||||
margin: 0.65rem 0 1rem;
|
||||
border: 1px solid var(--md-default-fg-color--lightest);
|
||||
border-radius: 0.2rem;
|
||||
scrollbar-gutter: stable;
|
||||
font-size: 0.72rem;
|
||||
}
|
||||
|
||||
.quality-table:focus {
|
||||
outline: 0.15rem solid var(--md-accent-fg-color);
|
||||
outline-offset: 0.1rem;
|
||||
}
|
||||
|
||||
.quality-table::before {
|
||||
content: "Scrollable table: Rank, Stemmer, and Output policy remain visible.";
|
||||
.quality-summary .md-typeset__table,
|
||||
.quality-summary table {
|
||||
display: block;
|
||||
padding: 0.35rem 0.55rem;
|
||||
color: var(--md-default-fg-color--light);
|
||||
font-size: 0.68rem;
|
||||
}
|
||||
|
||||
.quality-table .md-typeset__table,
|
||||
.quality-table table {
|
||||
width: 100%;
|
||||
margin: 0;
|
||||
overflow: visible;
|
||||
}
|
||||
|
||||
.quality-table table th:nth-child(1),
|
||||
.quality-table table td:nth-child(1),
|
||||
.quality-table table th:nth-child(2),
|
||||
.quality-table table td:nth-child(2),
|
||||
.quality-table table th:nth-child(3),
|
||||
.quality-table table td:nth-child(3) {
|
||||
position: sticky;
|
||||
z-index: 2;
|
||||
background: var(--md-default-bg-color);
|
||||
background-clip: padding-box;
|
||||
.quality-summary table {
|
||||
display: table;
|
||||
table-layout: fixed;
|
||||
}
|
||||
|
||||
.quality-table table th:nth-child(1),
|
||||
.quality-table table td:nth-child(1) {
|
||||
left: 0;
|
||||
min-width: 2.8rem;
|
||||
.quality-summary th,
|
||||
.quality-summary td {
|
||||
padding: 0.4rem 0.45rem !important;
|
||||
line-height: 1.3;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
.quality-table table th:nth-child(2),
|
||||
.quality-table table td:nth-child(2) {
|
||||
left: 2.8rem;
|
||||
min-width: 13rem;
|
||||
white-space: normal;
|
||||
.quality-summary th:nth-child(1),
|
||||
.quality-summary td:nth-child(1) {
|
||||
width: 7%;
|
||||
}
|
||||
|
||||
.quality-table table th:nth-child(3),
|
||||
.quality-table table td:nth-child(3) {
|
||||
left: 15.8rem;
|
||||
min-width: 8.5rem;
|
||||
box-shadow: 0.2rem 0 0.25rem rgb(0 0 0 / 8%);
|
||||
.quality-summary th:nth-child(2),
|
||||
.quality-summary td:nth-child(2) {
|
||||
width: 39%;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.quality-summary th:nth-child(n + 3),
|
||||
.quality-summary td:nth-child(n + 3) {
|
||||
width: 18%;
|
||||
}
|
||||
|
||||
.quality-summary td:nth-child(n + 3) {
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.quality-summary--oracle th:nth-child(1),
|
||||
.quality-summary--oracle td:nth-child(1) {
|
||||
width: 46%;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.quality-summary--oracle th:nth-child(n + 2),
|
||||
.quality-summary--oracle td:nth-child(n + 2) {
|
||||
width: 27%;
|
||||
}
|
||||
|
||||
.quality-details > summary {
|
||||
@@ -132,34 +130,17 @@
|
||||
}
|
||||
|
||||
@media screen and (max-width: 44.99em) {
|
||||
.quality-table table th:nth-child(2),
|
||||
.quality-table table td:nth-child(2) {
|
||||
min-width: 10rem;
|
||||
.quality-summary {
|
||||
font-size: 0.62rem;
|
||||
}
|
||||
|
||||
.quality-table table th:nth-child(3),
|
||||
.quality-table table td:nth-child(3) {
|
||||
position: static;
|
||||
min-width: 7.5rem;
|
||||
box-shadow: none;
|
||||
.quality-summary th,
|
||||
.quality-summary td {
|
||||
padding: 0.3rem 0.2rem !important;
|
||||
}
|
||||
}
|
||||
|
||||
@media print {
|
||||
.quality-table {
|
||||
overflow: visible;
|
||||
border: 0;
|
||||
}
|
||||
|
||||
.quality-table::before {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.quality-table table th,
|
||||
.quality-table table td {
|
||||
position: static !important;
|
||||
}
|
||||
|
||||
.quality-details:not([open]) > *:not(summary) {
|
||||
display: block;
|
||||
}
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Benchmarking
|
||||
|
||||
Radixor contains internal trie microbenchmarks, a separate stemmer comparison suite, and a dictionary coverage benchmark for Radixor itself. Published stemmer comparison results must come only from benchmark classes matching `.*StemmerComparisonBenchmark.*`; internal `FrequencyTrie*` microbenchmarks are not part of those results.
|
||||
Radixor contains internal trie microbenchmarks, a separate stemmer comparison suite, and a dictionary coverage benchmark for Radixor itself. The current default-model publication uses the same-language speed and exact-root methods selected by the command recorded on the [environment page](benchmarks/reference/environment.md). Internal `FrequencyTrie*` microbenchmarks, the optional `PolishPolimorfStemmerComparisonBenchmark`, and the separate German CISTEM gold-standard experiment are not part of these language tables.
|
||||
|
||||
Every current default Radixor benchmark scenario uses the model ID declared by its `Language.defaultModelId()`. The root JMH runtime configuration depends directly on all default model projects plus optional `pl-pl-polimorf`; no benchmark-pack project or artifact exists. These dependencies are benchmark-only and never enter the root published POM. A PoliMorf comparison must be labeled with model ID `pl-pl-polimorf`, while the default Polish row remains `pl-pl-unimorph`.
|
||||
|
||||
The optional model now has a verified complete compiled loading path. This does not alter existing benchmark rows or make PoliMorf part of the representative English JMH run. Any future full PoliMorf benchmark must provision its documented startup heap independently and record the exact model artifact version and checksum.
|
||||
The optional model now has a verified complete compiled loading path. It is not included in the 2026-07-23 corpus, accuracy, speed, coverage, or stemming-quality measurements. Any future full PoliMorf benchmark must provision its documented startup heap independently and record the exact model artifact version and checksum.
|
||||
|
||||
This page is the entry point for benchmark interpretation. Detailed tables and long reference material are split into focused subpages so that important points do not get buried.
|
||||
|
||||
@@ -14,7 +14,7 @@ This page is the entry point for benchmark interpretation. Detailed tables and l
|
||||
- Radixor is the quality-oriented baseline in same-language comparisons. Its exact-root accuracy is often close to 100%, while many faster competitors are light, minimal, possessive, or aggressive rule-based stemmers with much lower root agreement.
|
||||
- The measured Radixor cost buys dictionary-trained stemming precision. That precision improves search quality by mapping inflected forms to intended dictionary roots instead of approximate or over-reduced stems.
|
||||
- Speed benchmarks process changed dictionary tokens where the surface form differs from the expected root. Accuracy benchmarks process the complete dictionary.
|
||||
- Accuracy tables use deterministic auxiliary counters from the current JMH reports. Repeated measurement samples duplicate the same exact-root accounting and are not interpreted as timing results.
|
||||
- Accuracy tables use deterministic auxiliary counters from a single non-timed JMH evaluation, while Radixor counters are independently cross-checked by the default-model corpus report. Runtime scores from accuracy methods are not interpreted.
|
||||
- The historical Porter performance badge is retired. Benchmark reporting now uses speed and quality tables rather than a single Porter ratio.
|
||||
|
||||
## Benchmark Documentation Map
|
||||
@@ -44,4 +44,4 @@ The [English dictionary coverage benchmark](benchmarks/reference/english-coverag
|
||||
The current measured language results are published in [Language Benchmark Pages](benchmarks/languages/index.md). Generated local report files for this benchmark update are listed in [Benchmark environment and reports](benchmarks/reference/environment.md).
|
||||
|
||||
JMH TXT and CSV reports are still published as benchmark artifacts. They are no longer converted into a Shields endpoint benchmark badge.
|
||||
Model IDs, independent artifact versions, and descriptor checksums identify inputs for future reproducibility. Historical snapshots remain tied to the model inputs used when measured; the optional PoliMorf model must not be retroactively attributed to results that predate it. See [Model Selection and Loading](model-selection-and-loading.md) and [Reproducibility](benchmarks/reference/reproducibility.md).
|
||||
Model IDs, independent artifact versions, and descriptor checksums identify the inputs in the checked corpus snapshot. The optional PoliMorf model must not be attributed to the default Polish results. See [Model Selection and Loading](model-selection-and-loading.md) and [Reproducibility](benchmarks/reference/reproducibility.md).
|
||||
|
||||
@@ -1,309 +1,309 @@
|
||||
Stemmer,Language,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,Forms with multiple candidates,Maximum candidates for one form,Total candidate assignments,Distinct output stems,True-positive pairs,False-positive pairs,False-negative pairs,True-negative pairs,Over-stemming error pairs,Over-stemming possible pairs,Over-stemming percentage,Under-stemming error pairs,Under-stemming possible pairs,Under-stemming percentage,Pairwise precision,Pairwise recall,Pairwise specificity,Pairwise accuracy,Balanced accuracy,Pairwise F0.5,Pairwise F1,Pairwise F2,Jaccard index,Fowlkes-Mallows index,Matthews correlation coefficient,Pairwise error rate,Adjusted Rand Index,Homogeneity,Completeness,V-measure,Normalized mutual information
|
||||
"CZECH_LUCENE_CZECH_STEM_FILTER","CS_CZ","ALL_WORDS","PRIMARY_OUTPUT","5113","51676","2","51676","0","1","51676","9647","177249","14480","124586","1334862335","14480","1334876815","0.001085","124586","301835","41.276194","0.924476735392","0.587238060530","0.999989152557","0.999895844650","0.793613606543","0.829234311828","0.718241200736","0.633453389361","0.560355974266","0.736809286788","0.736765291417","0.000104155350","0.718191706079","0.993800637348","0.944976928457","0.968774025802","0.968774025802"
|
||||
"CZECH_LUCENE_CZECH_STEM_FILTER","CS_CZ","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50968","2","50968","0","1","50968","9558","174387","13950","124426","1298530265","13950","1298544215","0.001074","124426","298813","41.640089","0.925930645598","0.583599107134","0.999989257201","0.999893462107","0.791794182167","0.828708724235","0.715947860002","0.630197985095","0.557569149804","0.735100195918","0.735055396892","0.000106537893","0.715897321649","0.993897397445","0.944297456221","0.968462775946","0.968462775946"
|
||||
"CZECH_RADIXOR","CS_CZ","ALL_WORDS","PRIMARY_OUTPUT","5113","51676","2","51676","0","1","51676","5162","299762","3867","2073","1334872948","3867","1334876815","0.000290","2073","301835","0.686799","0.987264062392","0.993132009210","0.999997103103","0.999995551157","0.996564556157","0.988432097845","0.990189342389","0.991952846154","0.980569312599","0.990193689085","0.990191466141","0.000004448843","0.990187117482","0.998733220675","0.998685552738","0.998709386137","0.998709386137"
|
||||
"CZECH_RADIXOR","CS_CZ","ALL_WORDS","ANY_CANDIDATE","5113","51676","2","51080","596","4","52319","5166","301835","0","0","1334876815","0","1334876815","0.000000","0","301835","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","ALL_WORDS","ALL_CANDIDATES","5113","51676","2","51080","596","4","52319","5166","301835","5850","0","1334870965","5850","1334876815","0.000438","0","301835","0.000000","0.980987048442","1.000000000000","0.999995617573","0.999995618564","0.999997808787","0.984731579205","0.990402283764","0.996138677580","0.980987048442","0.990447902942","0.990445732657","0.000004381436","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50968","2","50968","0","1","50968","5037","297104","3863","1709","1298540352","3863","1298544215","0.000297","1709","298813","0.571930","0.987164705765","0.994280703985","0.999997025130","0.999995710028","0.997138864558","0.988579745136","0.990709926973","0.992849308824","0.981590876052","0.990716315904","0.990714173387","0.000004289972","0.990707781520","0.998726091764","0.999029907266","0.998877976413","0.998877976413"
|
||||
"CZECH_RADIXOR","CS_CZ","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","5038","50968","2","50428","540","4","51543","5040","298813","0","0","1298544215","0","1298544215","0.000000","0","298813","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","5038","50968","2","50428","540","4","51543","5040","298813","5782","0","1298538433","5782","1298544215","0.000445","0","298813","0.000000","0.981017416570","1.000000000000","0.999995547321","0.999995548346","0.999997773661","0.984756059381","0.990417760454","0.996144940117","0.981017416570","0.990463233325","0.990461028216","0.000004451654","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","ALL_WORDS","PRIMARY_OUTPUT","4179","28079","32","28079","0","1","28079","4184","89188","1165","707","394110021","1165","394111186","0.000296","707","89895","0.786473","0.987106128186","0.992135268925","0.999997043981","0.999995251155","0.996066156453","0.988107873355","0.989614309174","0.991125345329","0.979442126071","0.989617503860","0.989615130363","0.000004748845","0.989611934224","0.998465862775","0.998718664384","0.998592247580","0.998592247580"
|
||||
"DA_DK_RADIXOR","DA_DK","ALL_WORDS","ANY_CANDIDATE","4179","28079","32","27756","323","3","28405","4187","89895","0","0","394111186","0","394111186","0.000000","0","89895","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","ALL_WORDS","ALL_CANDIDATES","4179","28079","32","27756","323","3","28405","4187","89895","1849","0","394109337","1849","394111186","0.000469","0","89895","0.000000","0.979846093478","1.000000000000","0.999995308431","0.999995309500","0.999997654215","0.983811622975","0.989820468071","0.995903164910","0.979846093478","0.989871756076","0.989869434047","0.000004690500","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","28033","32","28033","0","1","28033","4170","89077","1165","663","392819623","1165","392820788","0.000297","663","89740","0.738801","0.987090268389","0.992611990194","0.999997034271","0.999995347541","0.996304512232","0.988189692661","0.989843428787","0.991502709249","0.979891095099","0.989847279032","0.989844954043","0.000004652459","0.989841102043","0.998463063294","0.998811876590","0.998637439483","0.998637439483"
|
||||
"DA_DK_RADIXOR","DA_DK","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4173","28033","32","27718","315","3","28351","4173","89740","0","0","392820788","0","392820788","0.000000","0","89740","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4173","28033","32","27718","315","3","28351","4173","89740","1849","0","392818939","1849","392820788","0.000471","0","89740","0.000000","0.979811986156","1.000000000000","0.999995293019","0.999995294094","0.999997646509","0.983784115625","0.989803065147","0.995896117847","0.979811986156","0.989854527774","0.989852198158","0.000004705906","","","","",""
|
||||
"ENGLISH_LUCENE_KSTEM_FILTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","371125","237565","1368501","76305","184489083270","1368501","184490451771","0.000742","76305","313870","24.311020","0.147917333410","0.756889795138","0.999992582267","0.999992168681","0.878441188702","0.176283968232","0.247471790726","0.415099040868","0.141208449266","0.334599940499","0.334597833111","0.000007831319","0.247469648794","0.980686838187","0.992107972963","0.986364345289","0.986364345289"
|
||||
"ENGLISH_LUCENE_KSTEM_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","347624","237551","1367069","74340","170473473135","1367069","170474840204","0.000802","74340","311891","23.835250","0.148041904002","0.761647498645","0.999991980817","0.999991544756","0.880819739731","0.176476898525","0.247899438093","0.416437018089","0.141486991947","0.335791223646","0.335788961354","0.000008455244","0.247897133948","0.979822223835","0.991990504792","0.985868818390","0.985868818390"
|
||||
"ENGLISH_LUCENE_MINIMAL_FILTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","453328","137225","1122264","176645","184489329507","1122264","184490451771","0.000608","176645","313870","56.279670","0.108952916619","0.437203300730","0.999993916953","0.999992959490","0.718598608842","0.128203906480","0.174435713655","0.272816484020","0.095551668577","0.218253464509","0.218250987161","0.000007040510","0.174433464995","0.995202198233","0.981173943304","0.988138284715","0.988138284715"
|
||||
"ENGLISH_LUCENE_MINIMAL_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","430129","136932","1120871","174959","170473719333","1120871","170474840204","0.000657","174959","311891","56.096200","0.108866014789","0.439037997249","0.999993425006","0.999992398716","0.719515711128","0.128139023335","0.174469673707","0.273277328232","0.095572048952","0.218623688336","0.218621020778","0.000007601284","0.174467253215","0.994993790771","0.980519680106","0.987703711280","0.987703711280"
|
||||
"ENGLISH_LUCENE_PORTER_COPIED","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","319968","285390","1557406","28480","184488894365","1557406","184490451771","0.000844","28480","313870","9.073820","0.154867928951","0.909261796285","0.999991558338","0.999991403982","0.954626677312","0.185678591198","0.264658505304","0.460562583837","0.152510906996","0.375253902399","0.375251973425","0.000008596018","0.264656367392","0.969648379409","0.997199419831","0.983230936080","0.983230936080"
|
||||
"ENGLISH_LUCENE_PORTER_COPIED","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","298779","283761","1552702","28130","170473287502","1552702","170474840204","0.000911","28130","311891","9.019177","0.154514956196","0.909808234287","0.999990891899","0.999990726907","0.954899563093","0.185277176317","0.264165961476","0.460049474275","0.152183881415","0.374938634269","0.374936557303","0.000009273093","0.264163659878","0.968644600847","0.997108656044","0.982670549062","0.982670549062"
|
||||
"ENGLISH_LUCENE_PORTER_FILTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","319968","285390","1557406","28480","184488894365","1557406","184490451771","0.000844","28480","313870","9.073820","0.154867928951","0.909261796285","0.999991558338","0.999991403982","0.954626677312","0.185678591198","0.264658505304","0.460562583837","0.152510906996","0.375253902399","0.375251973425","0.000008596018","0.264656367392","0.969648379409","0.997199419831","0.983230936080","0.983230936080"
|
||||
"ENGLISH_LUCENE_PORTER_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","298779","283761","1552702","28130","170473287502","1552702","170474840204","0.000911","28130","311891","9.019177","0.154514956196","0.909808234287","0.999990891899","0.999990726907","0.954899563093","0.185277176317","0.264165961476","0.460049474275","0.152183881415","0.374938634269","0.374936557303","0.000009273093","0.264163659878","0.968644600847","0.997108656044","0.982670549062","0.982670549062"
|
||||
"ENGLISH_LUCENE_POSSESSIVE_FILTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","591899","7","1115154","313863","184489336617","1115154","184490451771","0.000604","313863","313870","99.997770","0.000006277121","0.000022302227","0.999993955492","0.999992254263","0.500008128860","0.000007330589","0.000009796848","0.000014763939","0.000004898448","0.000011831896","0.000008625150","0.000007745737","0.000007141644","0.995789196698","0.958018540631","0.976538780935","0.976538780935"
|
||||
"ENGLISH_LUCENE_POSSESSIVE_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","568400","5","1113773","311886","170473726431","1113773","170474840204","0.000653","311886","311891","99.998397","0.000004489225","0.000016031242","0.999993466643","0.999991637145","0.500004748942","0.000005244385","0.000007014251","0.000010587200","0.000003507138","0.000008483387","0.000005026087","0.000008362855","0.000004155674","0.995605378040","0.956423154691","0.975621022465","0.975621022465"
|
||||
"ENGLISH_OPENNLP_PORTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","319968","285390","1557406","28480","184488894365","1557406","184490451771","0.000844","28480","313870","9.073820","0.154867928951","0.909261796285","0.999991558338","0.999991403982","0.954626677312","0.185678591198","0.264658505304","0.460562583837","0.152510906996","0.375253902399","0.375251973425","0.000008596018","0.264656367392","0.969648379409","0.997199419831","0.983230936080","0.983230936080"
|
||||
"ENGLISH_OPENNLP_PORTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","298779","283761","1552702","28130","170473287502","1552702","170474840204","0.000911","28130","311891","9.019177","0.154514956196","0.909808234287","0.999990891899","0.999990726907","0.954899563093","0.185277176317","0.264165961476","0.460049474275","0.152183881415","0.374938634269","0.374936557303","0.000009273093","0.264163659878","0.968644600847","0.997108656044","0.982670549062","0.982670549062"
|
||||
"ENGLISH_PAICE_HUSK_LANCASTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","268169","283991","3062661","29879","184487389110","3062661","184490451771","0.001660","29879","313870","9.519546","0.084858240415","0.904804536910","0.999983399352","0.999983237427","0.952393968131","0.103642734217","0.155164208820","0.308542866654","0.084107327905","0.277092260667","0.277089454298","0.000016762573","0.155161580693","0.937768073854","0.996599815184","0.966289292109","0.966289292109"
|
||||
"ENGLISH_PAICE_HUSK_LANCASTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","249411","282398","3045870","29493","170471794334","3045870","170474840204","0.001787","29493","311891","9.456188","0.084848335531","0.905438117804","0.999982133023","0.999981960051","0.952710125414","0.103632575002","0.155156958803","0.308575577075","0.084103067491","0.277173081705","0.277170064389","0.000018039949","0.155154132315","0.936076835754","0.996486960554","0.965337716341","0.965337716341"
|
||||
"ENGLISH_RADIXOR","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","390361","292001","1149886","21869","184489301885","1149886","184490451771","0.000623","21869","313870","6.967534","0.202513095686","0.930324656705","0.999993767233","0.999993648707","0.965159211969","0.240076409811","0.332621199859","0.541270431499","0.199487482886","0.434054059102","0.434052478080","0.000006351293","0.332619335001","0.994214506865","0.997769723414","0.995988942533","0.995988942533"
|
||||
"ENGLISH_RADIXOR","US_UK","ALL_WORDS","ANY_CANDIDATE","396939","607439","250964","578231","29208","1355","2838145","397392","313855","12","15","184490451759","12","184490451771","0.000000","15","313870","0.004779","0.999961767245","0.999952209513","0.999999999935","0.999999999854","0.999976104724","0.999959855684","0.999956988357","0.999954121045","0.999913980413","0.999956988368","0.999956988295","0.000000000146","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","ALL_WORDS","ALL_CANDIDATES","396939","607439","250964","578231","29208","1355","2838145","397392","313855","11482166","15","184478969605","11482166","184490451771","0.006224","15","313870","0.004779","0.026606853277","0.999952209513","0.999937762817","0.999937762842","0.999944986165","0.033038791524","0.051834488023","0.120237128281","0.026606819443","0.163112175274","0.163107098882","0.000062237158","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","367590","290572","1148489","21319","170473691715","1148489","170474840204","0.000674","21319","311891","6.835401","0.201917778329","0.931645991709","0.999993263000","0.999993137956","0.965819627354","0.239424468968","0.331901731173","0.540775136091","0.198970131062","0.433723286019","0.433721583515","0.000006862044","0.331899721995","0.993959181482","0.997731171071","0.995841604460","0.995841604460"
|
||||
"ENGLISH_RADIXOR","US_UK","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","374384","583910","228735","555084","28826","1355","2812871","374506","311891","0","0","170474840204","0","170474840204","0.000000","0","311891","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","374384","583910","228735","555084","28826","1355","2812871","374506","311891","11470018","0","170463370186","11470018","170474840204","0.006728","0","311891","0.000000","0.026472025883","1.000000000000","0.999932717239","0.999932717362","0.999966358619","0.032872482055","0.051578660140","0.119686728696","0.026472025883","0.162702261457","0.162696787836","0.000067282638","","","","",""
|
||||
"ENGLISH_SNOWBALL_ORIGINAL_PORTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","321092","285304","1555293","28566","184488896478","1555293","184490451771","0.000843","28566","313870","9.101220","0.155006228957","0.908987797496","0.999991569791","0.999991414969","0.954489683644","0.185835337999","0.264848800190","0.460750814660","0.152637303435","0.375364850057","0.375362921954","0.000008585031","0.264846663203","0.969891477221","0.997192899073","0.983352728141","0.983352728141"
|
||||
"ENGLISH_SNOWBALL_ORIGINAL_PORTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","299877","283675","1550615","28216","170473289589","1550615","170474840204","0.000910","28216","311891","9.046750","0.154651118416","0.909532496930","0.999990904142","0.999990738644","0.954761700536","0.185431499934","0.264353286139","0.460234326480","0.152308234175","0.375046954242","0.375044878196","0.000009261356","0.264350985524","0.968893806180","0.997101844661","0.982795461434","0.982795461434"
|
||||
"ENGLISH_SNOWBALL_PORTER2","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","318385","285334","1566711","28536","184488885060","1566711","184490451771","0.000849","28536","313870","9.091662","0.154064291094","0.909083378469","0.999991507902","0.999991353242","0.954537443185","0.184752753479","0.263476636895","0.459101696688","0.151726514306","0.374242282819","0.374240346981","0.000008646758","0.263474493989","0.969037354042","0.997181597682","0.982908049045","0.982908049045"
|
||||
"ENGLISH_SNOWBALL_PORTER2","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","297220","283730","1561891","28161","170473278313","1561891","170474840204","0.000916","28161","311891","9.029116","0.153731454074","0.909708840589","0.999990837997","0.999990672823","0.954849839293","0.184374949232","0.263015918336","0.458637294569","0.151421029768","0.373966392672","0.373964308569","0.000009327177","0.263013611479","0.968019617024","0.997095706553","0.982342555079","0.982342555079"
|
||||
"FINNISH_LUCENE_FINNISH_LIGHT_STEM_FILTER","FI_FI","ALL_WORDS","PRIMARY_OUTPUT","57027","1811717","292","1811717","0","1","1811717","439975","12355389","2223150","19168306","1641124591341","2223150","1641126814491","0.000135","19168306","31523695","60.806025","0.847505295284","0.391939745642","0.999998645351","0.999986965635","0.695969195497","0.687649407375","0.535999578676","0.439151826652","0.366119825424","0.576342788507","0.576337821084","0.000013034365","0.535993941880","0.988126027331","0.886473473160","0.934543630400","0.934543630400"
|
||||
"FINNISH_LUCENE_FINNISH_LIGHT_STEM_FILTER","FI_FI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1757055","274","1757055","0","1","1757055","431848","11988389","1806392","18825444","1543587637760","1806392","1543589444152","0.000117","18825444","30813833","61.094133","0.869052506162","0.389058673746","0.999998829746","0.999986634125","0.694528751746","0.697056446146","0.537492108587","0.437372459518","0.367513988637","0.581474346349","0.581469391800","0.000013365875","0.537486398327","0.989268269625","0.885293761089","0.934397488899","0.934397488899"
|
||||
"FINNISH_RADIXOR","FI_FI","ALL_WORDS","PRIMARY_OUTPUT","57027","1811717","292","1811717","0","1","1811717","69091","30552427","731279","971268","1641126083212","731279","1641126814491","0.000045","971268","31523695","3.081073","0.976624284859","0.969189271753","0.999999554404","0.999998962594","0.984594413078","0.975128170336","0.972892573600","0.970667204156","0.947215985975","0.972899675927","0.972899157490","0.000001037406","0.972892054895","0.996084757586","0.993746341306","0.994914175412","0.994914175412"
|
||||
"FINNISH_RADIXOR","FI_FI","ALL_WORDS","ANY_CANDIDATE","57027","1811717","292","1754389","57328","6","1876272","69769","31523695","0","0","1641126814491","0","1641126814491","0.000000","0","31523695","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","ALL_WORDS","ALL_CANDIDATES","57027","1811717","292","1754389","57328","6","1876272","69769","31523695","1683575","0","1641125130916","1683575","1641126814491","0.000103","0","31523695","0.000000","0.949301011495","1.000000000000","0.999998974135","0.999998974154","0.999999487067","0.959025334376","0.973991195713","0.989431554710","0.949301011495","0.974320794962","0.974320295201","0.000001025846","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1757055","274","1757055","0","1","1757055","54633","30078528","730145","735305","1543588714007","730145","1543589444152","0.000047","735305","30813833","2.386282","0.976300667023","0.976137178390","0.999999526982","0.999999050641","0.988068352686","0.976267964916","0.976218915862","0.976169871736","0.953542638154","0.976218919284","0.976218444595","0.000000949359","0.976218441173","0.996000407428","0.996068984852","0.996034694959","0.996034694959"
|
||||
"FINNISH_RADIXOR","FI_FI","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","54762","1757055","274","1712724","44331","6","1805864","54984","30813833","0","0","1543589444152","0","1543589444152","0.000000","0","30813833","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","54762","1757055","274","1712724","44331","6","1805864","54984","30813833","1653320","0","1543587790832","1653320","1543589444152","0.000107","0","30813833","0.000000","0.949077148834","1.000000000000","0.999998928912","0.999998928933","0.999999464456","0.958842548108","0.973873352732","0.989382907677","0.949077148834","0.974205906795","0.974205385065","0.000001071067","","","","",""
|
||||
"FRENCH_LUCENE_FRENCH_LIGHT_STEM_FILTER","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","245918","202782","276403","5251833","90395828427","276403","90396104830","0.000306","5251833","5454615","96.282377","0.423181026117","0.037176226003","0.999996942313","0.999938848002","0.518586584158","0.137547303040","0.068348107452","0.045471618191","0.035383242558","0.125428359900","0.125414592230","0.000061151998","0.068339028277","0.974109647704","0.812375827422","0.885921707253","0.885921707253"
|
||||
"FRENCH_LUCENE_FRENCH_LIGHT_STEM_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","245182","200690","262689","5239869","88711863817","262689","88712126506","0.000296","5239869","5440559","96.311225","0.433101197939","0.036887753630","0.999997038860","0.999937976681","0.518442396245","0.137570562409","0.067985131280","0.045148356975","0.035188720533","0.126396717862","0.126383026150","0.000062023319","0.067976159358","0.975085555241","0.811143708698","0.885591261484","0.885591261484"
|
||||
"FRENCH_LUCENE_FRENCH_MINIMAL_STEM_FILTER","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","269236","183612","160438","5271003","90395944392","160438","90396104830","0.000177","5271003","5454615","96.633823","0.533678244441","0.033661770812","0.999998225167","0.999939918724","0.516829997990","0.134399775137","0.063329059361","0.041424008382","0.032699958487","0.134031916915","0.134021061615","0.000060081276","0.063322352769","0.984019125555","0.810978546011","0.889158144694","0.889158144694"
|
||||
"FRENCH_LUCENE_FRENCH_MINIMAL_STEM_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","268411","181686","147476","5258873","88711979030","147476","88712126506","0.000166","5258873","5440559","96.660527","0.551965293685","0.033394730211","0.999998337589","0.999939061122","0.516696533900","0.134438681544","0.062979128454","0.041121435592","0.032513396928","0.135767198057","0.135756528540","0.000060938878","0.062972571968","0.985086367216","0.809773733549","0.888868235590","0.888868235590"
|
||||
"FRENCH_RADIXOR","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","60225","4985455","318767","469160","90395786063","318767","90396104830","0.000353","469160","5454615","8.601157","0.939903156391","0.913988429981","0.999996473664","0.999991284144","0.956992451823","0.934603310507","0.926764667966","0.919056419273","0.863524187383","0.926855226151","0.926850879168","0.000008715856","0.926760310630","0.988772235003","0.985214034569","0.986989927876","0.986989927876"
|
||||
"FRENCH_RADIXOR","FR_FR","ALL_WORDS","ANY_CANDIDATE","59240","425210","2301","382170","43040","56","477024","60383","5454383","12","232","90396104818","12","90396104830","0.000000","232","5454615","0.004253","0.999997799939","0.999957467209","0.999999999867","0.999999997301","0.999978733538","0.999989733133","0.999977633167","0.999965533495","0.999955267335","0.999977633371","0.999977632021","0.000000002699","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","ALL_WORDS","ALL_CANDIDATES","59240","425210","2301","382170","43040","56","477024","60383","5454383","1056255","232","90395048575","1056255","90396104830","0.001168","232","5454615","0.004253","0.837764747479","0.999957467209","0.999988315260","0.999988313399","0.999972891234","0.865852951156","0.911703747510","0.962682080453","0.837734895644","0.915275431226","0.915270082203","0.000011686601","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","58069","4975123","315266","465436","88711811240","315266","88712126506","0.000355","465436","5440559","8.554930","0.940407784758","0.914450702584","0.999996446190","0.999991200142","0.957223574387","0.935099145312","0.927247620620","0.919526848134","0.864363145162","0.927338427699","0.927334038919","0.000008799858","0.927243221287","0.988915897225","0.985549842615","0.987230000708","0.987230000708"
|
||||
"FRENCH_RADIXOR","FR_FR","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","57698","421231","2133","380101","41130","56","468574","58208","5440559","0","0","88712126506","0","88712126506","0.000000","0","5440559","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","57698","421231","2133","380101","41130","56","468574","58208","5440559","938985","0","88711187521","938985","88712126506","0.001058","0","5440559","0.000000","0.852813147774","1.000000000000","0.999989415370","0.999989416019","0.999994707685","0.878679151458","0.920560336911","0.966633773699","0.852813147774","0.923478829088","0.923473941734","0.000010583981","","","","",""
|
||||
"GERMAN_CISTEM","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","59097","1053889","477122","329983","44094768857","477122","44095245979","0.001082","329983","1383872","23.844908","0.688361481400","0.761550923785","0.999989179741","0.999981696901","0.880770051763","0.701851885397","0.723108954973","0.745693871888","0.566304351331","0.724031989665","0.724022910459","0.000018303099","0.723099826442","0.974048119240","0.975147027686","0.974597263694","0.974597263694"
|
||||
"GERMAN_CISTEM","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","23023","725447","156784","147964","11263599558","156784","11263756342","0.001392","147964","873411","16.940936","0.822286906717","0.830590638313","0.999986080665","0.999972946470","0.915288359489","0.823934343933","0.826417914358","0.828916502414","0.704184159310","0.826428343371","0.826414817348","0.000027053530","0.826404386881","0.985935685912","0.973569821618","0.979713735095","0.979713735095"
|
||||
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","98357","709263","205740","674609","44095040239","205740","44095245979","0.000467","674609","1383872","48.747933","0.775148278202","0.512520666651","0.999995334191","0.999980035912","0.756258000421","0.703092101246","0.617052253820","0.549774428024","0.446186239158","0.630301128270","0.630292039259","0.000019964088","0.617042686770","0.980753120457","0.936533167951","0.958133203614","0.958133203614"
|
||||
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","50335","471565","55477","401846","11263700865","55477","11263756342","0.000493","401846","873411","46.008809","0.894738939212","0.539911908597","0.999995074734","0.999959401861","0.769953491666","0.790797426464","0.673446377708","0.586423560557","0.507666155661","0.695039717114","0.695022690564","0.000040598139","0.673427319227","0.991320177896","0.915069562866","0.951669957566","0.951669957566"
|
||||
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","140505","271626","110840","1112246","44095135139","110840","44095245979","0.000251","1112246","1383872","80.372029","0.710196461908","0.196279713731","0.999997486350","0.999972263504","0.598138600041","0.466112921692","0.307558349534","0.229493166050","0.181724639931","0.373359288402","0.373350267608","0.000027736496","0.307548938689","0.983615403456","0.896263607272","0.937910029995","0.937910029995"
|
||||
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","80363","132221","21214","741190","11263735128","21214","11263756342","0.000188","741190","873411","84.861537","0.861739498811","0.151384628772","0.999998116614","0.999932318770","0.575691372693","0.444544636019","0.257528392768","0.181269722976","0.147794886125","0.361184321539","0.361168285320","0.000067681230","0.257511188319","0.992642524078","0.854402700840","0.918349417869","0.918349417869"
|
||||
"GERMAN_LUCENE_GERMAN_STEM_FILTER","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","81085","619354","331871","764518","44094914108","331871","44095245979","0.000753","764518","1383872","55.244849","0.651111987174","0.447551507654","0.999992473769","0.999975136671","0.723771990712","0.596821367368","0.530473894660","0.477402037056","0.360982967729","0.539820480819","0.539808754751","0.000024863329","0.530461889452","0.975549631706","0.942889706548","0.958941664321","0.958941664321"
|
||||
"GERMAN_LUCENE_GERMAN_STEM_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","41574","378734","78723","494677","11263677619","78723","11263756342","0.000699","494677","873411","56.637368","0.827911694432","0.433626322545","0.999993010946","0.999949097306","0.716809666745","0.700518896035","0.569153364571","0.479276535831","0.397773842757","0.599169678345","0.599148958371","0.000050902694","0.569130398173","0.988583531594","0.918717729459","0.952371013508","0.952371013508"
|
||||
"GERMAN_RADIXOR","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","68104","1128969","98192","254903","44095147787","98192","44095245979","0.000223","254903","1383872","18.419550","0.919984419322","0.815804496370","0.999997773184","0.999991992699","0.907901134777","0.897072808397","0.864768082211","0.834709150216","0.761754553110","0.866329859738","0.866325955582","0.000008007301","0.864764092865","0.989946248415","0.975085217969","0.982459538105","0.982459538105"
|
||||
"GERMAN_RADIXOR","DE_DE","ALL_WORDS","ANY_CANDIDATE","54092","296974","1474","248400","48574","8","361016","70717","1272705","1375","111167","44095244604","1375","44095245979","0.000003","111167","1383872","8.033041","0.998920789903","0.919669593720","0.999999968818","0.999997447832","0.959834781269","0.981996366774","0.957658377578","0.934497606897","0.918756727140","0.958476435291","0.958475209548","0.000002552168","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","ALL_WORDS","ALL_CANDIDATES","54092","296974","1474","248400","48574","8","361016","70717","1272705","244817","111167","44095001162","244817","44095245979","0.000555","111167","1383872","8.033041","0.838673179038","0.919669593720","0.999994447996","0.999991927184","0.959832020858","0.853710645080","0.877305874349","0.902242446842","0.781429112618","0.878238135035","0.878234164088","0.000008072816","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","17264","814297","47898","59114","11263708444","47898","11263756342","0.000425","59114","873411","6.768177","0.944446441930","0.932318232768","0.999995747600","0.999990500176","0.966156990184","0.941995622128","0.938343149309","0.934718891125","0.883847872972","0.938362743125","0.938357995965","0.000009499824","0.938338399230","0.994062310308","0.990664418294","0.992360455671","0.992360455671"
|
||||
"GERMAN_RADIXOR","DE_DE","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","16007","150098","228","135120","14978","8","167157","18366","873411","0","0","11263756342","0","11263756342","0.000000","0","873411","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","16007","150098","228","135120","14978","8","167157","18366","873411","97544","0","11263658798","97544","11263756342","0.000866","0","873411","0.000000","0.899538083639","1.000000000000","0.999991340012","0.999991340683","0.999995670006","0.917982540684","0.947112449481","0.978151677228","0.899538083639","0.948439815507","0.948435708759","0.000008659317","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","ALL_WORDS","PRIMARY_OUTPUT","2358","58714","0","58714","0","1","58714","2358","688361","25234","19916","1722904030","25234","1722929264","0.001465","19916","708277","2.811894","0.964638205144","0.971881057835","0.999985354013","0.999973805398","0.985933205924","0.966078126522","0.968246086849","0.970423799230","0.938446730860","0.968252859146","0.968239762122","0.000026194602","0.968232984327","0.993166390361","0.993627509527","0.993396896433","0.993396896433"
|
||||
"HE_IL_RADIXOR","HE_IL","ALL_WORDS","ANY_CANDIDATE","2358","58714","0","56674","2040","40","62376","2358","708277","0","0","1722929264","0","1722929264","0.000000","0","708277","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","ALL_WORDS","ALL_CANDIDATES","2358","58714","0","56674","2040","40","62376","2358","708277","86628","0","1722842636","86628","1722929264","0.005028","0","708277","0.000000","0.891020939609","1.000000000000","0.999949720513","0.999949741174","0.999974860256","0.910874182109","0.942370251906","0.976122467036","0.891020939609","0.943939055029","0.943915324345","0.000050258826","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","2358","58714","0","58714","0","1","58714","2358","688361","25234","19916","1722904030","25234","1722929264","0.001465","19916","708277","2.811894","0.964638205144","0.971881057835","0.999985354013","0.999973805398","0.985933205924","0.966078126522","0.968246086849","0.970423799230","0.938446730860","0.968252859146","0.968239762122","0.000026194602","0.968232984327","0.993166390361","0.993627509527","0.993396896433","0.993396896433"
|
||||
"HE_IL_RADIXOR","HE_IL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","2358","58714","0","56674","2040","40","62376","2358","708277","0","0","1722929264","0","1722929264","0.000000","0","708277","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","2358","58714","0","56674","2040","40","62376","2358","708277","86628","0","1722842636","86628","1722929264","0.005028","0","708277","0.000000","0.891020939609","1.000000000000","0.999949720513","0.999949741174","0.999974860256","0.910874182109","0.942370251906","0.976122467036","0.891020939609","0.943939055029","0.943915324345","0.000050258826","","","","",""
|
||||
"HUNGARIAN_LUCENE_HUNGARIAN_LIGHT_STEM_FILTER","HU_HU","ALL_WORDS","PRIMARY_OUTPUT","19406","916344","1","916344","0","1","916344","94328","14036270","4132555","8125833","419816410338","4132555","419820542893","0.000984","8125833","22162103","36.665442","0.772546931351","0.633345580968","0.999990156377","0.999970802427","0.816667868673","0.740017627855","0.696054898613","0.657022705053","0.533806904809","0.699492090778","0.699477892426","0.000029197573","0.696040442258","0.982615378770","0.926771756762","0.953876941828","0.953876941828"
|
||||
"HUNGARIAN_LUCENE_HUNGARIAN_LIGHT_STEM_FILTER","HU_HU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","878513","1","878513","0","1","878513","91516","13492703","3639046","7918708","385867055871","3639046","385870694917","0.000943","7918708","21411411","36.983588","0.787584676848","0.630164121365","0.999990569261","0.999970049260","0.815077345313","0.750107959995","0.700134758022","0.656404225003","0.538621031944","0.704491026122","0.704476576404","0.000029950740","0.700119966552","0.983686881315","0.925487153321","0.953699929950","0.953699929950"
|
||||
"HUNGARIAN_RADIXOR","HU_HU","ALL_WORDS","PRIMARY_OUTPUT","19406","916344","1","916344","0","1","916344","20535","21962266","272900","199837","419820269993","272900","419820542893","0.000065","199837","22162103","0.901706","0.987726648859","0.990982940563","0.999999349960","0.999998874014","0.995491145262","0.988376194087","0.989352115329","0.990329965723","0.978928596533","0.989353455019","0.989352892139","0.000001125986","0.989351552308","0.998036093538","0.997808712909","0.997922390271","0.997922390271"
|
||||
"HUNGARIAN_RADIXOR","HU_HU","ALL_WORDS","ANY_CANDIDATE","19406","916344","1","904024","12320","5","929326","20567","22162103","0","0","419820542893","0","419820542893","0.000000","0","22162103","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","ALL_WORDS","ALL_CANDIDATES","19406","916344","1","904024","12320","5","929326","20567","22162103","460158","0","419820082735","460158","419820542893","0.000110","0","22162103","0.000000","0.979659062372","1.000000000000","0.999998903917","0.999998903975","0.999999451959","0.983660778882","0.989725029923","0.995864516790","0.979659062372","0.989777279176","0.989776736737","0.000001096025","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","878513","1","878513","0","1","878513","18363","21247134","272775","164277","385870422142","272775","385870694917","0.000071","164277","21411411","0.767240","0.987324528185","0.992327595785","0.999999293092","0.999998867424","0.996163444439","0.988321101757","0.989819739994","0.991322930046","0.979844666523","0.989822900984","0.989822335019","0.000001132576","0.989819173678","0.997945135090","0.998273386381","0.998109233747","0.998109233747"
|
||||
"HUNGARIAN_RADIXOR","HU_HU","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","18360","878513","1","867360","11153","5","890245","18375","21411411","0","0","385870694917","0","385870694917","0.000000","0","21411411","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","18360","878513","1","867360","11153","5","890245","18375","21411411","458462","0","385870236455","458462","385870694917","0.000119","0","21411411","0.000000","0.979036823854","1.000000000000","0.999998811877","0.999998811943","0.999999405938","0.983158850285","0.989407384494","0.995735852714","0.979036823854","0.989462896653","0.989462308851","0.000001188057","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","ALL_WORDS","PRIMARY_OUTPUT","5113","51676","2","51676","0","1","51676","10920","213552","11408","88283","1334865407","11408","1334876815","0.000855","88283","301835","29.248762","0.949288762447","0.707512382593","0.999991453893","0.999925335085","0.853751918243","0.888559718726","0.810759403563","0.745486280807","0.681745481942","0.819532521678","0.819499025505","0.000074664915","0.810722859062","0.995776551361","0.952852006662","0.973841506371","0.973841506371"
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","ALL_WORDS","ANY_CANDIDATE","5113","51676","2","48359","3317","5","55596","11359","224312","10102","77523","1334866713","10102","1334876815","0.000757","77523","301835","25.683900","0.956905304291","0.743160998559","0.999992432261","0.999934372078","0.871576715410","0.904855299474","0.836596431881","0.777913569166","0.719093919606","0.843288029954","0.843258147533","0.000065627922","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","ALL_WORDS","ALL_CANDIDATES","5113","51676","2","48359","3317","5","55596","11359","224312","13917","77523","1334862898","13917","1334876815","0.001043","77523","301835","25.683900","0.941581419558","0.743160998559","0.999989574319","0.999931514783","0.871575286439","0.893850652440","0.830686733424","0.775860578084","0.710405634802","0.836508570179","0.836476906392","0.000068485217","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50968","2","50968","0","1","50968","10816","210827","11239","87986","1298532976","11239","1298544215","0.000866","87986","298813","29.445171","0.949388920411","0.705548286052","0.999991344923","0.999923605087","0.852769815487","0.888008949714","0.809504702628","0.743753342581","0.679973036781","0.818437368155","0.818403143837","0.000076394913","0.809467327086","0.995812473772","0.952393750423","0.973619286126","0.973619286126"
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","5038","50968","2","47731","3237","5","54804","11240","221382","10028","77431","1298534187","10028","1298544215","0.000772","77431","298813","25.912862","0.956665658355","0.740871381098","0.999992277506","0.999932663918","0.870431829302","0.904003665310","0.835052421340","0.775874033233","0.716815448726","0.841882537861","0.841851913927","0.000067336082","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","5038","50968","2","47731","3237","5","54804","11240","221382","13601","77431","1298530614","13601","1298544215","0.001047","77431","298813","25.912862","0.942119217135","0.740871381098","0.999989525963","0.999929913009","0.870430453530","0.893573737936","0.829462940899","0.773935751817","0.708617411512","0.835457458856","0.835425115185","0.000070086991","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","ALL_WORDS","PRIMARY_OUTPUT","4992","26477","85","26477","0","1","26477","15909","18482","1333","46084","350436627","1333","350437960","0.000380","46084","64566","71.375027","0.932727731517","0.286249728960","0.999996196188","0.999864717095","0.643122962574","0.642512480358","0.438060700869","0.332315636923","0.280459491039","0.516713712165","0.516673857221","0.000135282905","0.438012080403","0.996931617211","0.889026094124","0.939891935467","0.939891935467"
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","ALL_WORDS","ANY_CANDIDATE","4992","26477","85","25223","1254","3","27763","16027","21374","1164","43192","350436796","1164","350437960","0.000332","43192","64566","66.895889","0.948353891206","0.331041105226","0.999996678442","0.999873450270","0.665518891834","0.690740573172","0.490769654666","0.380588457347","0.325178761600","0.560307166017","0.560267948638","0.000126549730","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","ALL_WORDS","ALL_CANDIDATES","4992","26477","85","25223","1254","3","27763","16027","21374","1738","43192","350436222","1738","350437960","0.000496","43192","64566","66.895889","0.924800969193","0.331041105226","0.999995040492","0.999871812621","0.665518072859","0.680639942935","0.487556741714","0.379812066416","0.322363658301","0.553305643343","0.553264631551","0.000128187379","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25678","84","25678","0","1","25678","15258","18333","1310","44814","329602546","1310","329603856","0.000397","44814","63147","70.967742","0.933309575930","0.290322580645","0.999996025532","0.999860089122","0.645159303089","0.646808120294","0.442879574828","0.336717714000","0.284422172921","0.520538994337","0.520497519470","0.000139910878","0.442828931093","0.996884174988","0.889060613994","0.939890140871","0.939890140871"
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4796","25678","84","24492","1186","3","26896","15323","21212","1141","41935","329602715","1141","329603856","0.000346","41935","63147","66.408539","0.948955397486","0.335914611937","0.999996538269","0.999869334815","0.667955575103","0.695206444720","0.496187134503","0.385755489360","0.329952712792","0.564595416287","0.564554662442","0.000130665185","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4796","25678","84","24492","1186","3","26896","15323","21212","1712","41935","329602144","1712","329603856","0.000519","41935","63147","66.408539","0.925318443553","0.335914611937","0.999994805886","0.999867602764","0.667954708912","0.684951854459","0.492895400309","0.384956009176","0.327047903915","0.557519493726","0.557476858392","0.000132397236","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","ALL_WORDS","PRIMARY_OUTPUT","396939","607439","250964","607439","0","1","607439","557518","46002","1981986","267868","184488469785","1981986","184490451771","0.001074","267868","313870","85.343614","0.022683566175","0.146563864020","0.999989256972","0.999987805059","0.573276560496","0.027298226808","0.039286754363","0.070050933952","0.020036970960","0.057659267324","0.057655308782","0.000012194941","0.039283923590","0.993096189204","0.963677172906","0.978165531652","0.978165531652"
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","ALL_WORDS","ANY_CANDIDATE","396939","607439","250964","600602","6837","4","614296","557638","51229","1978852","262641","184488472919","1978852","184490451771","0.001073","262641","313870","83.678274","0.025234953679","0.163217255552","0.999989273960","0.999987850378","0.581603264756","0.030369825498","0.043711664621","0.077960810954","0.022344183028","0.064177721084","0.064173802046","0.000012149622","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","ALL_WORDS","ALL_CANDIDATES","396939","607439","250964","600602","6837","4","614296","557638","51229","2008917","262641","184488442854","2008917","184490451771","0.001089","262641","313870","83.678274","0.024866684206","0.163217255552","0.999989110997","0.999987687416","0.581603183275","0.029942881217","0.043158091605","0.077253888104","0.022054971033","0.063707707153","0.063703758400","0.000012312584","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","583910","228735","583910","0","1","583910","535362","45926","1978041","265965","170472862163","1978041","170474840204","0.001160","265965","311891","85.274984","0.022691081426","0.147250161114","0.999988396874","0.999986836756","0.573619278994","0.027311677226","0.039322595808","0.070190378755","0.020055617372","0.057803679777","0.057799415161","0.000013163244","0.039319549964","0.993066314983","0.962316521867","0.977449637185","0.977449637185"
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","374384","583910","228735","577124","6786","4","590716","535485","51150","1974950","260741","170472865254","1974950","170474840204","0.001158","260741","311891","83.600040","0.025245545630","0.163999602425","0.999988415006","0.999986885532","0.581994008716","0.030387494919","0.043755514884","0.078123472659","0.022367099418","0.064344847861","0.064340626006","0.000013114468","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","374384","583910","228735","577124","6786","4","590716","535485","51150","2004598","260741","170472835606","2004598","170474840204","0.001176","260741","311891","83.600040","0.024881454342","0.163999602425","0.999988241092","0.999986711618","0.581993921758","0.029965261387","0.043207600483","0.077422296168","0.022080830084","0.063879172034","0.063874918547","0.000013288382","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","154336","3422734","776728","2031881","90395328102","776728","90396104830","0.000859","2031881","5454615","37.250677","0.815041069547","0.627493232795","0.999991407506","0.999968931852","0.813742320150","0.769068574566","0.709075347131","0.657764674673","0.549277098051","0.715145268872","0.715130511338","0.000031068148","0.709060074832","0.978337247291","0.913705954219","0.944917713687","0.944917713687"
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","ALL_WORDS","ANY_CANDIDATE","59240","425210","2301","411699","13511","4","439015","154718","3610612","745831","1844003","90395358999","745831","90396104830","0.000825","1844003","5454615","33.806291","0.828798173189","0.661937093635","0.999991749302","0.999971351888","0.830964421468","0.789018996925","0.736029080656","0.689708764155","0.582314885091","0.740683639600","0.740669908387","0.000028648112","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","ALL_WORDS","ALL_CANDIDATES","59240","425210","2301","411699","13511","4","439015","154718","3610612","1043199","1844003","90395061631","1043199","90396104830","0.001154","1844003","5454615","33.806291","0.775839843947","0.661937093635","0.999988459691","0.999968062476","0.830962776663","0.750027659074","0.714376699201","0.681961135862","0.555665643861","0.716629033342","0.716613365354","0.000031937524","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","153822","3412548","763305","2028011","88711363201","763305","88712126506","0.000860","2028011","5440559","37.275784","0.817209801207","0.627242163903","0.999991395708","0.999968537054","0.813616779806","0.770536594362","0.709734150326","0.657825640123","0.550068151075","0.715952822518","0.715937898033","0.000031462946","0.709718690125","0.979328164393","0.913161860024","0.945088340370","0.945088340370"
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","57698","421231","2133","407794","13437","4","434961","154205","3600083","733584","1840476","88711392922","733584","88712126506","0.000827","1840476","5440559","33.828803","0.830724418835","0.661711967465","0.999991730736","0.999970985904","0.830851849101","0.790350629656","0.736648201095","0.689779349655","0.583090317150","0.741417756470","0.741403865778","0.000029014096","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","57698","421231","2133","407794","13437","4","434961","154205","3600083","1027635","1840476","88711098871","1027635","88712126506","0.001158","1840476","5440559","33.828803","0.777939148410","0.661711967465","0.999988416071","0.999967671442","0.830850191768","0.751538185756","0.715133880405","0.682093458746","0.556582409247","0.717475884237","0.717460037184","0.000032328558","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","182774","391862","203883","992010","44095042096","203883","44095245979","0.000462","992010","1383872","71.683653","0.657768004767","0.283163471766","0.999995376304","0.999972880172","0.641579424035","0.520145203475","0.395896782054","0.319562150060","0.246802560848","0.431573715426","0.431562811676","0.000027119828","0.395885371175","0.980462638581","0.886872825924","0.931322397630","0.931322397630"
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","ALL_WORDS","ANY_CANDIDATE","54092","296974","1474","289083","7891","3","305052","183111","408175","158403","975697","44095087576","158403","44095245979","0.000359","975697","1383872","70.504859","0.720421548313","0.294951411691","0.999996407708","0.999974281481","0.647473909700","0.559115650060","0.418544438463","0.334456395588","0.264657729653","0.460965674088","0.460955788036","0.000025718519","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","ALL_WORDS","ALL_CANDIDATES","54092","296974","1474","289083","7891","3","305052","183111","408175","242551","975697","44095003428","242551","44095245979","0.000550","975697","1383872","70.504859","0.627260936247","0.294951411691","0.999994499384","0.999972373218","0.647472955538","0.511911128190","0.401234052132","0.329906951166","0.250964847398","0.430129630047","0.430118032816","0.000027626782","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","86983","278093","84679","595318","11263671663","84679","11263756342","0.000752","595318","873411","68.160122","0.766577905682","0.318398783620","0.999992482170","0.999939634323","0.659195632895","0.598178360154","0.449922058465","0.360558871242","0.290257700216","0.494041974653","0.494019111671","0.000060365677","0.449897024669","0.988041339480","0.865580709092","0.922765807515","0.922765807515"
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","16007","150098","228","145109","4989","3","155207","87393","288864","60996","584547","11263695346","60996","11263756342","0.000542","584547","873411","66.926911","0.825655976676","0.330730893016","0.999994584755","0.999942692923","0.665362738886","0.635466205220","0.472281285177","0.375782098835","0.309141519702","0.522560942370","0.522540242219","0.000057307077","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","16007","150098","228","145109","4989","3","155207","87393","288864","96545","584547","11263659797","96545","11263756342","0.000857","584547","873411","66.926911","0.749499881944","0.330730893016","0.999991428703","0.999939537116","0.665361160860","0.598050472724","0.458944090497","0.372338300095","0.297811447117","0.497878263505","0.497854575726","0.000060462884","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","ALL_WORDS","PRIMARY_OUTPUT","9990","122341","1","122341","0","1","122341","18419","971262","52652","149705","7482425351","52652","7482478003","0.000704","149705","1120967","13.354987","0.948577712581","0.866450127435","0.999992963294","0.999972959935","0.933221545364","0.930929837176","0.905655838249","0.881717903868","0.827578626454","0.906584403102","0.906571161039","0.000027040065","0.905642343969","0.994545991966","0.970520439141","0.982386343372","0.982386343372"
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","ALL_WORDS","ANY_CANDIDATE","9990","122341","1","110894","11447","6","135231","19068","1040224","42213","80743","7482435790","42213","7482478003","0.000564","80743","1120967","7.202977","0.961001887408","0.927970225707","0.999994358420","0.999983569937","0.963982292063","0.954208759768","0.944197251162","0.934393641743","0.894293230626","0.944341642819","0.944333470354","0.000016430063","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","ALL_WORDS","ALL_CANDIDATES","9990","122341","1","110894","11447","6","135231","19068","1040224","82745","80743","7482395258","82745","7482478003","0.001106","80743","1120967","7.202977","0.926315864463","0.927970225707","0.999988941498","0.999978153827","0.963979583602","0.926646264647","0.927142307089","0.927638880888","0.864180136112","0.927142676087","0.927131751477","0.000021846173","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","120925","1","120925","0","1","120925","18149","965984","51950","148667","7310200749","51950","7310252699","0.000711","148667","1114651","13.337538","0.948965257080","0.866624620621","0.999992893543","0.999972560946","0.933308757082","0.931268723294","0.905927782480","0.881929423296","0.828032892137","0.906860880124","0.906847444801","0.000027439054","0.905914089179","0.994583905165","0.970514203019","0.982401644006","0.982401644006"
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","120925","1","109660","11265","6","133595","18789","1034283","41671","80368","7310211028","41671","7310252699","0.000570","80368","1114651","7.210149","0.961270649117","0.927898508143","0.999994299650","0.999983308321","0.963946403896","0.954405554191","0.944289819479","0.934386268967","0.894459328803","0.944437187555","0.944428885928","0.000016691679","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","120925","1","109660","11265","6","133595","18789","1034283","81865","80368","7310170834","81865","7310252699","0.001120","80368","1114651","7.210149","0.926653992123","0.927898508143","0.999988801345","0.999977810854","0.963943654744","0.926902628188","0.927275832560","0.927649337585","0.864412176686","0.927276041347","0.927264945147","0.000022189146","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","495840","9662476","536192","32310860","379566781918","536192","379567318110","0.000141","32310860","41973336","76.979490","0.947425291224","0.230205099733","0.999998587360","0.999913471422","0.615101843546","0.583708381625","0.370408466579","0.271277635967","0.227301418167","0.467014061518","0.466991649518","0.000086528578","0.370381248953","0.993314263125","0.790558492734","0.880413722434","0.880413722434"
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","ALL_WORDS","ANY_CANDIDATE","65059","871332","3589","853455","17877","5","890999","496361","10079118","416345","31894218","379566901765","416345","379567318110","0.000110","31894218","41973336","75.986855","0.960330954432","0.240131449166","0.999998903106","0.999914884689","0.620065176136","0.600267728541","0.384194728757","0.282504215637","0.237772914592","0.480214185303","0.480192080762","0.000085115311","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","ALL_WORDS","ALL_CANDIDATES","65059","871332","3589","853455","17877","5","890999","496361","10079118","888077","31894218","379566430033","888077","379567318110","0.000234","31894218","41973336","75.986855","0.919024235459","0.240131449166","0.999997660291","0.999913642011","0.620064554728","0.587073016700","0.380771322449","0.281759130783","0.235155989841","0.469772946730","0.469749183448","0.000086357989","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","495045","9628515","531181","32234855","377860138584","531181","377860669765","0.000141","32234855","41863370","77.000144","0.947716841134","0.229998564377","0.999998594241","0.999913295008","0.614998579309","0.583531128169","0.370163304101","0.271052948234","0.227116805648","0.466876335765","0.466853897372","0.000086704992","0.370136051001","0.993362468962","0.790499503024","0.880396073652","0.880396073652"
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","64918","869371","3525","851564","17807","5","888962","495572","10041262","412198","31822108","377860257567","412198","377860669765","0.000109","31822108","41863370","76.014205","0.960568271175","0.239857947413","0.999998909127","0.999914702064","0.619928428270","0.599999808789","0.383863548307","0.282205460900","0.237519268813","0.479999931119","0.479977799265","0.000085297936","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","64918","869371","3525","851564","17807","5","888962","495572","10041262","878949","31822108","377859790816","878949","377860669765","0.000233","31822108","41863370","76.014205","0.919511720057","0.239857947413","0.999997673881","0.999913466955","0.619927810647","0.586904802235","0.380469146267","0.281467012980","0.234925531298","0.469629847641","0.469606065497","0.000086533045","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","ALL_WORDS","PRIMARY_OUTPUT","1493","14245","4","14245","0","1","14245","3137","50416","794","14924","101386756","794","101387550","0.000783","14924","65340","22.840526","0.984495215778","0.771594735231","0.999992168664","0.999845070949","0.885793451947","0.933007624547","0.865139425139","0.806475349522","0.762331024889","0.871568313648","0.871498740996","0.000154929051","0.865063055969","0.998114340300","0.949803904722","0.973360047526","0.973360047526"
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","ALL_WORDS","ANY_CANDIDATE","1493","14245","4","12923","1322","6","15740","3311","55875","326","9465","101387224","326","101387550","0.000322","9465","65340","14.485767","0.994199391470","0.855142332415","0.999996784615","0.999903492153","0.927569558515","0.962883947281","0.919442821764","0.879752236578","0.850896963421","0.922053136488","0.922008115880","0.000096507847","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","ALL_WORDS","ALL_CANDIDATES","1493","14245","4","12923","1322","6","15740","3311","55875","1271","9465","101386279","1271","101387550","0.001254","9465","65340","14.485767","0.977758723270","0.855142332415","0.999987463944","0.999894177485","0.927564898180","0.950500809733","0.912349166435","0.877142031861","0.838825419225","0.914397547654","0.914347195556","0.000105822515","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14236","4","14236","0","1","14236","3134","50404","794","14920","101258612","794","101259406","0.000784","14920","65324","22.839998","0.984491581702","0.771600024493","0.999992158753","0.999844914465","0.885796091623","0.933006560145","0.865141346698","0.806479484406","0.762334008893","0.871569692311","0.871500048785","0.000155085535","0.865064900251","0.998112856419","0.949788155938","0.973351072054","0.973351072054"
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14236","4","12915","1321","6","15730","3308","55859","326","9465","101259080","326","101259406","0.000322","9465","65324","14.489315","0.994197739610","0.855106851999","0.999996780546","0.999903370085","0.927551816273","0.962873710629","0.919421606630","0.879721936116","0.850860624524","0.922033242016","0.921988165267","0.000096629915","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14236","4","12915","1321","6","15730","3308","55859","1271","9465","101258135","1271","101259406","0.001255","9465","65324","14.489315","0.977752494311","0.855106851999","0.999987448080","0.999894043636","0.927547150039","0.950487333415","0.912326261290","0.877111165546","0.838786695698","0.914375665383","0.914325250217","0.000105956364","","","","",""
|
||||
"ITALIAN_LUCENE_ITALIAN_LIGHT_STEM_FILTER","IT_IT","ALL_WORDS","PRIMARY_OUTPUT","10009","327551","0","327551","0","1","327551","244870","109684","10589","6034130","53638510622","10589","53638521211","0.000020","6034130","6143814","98.214725","0.911958627456","0.017852754006","0.999999802586","0.999887319289","0.508926278296","0.082781551919","0.035019947839","0.022207258650","0.017822037328","0.127596916262","0.127588341500","0.000112680711","0.035015703871","0.997481424185","0.737537113266","0.848036553212","0.848036553212"
|
||||
"ITALIAN_LUCENE_ITALIAN_LIGHT_STEM_FILTER","IT_IT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","327469","0","327469","0","1","327469","244808","109658","10588","6032516","53611656484","10588","53611667072","0.000020","6032516","6142174","98.214671","0.911947174958","0.017853287777","0.999999802506","0.999887292971","0.508926545141","0.082783771729","0.035020966336","0.022207918023","0.017822564890","0.127598022525","0.127589445488","0.000112707029","0.035016721203","0.997481197125","0.737534145120","0.848034509070","0.848034509070"
|
||||
"ITALIAN_RADIXOR","IT_IT","ALL_WORDS","PRIMARY_OUTPUT","10009","327551","0","327551","0","1","327551","10010","6100906","124172","42908","53638397039","124172","53638521211","0.000231","42908","6143814","0.698394","0.980052940702","0.993016064614","0.999997685022","0.999996885431","0.996506874818","0.982618418699","0.986491918597","0.990396078172","0.973343909830","0.986513210398","0.986511657877","0.000003114569","0.986490361200","0.995780270704","0.997112550353","0.996445965204","0.996445965204"
|
||||
"ITALIAN_RADIXOR","IT_IT","ALL_WORDS","ANY_CANDIDATE","10009","327551","0","321297","6254","4","334175","10012","6143734","0","80","53638521211","0","53638521211","0.000000","80","6143814","0.001302","1.000000000000","0.999986978772","1.000000000000","0.999999998509","0.999993489386","0.999997395727","0.999993489344","0.999989582991","0.999986978772","0.999993489365","0.999993488619","0.000000001491","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","ALL_WORDS","ALL_CANDIDATES","10009","327551","0","321297","6254","4","334175","10012","6143734","170950","80","53638350261","170950","53638521211","0.000319","80","6143814","0.001302","0.972928178195","0.999986978772","0.999996812925","0.999996811799","0.999991895849","0.978222150749","0.986272020913","0.994455476443","0.972915852437","0.986364795335","0.986363222748","0.000003188201","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","327469","0","327469","0","1","327469","10007","6099346","124171","42828","53611542901","124171","53611667072","0.000232","42828","6142174","0.697278","0.980048098206","0.993027224563","0.999997683881","0.999996885382","0.996512454222","0.982616709845","0.986494972258","0.990403969991","0.973349855458","0.986516316590","0.986514764059","0.000003114618","0.986493414837","0.995779575755","0.997114909855","0.996446795437","0.996446795437"
|
||||
"ITALIAN_RADIXOR","IT_IT","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","10007","327469","0","321217","6252","4","334089","10007","6142174","0","0","53611667072","0","53611667072","0.000000","0","6142174","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","10007","327469","0","321217","6252","4","334089","10007","6142174","170949","0","53611496123","170949","53611667072","0.000319","0","6142174","0.000000","0.972921642743","1.000000000000","0.999996811347","0.999996811712","0.999998405674","0.978219357390","0.986274996092","0.994464412864","0.972921642743","0.986367904356","0.986366331762","0.000003188288","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","ALL_WORDS","PRIMARY_OUTPUT","4992","26477","85","26477","0","1","26477","5015","63102","1214","1464","350436746","1214","350437960","0.000346","1464","64566","2.267447","0.981124448038","0.977325527367","0.999996535763","0.999992359542","0.988661031565","0.980362303079","0.979221303208","0.978082956166","0.959288537549","0.979223145453","0.979219325206","0.000007640458","0.979217482290","0.997464133435","0.997003025118","0.997233525974","0.997233525974"
|
||||
"NL_NL_RADIXOR","NL_NL","ALL_WORDS","ANY_CANDIDATE","4992","26477","85","25905","572","3","27061","5016","64566","0","0","350437960","0","350437960","0.000000","0","64566","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","ALL_WORDS","ALL_CANDIDATES","4992","26477","85","25905","572","3","27061","5016","64566","2651","0","350435309","2651","350437960","0.000756","0","64566","0.000000","0.960560572474","1.000000000000","0.999992435180","0.999992436574","0.999996217590","0.968197604323","0.979883596519","0.991855131329","0.960560572474","0.980081921308","0.980078214229","0.000007563426","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25678","84","25678","0","1","25678","4797","61763","1214","1384","329602642","1214","329603856","0.000368","1384","63147","2.191711","0.980723121139","0.978082885964","0.999996316791","0.999992119320","0.989039601378","0.980193934392","0.979401224192","0.978609795129","0.959633939808","0.979402113872","0.979398173122","0.000007880680","0.979397283106","0.997373193672","0.997139403762","0.997256285015","0.997256285015"
|
||||
"NL_NL_RADIXOR","NL_NL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4796","25678","84","25129","549","3","26239","4797","63147","0","0","329603856","0","329603856","0.000000","0","63147","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4796","25678","84","25129","549","3","26239","4797","63147","2651","0","329601205","2651","329603856","0.000804","0","63147","0.000000","0.959710021581","1.000000000000","0.999991957012","0.999991958552","0.999995978506","0.967506182222","0.979440846873","0.991673628866","0.959710021581","0.979647906945","0.979643967288","0.000008041448","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","ALL_WORDS","PRIMARY_OUTPUT","4688","18250","23","18250","0","1","18250","4680","26716","6230","3936","166485243","6230","166491473","0.003742","3936","30652","12.840924","0.810902689249","0.871590760799","0.999962580666","0.999938951055","0.935776670732","0.822354650447","0.840152206044","0.858737158800","0.724364188493","0.840699287413","0.840668985911","0.000061048945","0.840121715471","0.983845117159","0.986801676045","0.985321178741","0.985321178741"
|
||||
"NN_NO_RADIXOR","NN_NO","ALL_WORDS","ANY_CANDIDATE","4688","18250","23","15846","2404","5","21513","4693","30652","0","0","166491473","0","166491473","0.000000","0","30652","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","ALL_WORDS","ALL_CANDIDATES","4688","18250","23","15846","2404","5","21513","4693","30652","13214","0","166478259","13214","166491473","0.007937","0","30652","0.000000","0.698764418912","1.000000000000","0.999920632572","0.999920647181","0.999960316286","0.743561877778","0.822673716418","0.920624241623","0.698764418912","0.835921299473","0.835888126353","0.000079352819","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","18219","23","18219","0","1","18219","4668","26671","6230","3924","165920046","6230","165926276","0.003755","3924","30595","12.825625","0.810644053372","0.871743748979","0.999962453204","0.999938815429","0.935853101091","0.822169063928","0.840084414766","0.858797921188","0.724263408011","0.840638974932","0.840608609146","0.000061184571","0.840053856992","0.983814668550","0.986841730061","0.985325874420","0.985325874420"
|
||||
"NN_NO_RADIXOR","NN_NO","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4681","18219","23","15820","2399","5","21477","4681","30595","0","0","165926276","0","165926276","0.000000","0","30595","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4681","18219","23","15820","2399","5","21477","4681","30595","13214","0","165913062","13214","165926276","0.007964","0","30595","0.000000","0.698372480541","1.000000000000","0.999920362222","0.999920376903","0.999960181111","0.743206805583","0.822402021397","0.920488118949","0.698372480541","0.835686831618","0.835653554835","0.000079623097","","","","",""
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_LIGHT_STEM_FILTER","NB_NO","ALL_WORDS","PRIMARY_OUTPUT","17929","75310","252","75310","0","1","75310","25999","99529","25171","42651","2835593044","25171","2835618215","0.000888","42651","142180","29.997890","0.798147554130","0.700021100014","0.999991123276","0.999976083311","0.850006111645","0.776381478361","0.745870803357","0.717667503101","0.594732030284","0.747475838282","0.747464055956","0.000023916689","0.745858895755","0.987774378220","0.965622291196","0.976572729149","0.976572729149"
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_LIGHT_STEM_FILTER","NB_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","75251","252","75251","0","1","75251","25985","99450","25118","42641","2831151666","25118","2831176784","0.000887","42641","142091","30.009642","0.798359129150","0.699903582915","0.999991128071","0.999976068044","0.849947355493","0.776512696705","0.745896444523","0.717602881668","0.594764635875","0.747512150366","0.747500361706","0.000023931956","0.745884529658","0.987789184092","0.965602788874","0.976569991255","0.976569991255"
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_MINIMAL_STEM_FILTER","NB_NO","ALL_WORDS","PRIMARY_OUTPUT","17929","75310","252","75310","0","1","75310","27457","94526","14772","47654","2835603443","14772","2835618215","0.000521","47654","142180","33.516669","0.864846566268","0.664833309889","0.999994790554","0.999977986151","0.832414050221","0.815762584315","0.751763573752","0.697075888841","0.602260563739","0.758273568838","0.758263230800","0.000022013849","0.751752754540","0.992088894987","0.962515968891","0.977078714611","0.977078714611"
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_MINIMAL_STEM_FILTER","NB_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","75251","252","75251","0","1","75251","27443","94447","14719","47644","2831162065","14719","2831176784","0.000520","47644","142091","33.530625","0.865168642251","0.664693752595","0.999994801102","0.999977973869","0.832344276848","0.815949754214","0.751795969864","0.696994967013","0.602302149098","0.758335144541","0.758324803793","0.000022026131","0.751785145440","0.992107474043","0.962494026490","0.977076419047","0.977076419047"
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","ALL_WORDS","PRIMARY_OUTPUT","17929","75310","252","75310","0","1","75310","17886","135010","11482","7170","2835606733","11482","2835618215","0.000405","7170","142180","5.042903","0.921620293258","0.949570966381","0.999995950795","0.999993422575","0.974783458588","0.927078011613","0.935386875069","0.943846020481","0.878616704195","0.935491246621","0.935487968734","0.000006577425","0.935383586923","0.993354053349","0.994615320153","0.993984286646","0.993984286646"
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","ALL_WORDS","ANY_CANDIDATE","17929","75310","252","71073","4237","9","79825","17962","142180","0","0","2835618215","0","2835618215","0.000000","0","142180","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","ALL_WORDS","ALL_CANDIDATES","17929","75310","252","71073","4237","9","79825","17962","142180","20161","0","2835598054","20161","2835618215","0.000711","0","142180","0.000000","0.875810793330","1.000000000000","0.999992890087","0.999992890443","0.999996445043","0.898118108406","0.933794385280","0.972422273928","0.875810793330","0.935847633608","0.935844306704","0.000007109557","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","75251","252","75251","0","1","75251","17838","134987","11482","7104","2831165302","11482","2831176784","0.000406","7104","142091","4.999613","0.921607985307","0.950003870759","0.999995944443","0.999993435568","0.974999907601","0.927150543912","0.935590518436","0.944185565020","0.878976122105","0.935698217036","0.935694946007","0.000006564432","0.935587236809","0.993348241255","0.994693619298","0.994020475044","0.994020475044"
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","17914","75251","252","71047","4204","9","79733","17914","142091","0","0","2831176784","0","2831176784","0.000000","0","142091","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","17914","75251","252","71047","4204","9","79733","17914","142091","20161","0","2831156623","20161","2831176784","0.000712","0","142091","0.000000","0.875742671893","1.000000000000","0.999992878933","0.999992879290","0.999996439466","0.898060798964","0.933755663840","0.972405477022","0.875742671893","0.935811237319","0.935807905326","0.000007120710","","","","",""
|
||||
"PERSIAN_LUCENE_PERSIAN_STEM_FILTER","FA_IR","ALL_WORDS","PRIMARY_OUTPUT","69","3701","0","3701","0","1","3701","3190","430","179","98018","6748223","179","6748402","0.002652","98018","98448","99.563221","0.706075533662","0.004367788071","0.999973475202","0.985658076342","0.502170631636","0.021311605408","0.008681870034","0.005451304637","0.004359860890","0.055533668104","0.054800599292","0.014341923658","0.008506575635","0.985685778881","0.520346904570","0.681125382676","0.681125382676"
|
||||
"PERSIAN_LUCENE_PERSIAN_STEM_FILTER","FA_IR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","69","3701","0","3701","0","1","3701","3190","430","179","98018","6748223","179","6748402","0.002652","98018","98448","99.563221","0.706075533662","0.004367788071","0.999973475202","0.985658076342","0.502170631636","0.021311605408","0.008681870034","0.005451304637","0.004359860890","0.055533668104","0.054800599292","0.014341923658","0.008506575635","0.985685778881","0.520346904570","0.681125382676","0.681125382676"
|
||||
"PERSIAN_RADIXOR","FA_IR","ALL_WORDS","PRIMARY_OUTPUT","69","3701","0","3701","0","1","3701","69","93636","8621","4812","6739781","8621","6748402","0.127749","4812","98448","4.887860","0.915692813206","0.951121404193","0.998722512381","0.998038075904","0.974921958287","0.922565796215","0.933070924989","0.943818050233","0.874538848780","0.933239001706","0.932248664283","0.001961924096","0.932075735269","0.980342969277","0.984586447427","0.982460126226","0.982460126226"
|
||||
"PERSIAN_RADIXOR","FA_IR","ALL_WORDS","ANY_CANDIDATE","69","3701","0","3387","314","2","4015","69","98448","0","0","6748402","0","6748402","0.000000","0","98448","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","ALL_WORDS","ALL_CANDIDATES","69","3701","0","3387","314","2","4015","69","98448","13433","0","6734969","13433","6748402","0.199055","0","98448","0.000000","0.879934930864","1.000000000000","0.998009454683","0.998038075904","0.999004727341","0.901584696651","0.936133391021","0.973435401930","0.879934930864","0.938048469358","0.937114390300","0.001961924096","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","69","3701","0","3701","0","1","3701","69","93636","8621","4812","6739781","8621","6748402","0.127749","4812","98448","4.887860","0.915692813206","0.951121404193","0.998722512381","0.998038075904","0.974921958287","0.922565796215","0.933070924989","0.943818050233","0.874538848780","0.933239001706","0.932248664283","0.001961924096","0.932075735269","0.980342969277","0.984586447427","0.982460126226","0.982460126226"
|
||||
"PERSIAN_RADIXOR","FA_IR","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","69","3701","0","3387","314","2","4015","69","98448","0","0","6748402","0","6748402","0.000000","0","98448","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","69","3701","0","3387","314","2","4015","69","98448","13433","0","6734969","13433","6748402","0.199055","0","98448","0.000000","0.879934930864","1.000000000000","0.998009454683","0.998038075904","0.999004727341","0.901584696651","0.936133391021","0.973435401930","0.879934930864","0.938048469358","0.937114390300","0.001961924096","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","ALL_WORDS","PRIMARY_OUTPUT","9990","122341","1","122341","0","1","122341","15519","1004747","99228","116220","7482378775","99228","7482478003","0.001326","116220","1120967","10.367834","0.910117529835","0.896321657997","0.999986738618","0.999971210643","0.948154198307","0.907324485129","0.903166914014","0.899047271013","0.823431500703","0.903193253581","0.903178865015","0.000028789357","0.903152518035","0.990022261217","0.977053921984","0.983495343428","0.983495343428"
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","ALL_WORDS","ANY_CANDIDATE","9990","122341","1","109468","12873","5","136636","16295","1093112","85532","27855","7482392471","85532","7482478003","0.001143","27855","1120967","2.484908","0.927431862377","0.975150918805","0.999988569028","0.999984848600","0.987569743916","0.936598359399","0.950692965028","0.965218263555","0.906019814355","0.950992130738","0.950984648194","0.000015151400","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","ALL_WORDS","ALL_CANDIDATES","9990","122341","1","109468","12873","5","136636","16295","1093112","143096","27855","7482334907","143096","7482478003","0.001912","27855","1120967","2.484908","0.884246016852","0.975150918805","0.999980875854","0.999977156579","0.987565897330","0.901045352805","0.927476322293","0.955504786999","0.864760696263","0.928586730350","0.928575670129","0.000022843421","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","120925","1","120925","0","1","120925","15277","999138","99224","115513","7310153475","99224","7310252699","0.001357","115513","1114651","10.363154","0.909661841906","0.896368459724","0.999986426735","0.999970629707","0.948177443229","0.906971715650","0.902966227492","0.898995962905","0.823097930182","0.902990688822","0.902976009256","0.000029370293","0.902951540918","0.989889334726","0.977011745487","0.983408384376","0.983408384376"
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","120925","1","108162","12763","5","135105","16044","1087157","85532","27494","7310167167","85532","7310252699","0.001170","27494","1114651","2.466602","0.927063356099","0.975333983462","0.999988299720","0.999984541059","0.987661141591","0.936331423447","0.950586270515","0.965281863330","0.905826028197","0.950892420848","0.950884788442","0.000015458941","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","120925","1","108162","12763","5","135105","16044","1087157","143085","27494","7310109614","143085","7310252699","0.001957","27494","1114651","2.466602","0.883693614752","0.975333983462","0.999980426805","0.999976669344","0.987657205134","0.900617649988","0.927255102898","0.955516285728","0.864376148890","0.928383764096","0.928372472930","0.000023330656","","","","",""
|
||||
"POLISH_LUCENE_STEMPEL_DIRECT","PL_PL","ALL_WORDS","PRIMARY_OUTPUT","9990","122341","1","122341","0","1","122341","31432","797573","66669","323394","7482411334","66669","7482478003","0.000891","323394","1120967","28.849556","0.922858412343","0.711504442147","0.999991089984","0.999947877619","0.855747766065","0.871105640425","0.803515398127","0.745658746735","0.671563509358","0.810319603524","0.810295555502","0.000052122381","0.803489769425","0.991766794523","0.931068514076","0.960459620808","0.960459620808"
|
||||
"POLISH_LUCENE_STEMPEL_DIRECT","PL_PL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","120925","1","120925","0","1","120925","30830","794493","66274","320158","7310186425","66274","7310252699","0.000907","320158","1114651","28.722712","0.923005877316","0.712772876892","0.999990934103","0.999947146412","0.856381905497","0.871590591697","0.804379630033","0.746792242917","0.672771767894","0.811106376847","0.811081975971","0.000052853588","0.804353636298","0.991711086900","0.931317880193","0.960566151650","0.960566151650"
|
||||
"POLISH_LUCENE_STEMPEL_FILTER","PL_PL","ALL_WORDS","PRIMARY_OUTPUT","9990","122341","1","122341","0","1","122341","31432","797573","66669","323394","7482411334","66669","7482478003","0.000891","323394","1120967","28.849556","0.922858412343","0.711504442147","0.999991089984","0.999947877619","0.855747766065","0.871105640425","0.803515398127","0.745658746735","0.671563509358","0.810319603524","0.810295555502","0.000052122381","0.803489769425","0.991766794523","0.931068514076","0.960459620808","0.960459620808"
|
||||
"POLISH_LUCENE_STEMPEL_FILTER","PL_PL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","120925","1","120925","0","1","120925","30830","794493","66274","320158","7310186425","66274","7310252699","0.000907","320158","1114651","28.722712","0.923005877316","0.712772876892","0.999990934103","0.999947146412","0.856381905497","0.871590591697","0.804379630033","0.746792242917","0.672771767894","0.811106376847","0.811081975971","0.000052853588","0.804353636298","0.991711086900","0.931317880193","0.960566151650","0.960566151650"
|
||||
"POLISH_RADIXOR","PL_PL","ALL_WORDS","PRIMARY_OUTPUT","9990","122341","1","122341","0","1","122341","10074","1099420","13669","21547","7482464334","13669","7482478003","0.000183","21547","1120967","1.922180","0.987719760055","0.980778203105","0.999998173199","0.999995294243","0.990388188152","0.986323599045","0.984236742499","0.982158698021","0.968962733423","0.984242862020","0.984240510632","0.000004705757","0.984234389298","0.996967243455","0.996469409869","0.996718264498","0.996718264498"
|
||||
"POLISH_RADIXOR","PL_PL","ALL_WORDS","ANY_CANDIDATE","9990","122341","1","119475","2866","4","125778","10079","1120967","0","0","7482478003","0","7482478003","0.000000","0","1120967","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","ALL_WORDS","ALL_CANDIDATES","9990","122341","1","119475","2866","4","125778","10079","1120967","38073","0","7482439930","38073","7482478003","0.000509","0","1120967","0.000000","0.967151263114","1.000000000000","0.999994911712","0.999994912475","0.999997455856","0.973547222425","0.983301367057","0.993252946885","0.967151263114","0.983438489746","0.983435987734","0.000005087525","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","120925","1","120925","0","1","120925","9844","1093651","13669","21000","7310239030","13669","7310252699","0.000187","21000","1114651","1.883998","0.987655781527","0.981160022285","0.999998130160","0.999995258206","0.990579076223","0.986349757961","0.984397186102","0.982452329568","0.969273787578","0.984402543989","0.984400174373","0.000004741794","0.984394814870","0.996926141446","0.996646530259","0.996786316244","0.996786316244"
|
||||
"POLISH_RADIXOR","PL_PL","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","120925","1","118145","2780","4","124274","9847","1114651","0","0","7310252699","0","7310252699","0.000000","0","1114651","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","120925","1","118145","2780","4","124274","9847","1114651","38073","0","7310214626","38073","7310252699","0.000521","0","1114651","0.000000","0.966971278467","1.000000000000","0.999994791835","0.999994792629","0.999997395918","0.973401318686","0.983208335630","0.993214975136","0.966971278467","0.983346977657","0.983344416937","0.000005207371","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_LIGHT_STEM_FILTER","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","112814","149830","2511","5339230","22358201245","2511","22358203756","0.000011","5339230","5489060","97.270389","0.983517240927","0.027296112631","0.999999887692","0.999761142266","0.513648000162","0.122843213263","0.053118010934","0.033885033146","0.027283631587","0.163848092400","0.163827867245","0.000238857734","0.053105458848","0.999226179883","0.720580123442","0.837329788424","0.837329788424"
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_LIGHT_STEM_FILTER","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","112814","149830","2511","5339230","22358201245","2511","22358203756","0.000011","5339230","5489060","97.270389","0.983517240927","0.027296112631","0.999999887692","0.999761142266","0.513648000162","0.122843213263","0.053118010934","0.033885033146","0.027283631587","0.163848092400","0.163827867245","0.000238857734","0.053105458848","0.999226179883","0.720580123442","0.837329788424","0.837329788424"
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_MINIMAL_STEM_FILTER","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","167745","43406","598","5445654","22358203158","598","22358203756","0.000003","5445654","5489060","99.209227","0.986410326334","0.007907729192","0.999999973254","0.999756469021","0.503953851223","0.038310165654","0.015689679353","0.009864890589","0.007906867787","0.088319113068","0.088308061848","0.000243530979","0.015685836581","0.999664059174","0.692382565086","0.818121623354","0.818121623354"
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_MINIMAL_STEM_FILTER","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","167745","43406","598","5445654","22358203158","598","22358203756","0.000003","5445654","5489060","99.209227","0.986410326334","0.007907729192","0.999999973254","0.999756469021","0.503953851223","0.038310165654","0.015689679353","0.009864890589","0.007906867787","0.088319113068","0.088308061848","0.000243530979","0.015685836581","0.999664059174","0.692382565086","0.818121623354","0.818121623354"
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_STEM_FILTER","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","27586","3803488","99075","1685572","22358104681","99075","22358203756","0.000443","1685572","5489060","30.707844","0.974612837768","0.692921556696","0.999995568741","0.999920198913","0.846458562719","0.901329863268","0.809974591186","0.735433886866","0.680636384053","0.821784792219","0.821750382444","0.000079801087","0.809935821347","0.996728545383","0.918475110538","0.956003146785","0.956003146785"
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_STEM_FILTER","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","27586","3803488","99075","1685572","22358104681","99075","22358203756","0.000443","1685572","5489060","30.707844","0.974612837768","0.692921556696","0.999995568741","0.999920198913","0.846458562719","0.901329863268","0.809974591186","0.735433886866","0.680636384053","0.821784792219","0.821750382444","0.000079801087","0.809935821347","0.996728545383","0.918475110538","0.956003146785","0.956003146785"
|
||||
"PORTUGUESE_RADIXOR","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","4001","5472616","20678","16444","22358183078","20678","22358203756","0.000092","16444","5489060","0.299578","0.996235774018","0.997004222945","0.999999075149","0.999998340077","0.998501649047","0.996389369023","0.996619850353","0.996850438335","0.993262474550","0.996619924417","0.996619094289","0.000001659923","0.996619020188","0.999299376330","0.999346803887","0.999323089546","0.999323089546"
|
||||
"PORTUGUESE_RADIXOR","PT_PT","ALL_WORDS","ANY_CANDIDATE","4001","211489","0","210699","790","3","212297","4001","5489060","0","0","22358203756","0","22358203756","0.000000","0","5489060","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","ALL_WORDS","ALL_CANDIDATES","4001","211489","0","210699","790","3","212297","4001","5489060","38310","0","22358165446","38310","22358203756","0.000171","0","5489060","0.000000","0.993069036450","1.000000000000","0.999998286535","0.999998286956","0.999999143267","0.994447532369","0.996522466897","0.998606078314","0.993069036450","0.996528492543","0.996527638784","0.000001713044","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","4001","5472616","20678","16444","22358183078","20678","22358203756","0.000092","16444","5489060","0.299578","0.996235774018","0.997004222945","0.999999075149","0.999998340077","0.998501649047","0.996389369023","0.996619850353","0.996850438335","0.993262474550","0.996619924417","0.996619094289","0.000001659923","0.996619020188","0.999299376330","0.999346803887","0.999323089546","0.999323089546"
|
||||
"PORTUGUESE_RADIXOR","PT_PT","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4001","211489","0","210699","790","3","212297","4001","5489060","0","0","22358203756","0","22358203756","0.000000","0","5489060","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4001","211489","0","210699","790","3","212297","4001","5489060","38310","0","22358165446","38310","22358203756","0.000171","0","5489060","0.000000","0.993069036450","1.000000000000","0.999998286535","0.999998286956","0.999999143267","0.994447532369","0.996522466897","0.998606078314","0.993069036450","0.996528492543","0.996527638784","0.000001713044","","","","",""
|
||||
"RUSSIAN_LUCENE_RUSSIAN_LIGHT_STEM_FILTER","RU_RU","ALL_WORDS","PRIMARY_OUTPUT","37410","768882","10","768882","0","1","768882","232250","3081067","321183","10008438","295575969833","321183","295576291016","0.000109","10008438","13089505","76.461547","0.905596884415","0.235384531348","0.999998913367","0.999965054154","0.617691722357","0.577011147253","0.373649378129","0.276277984307","0.229747124085","0.461696326851","0.461686629842","0.000034945846","0.373637933830","0.994310930069","0.870888421754","0.928516167212","0.928516167212"
|
||||
"RUSSIAN_LUCENE_RUSSIAN_LIGHT_STEM_FILTER","RU_RU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","768133","10","768133","0","1","768133","232143","3078888","318921","10008238","295000362731","318921","295000681652","0.000108","10008238","13087126","76.473918","0.906139220892","0.235260820443","0.999998918914","0.999964994315","0.617629869679","0.577038425373","0.373539598427","0.276151716079","0.229664120975","0.461713175622","0.461703471649","0.000035005685","0.373528142103","0.994349544377","0.870767393403","0.928464208705","0.928464208705"
|
||||
"RUSSIAN_RADIXOR","RU_RU","ALL_WORDS","PRIMARY_OUTPUT","37410","768882","10","768882","0","1","768882","37561","12823203","155850","266302","295576135166","155850","295576291016","0.000053","266302","13089505","2.034470","0.987992190185","0.979655304001","0.999999472725","0.999998571830","0.989827388363","0.986313480705","0.983806085477","0.981311406466","0.968128298562","0.983814916245","0.983814202914","0.000001428170","0.983805371373","0.997699288696","0.997273959852","0.997486578934","0.997486578934"
|
||||
"RUSSIAN_RADIXOR","RU_RU","ALL_WORDS","ANY_CANDIDATE","37410","768882","10","749720","19162","4","788492","37593","13089492","0","13","295576291016","0","295576291016","0.000000","13","13089505","0.000099","1.000000000000","0.999999006838","1.000000000000","0.999999999956","0.999999503419","0.999999801367","0.999999503419","0.999999205470","0.999999006838","0.999999503419","0.999999503397","0.000000000044","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ALL_WORDS","ALL_CANDIDATES","37410","768882","10","749720","19162","4","788492","37593","13089492","434710","13","295575856306","434710","295576291016","0.000147","13","13089505","0.000099","0.967856883534","0.999999006838","0.999998529280","0.999998529301","0.999998768059","0.974118939969","0.983665447282","0.993400920813","0.967855953192","0.983796687479","0.983795964011","0.000001470699","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","768133","10","768133","0","1","768133","37282","12821513","155850","265613","295000525802","155850","295000681652","0.000053","265613","13087126","2.029575","0.987990626447","0.979704252867","0.999999471696","0.999998571379","0.989851862281","0.986322156837","0.983829991833","0.981350389119","0.968174600634","0.983838715706","0.983838002141","0.000001428621","0.983829277503","0.997696524283","0.997321167437","0.997508810549","0.997508810549"
|
||||
"RUSSIAN_RADIXOR","RU_RU","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","37297","768133","10","749142","18991","4","787549","37306","13087126","0","0","295000681652","0","295000681652","0.000000","0","13087126","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","37297","768133","10","749142","18991","4","787549","37306","13087126","434710","0","295000246942","434710","295000681652","0.000147","0","13087126","0.000000","0.967851259252","1.000000000000","0.999998526410","0.999998526476","0.999999263205","0.974114570610","0.983663023007","0.993400519870","0.967851259252","0.983794317554","0.983793592699","0.000001473524","","","","",""
|
||||
"SNOWBALL_DANISH_DIRECT","DA_DK","ALL_WORDS","PRIMARY_OUTPUT","4179","28079","32","28079","0","1","28079","5553","78732","6341","11163","394104845","6341","394111186","0.001609","11163","89895","12.417821","0.925464013259","0.875821792091","0.999983910632","0.999955596266","0.937902851361","0.915090414169","0.899958849618","0.885319563795","0.818113803566","0.900300811178","0.900278764621","0.000044403734","0.899936659693","0.994195946109","0.978579164615","0.986325742978","0.986325742978"
|
||||
"SNOWBALL_DANISH_DIRECT","DA_DK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","28033","32","28033","0","1","28033","5539","78627","6341","11113","392814447","6341","392820788","0.001614","11113","89740","12.383552","0.925371904717","0.876164475150","0.999983857779","0.999955577673","0.938074166465","0.915093153823","0.900096160451","0.885582797210","0.818340774971","0.900432112497","0.900410054086","0.000044422327","0.900073960926","0.994185354918","0.978644331817","0.986353630937","0.986353630937"
|
||||
"SNOWBALL_DANISH_LUCENE_FILTER","DA_DK","ALL_WORDS","PRIMARY_OUTPUT","4179","28079","32","28079","0","1","28079","5546","78744","6507","11151","394104679","6507","394111186","0.001651","11151","89895","12.404472","0.923672449590","0.875955281161","0.999983489431","0.999955205602","0.937969385296","0.913717599716","0.899181254496","0.885100184115","0.816829526358","0.899497504322","0.899475250557","0.000044794398","0.899158868074","0.994052746860","0.978603476314","0.986267614487","0.986267614487"
|
||||
"SNOWBALL_DANISH_LUCENE_FILTER","DA_DK","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","28033","32","28033","0","1","28033","5539","78627","6341","11113","392814447","6341","392820788","0.001614","11113","89740","12.383552","0.925371904717","0.876164475150","0.999983857779","0.999955577673","0.938074166465","0.915093153823","0.900096160451","0.885582797210","0.818340774971","0.900432112497","0.900410054086","0.000044422327","0.900073960926","0.994185354918","0.978644331817","0.986353630937","0.986353630937"
|
||||
"SNOWBALL_DUTCH_DIRECT","NL_NL","ALL_WORDS","PRIMARY_OUTPUT","4992","26477","85","26477","0","1","26477","12051","29325","4382","35241","350433578","4382","350437960","0.001250","35241","64566","54.581359","0.869997329931","0.454186413902","0.999987495647","0.999886953739","0.727086954774","0.735353119953","0.596806854375","0.502190286022","0.425320531415","0.628602392126","0.628557411604","0.000113046261","0.596755898222","0.992814719235","0.917346281080","0.953589661124","0.953589661124"
|
||||
"SNOWBALL_DUTCH_DIRECT","NL_NL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25678","84","25678","0","1","25678","11466","29111","4382","34036","329599474","4382","329603856","0.001329","34036","63147","53.899631","0.869166691547","0.461003689803","0.999986705253","0.999883464224","0.730495197528","0.738411822300","0.602462748344","0.508789468717","0.431088865524","0.633000040962","0.632953313739","0.000116535776","0.602409959779","0.992557044900","0.918058993802","0.953855618789","0.953855618789"
|
||||
"SNOWBALL_DUTCH_LUCENE_FILTER","NL_NL","ALL_WORDS","PRIMARY_OUTPUT","4992","26477","85","26477","0","1","26477","14573","15302","1588","49264","350436372","1588","350437960","0.000453","49264","64566","76.300220","0.905979869745","0.236997800700","0.999995468527","0.999854916880","0.618496634614","0.579068464950","0.375712040856","0.278062466837","0.231308764398","0.463373754768","0.463333378452","0.000145083120","0.375664346452","0.995827666179","0.888410302915","0.939057139355","0.939057139355"
|
||||
"SNOWBALL_DUTCH_LUCENE_FILTER","NL_NL","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25678","84","25678","0","1","25678","14116","14972","1544","48175","329602312","1544","329603856","0.000468","48175","63147","76.290243","0.906514894648","0.237097565997","0.999995315589","0.999849184178","0.618546440793","0.579362438183","0.375883408860","0.278182412747","0.231438685443","0.463608105042","0.463566154833","0.000150815822","0.375833834664","0.995816517119","0.887491664267","0.938538755173","0.938538755173"
|
||||
"SNOWBALL_FINNISH_DIRECT","FI_FI","ALL_WORDS","PRIMARY_OUTPUT","57027","1811717","292","1811717","0","1","1811717","381483","15114332","1544812","16409363","1641125269679","1544812","1641126814491","0.000094","16409363","31523695","52.054060","0.907269425128","0.479459403474","0.999999058688","0.999989060059","0.739729231081","0.769880311353","0.627374073993","0.529384116965","0.457061215373","0.659544431681","0.659540149918","0.000010939941","0.627369124557","0.991871857177","0.904138579582","0.945975396220","0.945975396220"
|
||||
"SNOWBALL_FINNISH_DIRECT","FI_FI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1757055","274","1757055","0","1","1757055","372232","14692070","1513705","16121763","1543587930447","1513705","1543589444152","0.000098","16121763","30813833","52.319888","0.906594717007","0.476801117213","0.999999019360","0.999988575255","0.738400068286","0.768116957494","0.624933751043","0.526744348874","0.454475376380","0.657468914800","0.657464451566","0.000011424745","0.624928589970","0.991731678095","0.902933406396","0.945251664438","0.945251664438"
|
||||
"SNOWBALL_FINNISH_LUCENE_FILTER","FI_FI","ALL_WORDS","PRIMARY_OUTPUT","57027","1811717","292","1811717","0","1","1811717","377778","15153638","1922153","16370057","1641124892338","1922153","1641126814491","0.000117","16370057","31523695","51.929372","0.887434028678","0.480706275073","0.999998828760","0.999988854086","0.740352551917","0.758996033322","0.623613097472","0.529216231176","0.453079796332","0.653142485450","0.653138019077","0.000011145914","0.623608016975","0.990717710840","0.904385055188","0.945584912497","0.945584912497"
|
||||
"SNOWBALL_FINNISH_LUCENE_FILTER","FI_FI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1757055","274","1757055","0","1","1757055","372232","14692070","1513705","16121763","1543587930447","1513705","1543589444152","0.000098","16121763","30813833","52.319888","0.906594717007","0.476801117213","0.999999019360","0.999988575255","0.738400068286","0.768116957494","0.624933751043","0.526744348874","0.454475376380","0.657468914800","0.657464451566","0.000011424745","0.624928589970","0.991731678095","0.902933406396","0.945251664438","0.945251664438"
|
||||
"SNOWBALL_FRENCH_DIRECT","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","85627","3766640","1654723","1687975","90394450107","1654723","90396104830","0.001831","1687975","5454615","30.945814","0.694777309691","0.690541862258","0.999981694753","0.999963023890","0.845261778506","0.693926068790","0.692653111288","0.691384815533","0.529815856272","0.692656348624","0.692637859933","0.000036976110","0.692634622294","0.959459328254","0.944947915186","0.952148333884","0.952148333884"
|
||||
"SNOWBALL_FRENCH_DIRECT","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","84526","3758589","1646111","1681970","88710480395","1646111","88712126506","0.001856","1681970","5440559","30.915389","0.695429718578","0.690846106071","0.999981444352","0.999962486787","0.845413775212","0.694508136723","0.693130334647","0.691757988461","0.530374491828","0.693134123475","0.693115366288","0.000037513213","0.693111577099","0.959520798119","0.944537159644","0.951970023370","0.951970023370"
|
||||
"SNOWBALL_FRENCH_LUCENE_FILTER","FR_FR","ALL_WORDS","PRIMARY_OUTPUT","59240","425210","2301","425210","0","1","425210","85202","3763777","1661388","1690838","90394443442","1661388","90396104830","0.001838","1690838","5454615","30.998301","0.693762678186","0.690016985617","0.999981621022","0.999962918494","0.844999303320","0.693010289898","0.691884762376","0.690762884895","0.528917286853","0.691887297134","0.691868755638","0.000037081506","0.691866220643","0.958697792387","0.944714715363","0.951654891797","0.951654891797"
|
||||
"SNOWBALL_FRENCH_LUCENE_FILTER","FR_FR","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","421231","2133","421231","0","1","421231","84810","3755856","1641925","1684703","88710484581","1641925","88712126506","0.001851","1684703","5440559","30.965623","0.695814817237","0.690343767984","0.999981491538","0.999962503165","0.845162629761","0.694713680979","0.693068495729","0.691431084156","0.530302080457","0.693073894149","0.693055145391","0.000037496835","0.693049746458","0.959566165512","0.944384738154","0.951914926182","0.951914926182"
|
||||
"SNOWBALL_GERMAN_DIRECT","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","81649","771138","190680","612734","44095055299","190680","44095245979","0.000432","612734","1383872","44.276783","0.801750435114","0.557232171762","0.999995675724","0.999981780603","0.778613923743","0.737064397386","0.657493530688","0.593429030432","0.489750735447","0.668401927114","0.668393541401","0.000018219397","0.657484715679","0.983724573695","0.949324273697","0.966218331938","0.966218331938"
|
||||
"SNOWBALL_GERMAN_DIRECT","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","37843","516936","87697","356475","11263668645","87697","11263756342","0.000779","356475","873411","40.814118","0.854958297017","0.591858815609","0.999992214231","0.999960569321","0.795925514920","0.785153327381","0.699486618802","0.630674793334","0.537854226580","0.711347035607","0.711329191110","0.000039430679","0.699467554207","0.988417636496","0.932451723900","0.959619376607","0.959619376607"
|
||||
"SNOWBALL_GERMAN_LUCENE_FILTER","DE_DE","ALL_WORDS","PRIMARY_OUTPUT","54092","296974","1474","296974","0","1","296974","86669","751056","295701","632816","44094950278","295701","44095245979","0.000671","632816","1383872","45.727929","0.717507501741","0.542720714054","0.999993294039","0.999978943584","0.771357004047","0.674088567377","0.617993120299","0.570516594262","0.447170798768","0.624024185176","0.624014089276","0.000021056416","0.617982794334","0.975844522648","0.942925157079","0.959102449371","0.959102449371"
|
||||
"SNOWBALL_GERMAN_LUCENE_FILTER","DE_DE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","150098","228","150098","0","1","150098","46077","481501","77653","391910","11263678689","77653","11263756342","0.000689","391910","873411","44.871200","0.861124126806","0.551287996144","0.999993105941","0.999958315274","0.775640551042","0.774110642769","0.672222202832","0.594035281304","0.506276128631","0.689004640259","0.688986412764","0.000041684726","0.672202362239","0.989021274644","0.919542244735","0.953017108147","0.953017108147"
|
||||
"SNOWBALL_HUNGARIAN_DIRECT","HU_HU","ALL_WORDS","PRIMARY_OUTPUT","19406","916344","1","916344","0","1","916344","116105","14287912","1506056","7874191","419819036837","1506056","419820542893","0.000359","7874191","22162103","35.529981","0.904643595580","0.644700189328","0.999996412620","0.999977657711","0.822348300974","0.837136808086","0.752865700984","0.684009307333","0.603676525918","0.763690969794","0.763680928295","0.000022342289","0.752854843818","0.991947513126","0.924304257644","0.956931989551","0.956931989551"
|
||||
"SNOWBALL_HUNGARIAN_DIRECT","HU_HU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","878513","1","878513","0","1","878513","111379","13776526","1496670","7634885","385869198247","1496670","385870694917","0.000388","7634885","21411411","35.658019","0.902006757459","0.643419810119","0.999996121317","0.999976336507","0.821707965718","0.834898516372","0.751079383241","0.682554714263","0.601382804609","0.761819543337","0.761808891723","0.000023663493","0.751067882221","0.991609896137","0.923288102987","0.956230170303","0.956230170303"
|
||||
"SNOWBALL_HUNGARIAN_LUCENE_FILTER","HU_HU","ALL_WORDS","PRIMARY_OUTPUT","19406","916344","1","916344","0","1","916344","114867","14299358","1792049","7862745","419818750844","1792049","419820542893","0.000427","7862745","22162103","35.478334","0.888633169244","0.645216656560","0.999995731393","0.999977003783","0.822606193976","0.826287586346","0.747610245439","0.682613266689","0.596946950992","0.757205997314","0.757195513225","0.000022996217","0.747599036407","0.990687085622","0.924490230693","0.956444632598","0.956444632598"
|
||||
"SNOWBALL_HUNGARIAN_LUCENE_FILTER","HU_HU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","878513","1","878513","0","1","878513","111379","13776526","1496670","7634885","385869198247","1496670","385870694917","0.000388","7634885","21411411","35.658019","0.902006757459","0.643419810119","0.999996121317","0.999976336507","0.821707965718","0.834898516372","0.751079383241","0.682554714263","0.601382804609","0.761819543337","0.761808891723","0.000023663493","0.751067882221","0.991609896137","0.923288102987","0.956230170303","0.956230170303"
|
||||
"SNOWBALL_ITALIAN_DIRECT","IT_IT","ALL_WORDS","PRIMARY_OUTPUT","10009","327551","0","327551","0","1","327551","46828","4499650","504775","1644164","53638016436","504775","53638521211","0.000941","1644164","6143814","26.761292","0.899134266174","0.732387080729","0.999990589319","0.999959941236","0.866188835024","0.859975076366","0.807239600802","0.760598128154","0.676782697802","0.811488952720","0.811469907061","0.000040058764","0.807219778599","0.987993752409","0.933407841859","0.959925420024","0.959925420024"
|
||||
"SNOWBALL_ITALIAN_DIRECT","IT_IT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","327469","0","327469","0","1","327469","46814","4498652","504774","1643522","53611162298","504774","53611667072","0.000942","1643522","6142174","26.757985","0.899114326863","0.732420149608","0.999990584624","0.999959933163","0.866205367116","0.859969602244","0.807251650876","0.760623806435","0.676799637969","0.811498274672","0.811479224597","0.000040066837","0.807231824542","0.987990915845","0.933413003102","0.959926810498","0.959926810498"
|
||||
"SNOWBALL_ITALIAN_LUCENE_FILTER","IT_IT","ALL_WORDS","PRIMARY_OUTPUT","10009","327551","0","327551","0","1","327551","46828","4499650","504775","1644164","53638016436","504775","53638521211","0.000941","1644164","6143814","26.761292","0.899134266174","0.732387080729","0.999990589319","0.999959941236","0.866188835024","0.859975076366","0.807239600802","0.760598128154","0.676782697802","0.811488952720","0.811469907061","0.000040058764","0.807219778599","0.987993752409","0.933407841859","0.959925420024","0.959925420024"
|
||||
"SNOWBALL_ITALIAN_LUCENE_FILTER","IT_IT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","327469","0","327469","0","1","327469","46814","4498652","504774","1643522","53611162298","504774","53611667072","0.000942","1643522","6142174","26.757985","0.899114326863","0.732420149608","0.999990584624","0.999959933163","0.866205367116","0.859969602244","0.807251650876","0.760623806435","0.676799637969","0.811498274672","0.811479224597","0.000040066837","0.807231824542","0.987990915845","0.933413003102","0.959926810498","0.959926810498"
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT","NB_NO","ALL_WORDS","PRIMARY_OUTPUT","17929","75310","252","75310","0","1","75310","24394","106626","23997","35554","2835594218","23997","2835618215","0.000846","35554","142180","25.006330","0.816288096277","0.749936699958","0.999991537295","0.999978999989","0.874964118626","0.802094867845","0.781706946038","0.762329786671","0.641641141674","0.782409356499","0.782398932962","0.000021000011","0.781696464373","0.988328173631","0.971119668400","0.979648355684","0.979648355684"
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT","NB_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","75251","252","75251","0","1","75251","24367","106567","23997","35524","2831152787","23997","2831176784","0.000848","35524","142091","25.000880","0.816205079501","0.749991202821","0.999991524019","0.999978977642","0.874991363420","0.802043209347","0.781698483431","0.762360357576","0.641629738452","0.782397999310","0.782387564360","0.000021022358","0.781687990535","0.988317966243","0.971127035574","0.979647089734","0.979647089734"
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER","NB_NO","ALL_WORDS","PRIMARY_OUTPUT","17929","75310","252","75310","0","1","75310","24396","106589","24046","35591","2835594169","24046","2835618215","0.000848","35591","142180","25.032353","0.815929880966","0.749676466451","0.999991520015","0.999978969662","0.874833993233","0.801758635215","0.781401315910","0.762052176648","0.641229410562","0.782101930719","0.782091491842","0.000021030338","0.781390819068","0.988295184140","0.971086244692","0.979615142710","0.979615142710"
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER","NB_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","75251","252","75251","0","1","75251","24381","106512","23993","35579","2831152791","23993","2831176784","0.000847","35579","142091","25.039587","0.816152637830","0.749604126933","0.999991525432","0.999978959629","0.874797826182","0.801914137847","0.781464144742","0.762031224736","0.641314033862","0.782170943927","0.782160500766","0.000021040371","0.781453643056","0.988310445474","0.971073741466","0.979616277822","0.979616277822"
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT","NN_NO","ALL_WORDS","PRIMARY_OUTPUT","4688","18250","23","18250","0","1","18250","6138","22004","8274","8648","166483199","8274","166491473","0.004970","8648","30652","28.213493","0.726732280864","0.717865065901","0.999950303761","0.999898379870","0.858907684831","0.724941356316","0.722271459051","0.719621155632","0.565277706417","0.722285066089","0.722234252664","0.000101620130","0.722220641604","0.980542486408","0.964998187466","0.972708239744","0.972708239744"
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT","NN_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","18219","23","18219","0","1","18219","6120","21971","8274","8624","165918002","8274","165926276","0.004987","8624","30595","28.187612","0.726434121342","0.718123876450","0.999950134480","0.999898178365","0.859037005465","0.724756721095","0.722255095332","0.719770679771","0.565257660346","0.722267047015","0.722216132089","0.000101821635","0.722204176866","0.980505813025","0.965064509051","0.972723884952","0.972723884952"
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER","NN_NO","ALL_WORDS","PRIMARY_OUTPUT","4688","18250","23","18250","0","1","18250","6144","21978","8295","8674","166483178","8295","166491473","0.004982","8674","30652","28.298317","0.725993459518","0.717016834138","0.999950177629","0.999898097625","0.858483505883","0.724180198229","0.721477226098","0.718794356395","0.564305338023","0.721491186328","0.721440231913","0.000101902375","0.721426267560","0.980461058483","0.964862123312","0.972599049418","0.972599049418"
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER","NN_NO","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","18219","23","18219","0","1","18219","6130","21948","8274","8647","165918002","8274","165926276","0.004987","8647","30595","28.262788","0.726225928132","0.717372119627","0.999950134480","0.999898039774","0.858661127054","0.724437725685","0.721771872996","0.719125568472","0.564665929147","0.721785448310","0.721734464790","0.000101960226","0.721720885459","0.980505813025","0.964944952705","0.972663150405","0.972663150405"
|
||||
"SNOWBALL_PORTUGUESE_DIRECT","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","11315","4817239","167230","671821","22358036526","167230","22358203756","0.000748","671821","5489060","12.239272","0.966449786326","0.877607277020","0.999992520419","0.999962481554","0.938799898719","0.947270839082","0.919888415834","0.894044585092","0.851660540743","0.920957852105","0.920939611009","0.000037518446","0.919869695779","0.996663145176","0.967923515462","0.982083116554","0.982083116554"
|
||||
"SNOWBALL_PORTUGUESE_DIRECT","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","11315","4817239","167230","671821","22358036526","167230","22358203756","0.000748","671821","5489060","12.239272","0.966449786326","0.877607277020","0.999992520419","0.999962481554","0.938799898719","0.947270839082","0.919888415834","0.894044585092","0.851660540743","0.920957852105","0.920939611009","0.000037518446","0.919869695779","0.996663145176","0.967923515462","0.982083116554","0.982083116554"
|
||||
"SNOWBALL_PORTUGUESE_LUCENE_FILTER","PT_PT","ALL_WORDS","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","11315","4817239","167230","671821","22358036526","167230","22358203756","0.000748","671821","5489060","12.239272","0.966449786326","0.877607277020","0.999992520419","0.999962481554","0.938799898719","0.947270839082","0.919888415834","0.894044585092","0.851660540743","0.920957852105","0.920939611009","0.000037518446","0.919869695779","0.996663145176","0.967923515462","0.982083116554","0.982083116554"
|
||||
"SNOWBALL_PORTUGUESE_LUCENE_FILTER","PT_PT","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211489","0","211489","0","1","211489","11315","4817239","167230","671821","22358036526","167230","22358203756","0.000748","671821","5489060","12.239272","0.966449786326","0.877607277020","0.999992520419","0.999962481554","0.938799898719","0.947270839082","0.919888415834","0.894044585092","0.851660540743","0.920957852105","0.920939611009","0.000037518446","0.919869695779","0.996663145176","0.967923515462","0.982083116554","0.982083116554"
|
||||
"SNOWBALL_RUSSIAN_DIRECT","RU_RU","ALL_WORDS","PRIMARY_OUTPUT","37410","768882","10","768882","0","1","768882","64358","8766656","3782908","4322849","295572508108","3782908","295576291016","0.001280","4322849","13089505","33.025305","0.698562595481","0.669746946122","0.999987201585","0.999972577645","0.834867073854","0.692602792505","0.683851352013","0.675318311031","0.519585195076","0.684003044583","0.683989349009","0.000027422355","0.683837646322","0.974179960240","0.953661001039","0.963811283954","0.963811283954"
|
||||
"SNOWBALL_RUSSIAN_DIRECT","RU_RU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","768133","10","768133","0","1","768133","64159","8764719","3782908","4322407","294996898744","3782908","295000681652","0.001282","4322407","13087126","33.027931","0.698516062041","0.669720685810","0.999987176613","0.999972525638","0.834853931211","0.692560581516","0.683815365804","0.675288254087","0.519543647630","0.683966853085","0.683953131513","0.000027474362","0.683801634112","0.974148936252","0.953633561396","0.963782086975","0.963782086975"
|
||||
"SNOWBALL_RUSSIAN_LUCENE_FILTER","RU_RU","ALL_WORDS","PRIMARY_OUTPUT","37410","768882","10","768882","0","1","768882","64266","8766889","3785790","4322616","295572505226","3785790","295576291016","0.001281","4322616","13089505","33.023525","0.698407806015","0.669764746642","0.999987191835","0.999972568683","0.834875969239","0.692484865100","0.683786451263","0.675303850911","0.519510266339","0.683936347366","0.683922647122","0.000027431317","0.683772741022","0.974131099393","0.953673855106","0.963793934411","0.963793934411"
|
||||
"SNOWBALL_RUSSIAN_LUCENE_FILTER","RU_RU","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","768133","10","768133","0","1","768133","64159","8764719","3782908","4322407","294996898744","3782908","295000681652","0.001282","4322407","13087126","33.027931","0.698516062041","0.669720685810","0.999987176613","0.999972525638","0.834853931211","0.692560581516","0.683815365804","0.675288254087","0.519543647630","0.683966853085","0.683953131513","0.000027474362","0.683801634112","0.974148936252","0.953633561396","0.963782086975","0.963782086975"
|
||||
"SNOWBALL_SPANISH_DIRECT","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","195021","12811687","2228819","29161649","379565089291","2228819","379567318110","0.000587","29161649","41973336","69.476605","0.851812232913","0.305233946618","0.999994128001","0.999917308483","0.652614037309","0.627191552465","0.449423738186","0.350172671706","0.289843040458","0.509903921959","0.509876023351","0.000082691517","0.449391616998","0.981405614580","0.852462513401","0.912400934512","0.912400934512"
|
||||
"SNOWBALL_SPANISH_DIRECT","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","194444","12787018","2201196","29076352","377858468569","2201196","377860669765","0.000583","29076352","41863370","69.455354","0.853138205793","0.305446455935","0.999994174583","0.999917233823","0.652720315259","0.627945981812","0.449838583213","0.350441220963","0.290188220622","0.510478247707","0.510450359519","0.000082766177","0.449806446076","0.981468761133","0.852555702466","0.912481600659","0.912481600659"
|
||||
"SNOWBALL_SPANISH_LUCENE_FILTER","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","194971","12811693","2230481","29161643","379565087629","2230481","379567318110","0.000588","29161643","41973336","69.476591","0.851718175843","0.305234089566","0.999994123622","0.999917304121","0.652614106594","0.627150877515","0.449410800675","0.350169642836","0.289832278511","0.509875888791","0.509847985527","0.000082695879","0.449378676109","0.981386049614","0.852460744495","0.912391466047","0.912391466047"
|
||||
"SNOWBALL_SPANISH_LUCENE_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","194444","12787018","2201196","29076352","377858468569","2201196","377860669765","0.000583","29076352","41863370","69.455354","0.853138205793","0.305446455935","0.999994174583","0.999917233823","0.652720315259","0.627945981812","0.449838583213","0.350441220963","0.290188220622","0.510478247707","0.510450359519","0.000082766177","0.449806446076","0.981468761133","0.852555702466","0.912481600659","0.912481600659"
|
||||
"SNOWBALL_SWEDISH_DIRECT","SV_SE","ALL_WORDS","PRIMARY_OUTPUT","12371","98108","68","98108","0","1","98108","25915","237017","67105","148325","4812088331","67105","4812155436","0.001394","148325","385342","38.491781","0.779348419384","0.615082186733","0.999986055105","0.999955235704","0.807534120919","0.739831942216","0.687539886056","0.642151948805","0.523855832838","0.692360693585","0.692339154006","0.000044764296","0.687517812951","0.984860422704","0.942685282143","0.963311451566","0.963311451566"
|
||||
"SNOWBALL_SWEDISH_DIRECT","SV_SE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","97881","68","97881","0","1","97881","25840","236588","67105","147975","4789844472","67105","4789911577","0.001401","147975","384563","38.478741","0.779036724587","0.615212591955","0.999985990347","0.999955100897","0.807599291151","0.739644914918","0.687500000000","0.642223301999","0.523809523810","0.692295603454","0.692273994517","0.000044899103","0.687477858823","0.984821307273","0.942694565973","0.963297587020","0.963297587020"
|
||||
"SNOWBALL_SWEDISH_LUCENE_FILTER","SV_SE","ALL_WORDS","PRIMARY_OUTPUT","12371","98108","68","98108","0","1","98108","26781","230676","64262","154666","4812091174","64262","4812155436","0.001335","154666","385342","40.137333","0.782116919488","0.598626674487","0.999986645901","0.999954508853","0.799306660194","0.736939762085","0.678179573117","0.628097931391","0.513064830384","0.684248529829","0.684226838572","0.000045491147","0.678157227687","0.985207247898","0.939659207875","0.961894327137","0.961894327137"
|
||||
"SNOWBALL_SWEDISH_LUCENE_FILTER","SV_SE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","97881","68","97881","0","1","97881","26706","230247","64262","154316","4789847315","64262","4789911577","0.001342","154316","384563","40.127625","0.781799537535","0.598723746174","0.999986583886","0.999954370671","0.799355165030","0.736743719918","0.678122496584","0.628142458291","0.512999498691","0.684165146635","0.684143384784","0.000045629329","0.678100081584","0.985169028543","0.939660518299","0.961876797342","0.961876797342"
|
||||
"SNOWBALL_YIDDISH_DIRECT","YI","ALL_WORDS","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","1087","4962","1151","1382","6391758","1151","6392909","0.018004","1382","6344","21.784363","0.811712743334","0.782156368222","0.999819956768","0.999604172550","0.890988162495","0.805624107027","0.796660512162","0.787894185271","0.662041360907","0.796797522188","0.796599716782","0.000395827450","0.796462473806","0.982918530193","0.962014249878","0.972354049666","0.972354049666"
|
||||
"SNOWBALL_YIDDISH_DIRECT","YI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","1087","4962","1151","1382","6391758","1151","6392909","0.018004","1382","6344","21.784363","0.811712743334","0.782156368222","0.999819956768","0.999604172550","0.890988162495","0.805624107027","0.796660512162","0.787894185271","0.662041360907","0.796797522188","0.796599716782","0.000395827450","0.796462473806","0.982918530193","0.962014249878","0.972354049666","0.972354049666"
|
||||
"SNOWBALL_YIDDISH_LUCENE_FILTER","YI","ALL_WORDS","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","1087","4962","1151","1382","6391758","1151","6392909","0.018004","1382","6344","21.784363","0.811712743334","0.782156368222","0.999819956768","0.999604172550","0.890988162495","0.805624107027","0.796660512162","0.787894185271","0.662041360907","0.796797522188","0.796599716782","0.000395827450","0.796462473806","0.982918530193","0.962014249878","0.972354049666","0.972354049666"
|
||||
"SNOWBALL_YIDDISH_LUCENE_FILTER","YI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","1087","4962","1151","1382","6391758","1151","6392909","0.018004","1382","6344","21.784363","0.811712743334","0.782156368222","0.999819956768","0.999604172550","0.890988162495","0.805624107027","0.796660512162","0.787894185271","0.662041360907","0.796797522188","0.796599716782","0.000395827450","0.796462473806","0.982918530193","0.962014249878","0.972354049666","0.972354049666"
|
||||
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","405552","1244317","147956","40729019","379567170154","147956","379567318110","0.000039","40729019","41973336","97.035458","0.893730611741","0.029645415842","0.999999610198","0.999892318297","0.514822513020","0.130863846499","0.057387272020","0.036752000024","0.029541282827","0.162772895888","0.162762055080","0.000107681703","0.057380579619","0.993823553768","0.756690454887","0.859195405761","0.859195405761"
|
||||
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","404617","1241848","146613","40621522","377860523152","146613","377860669765","0.000039","40621522","41863370","97.033569","0.894406108634","0.029664310351","0.999999611992","0.999892119974","0.514831961171","0.130949068412","0.057424066047","0.036775459718","0.029560783207","0.162886280533","0.162875426941","0.000107880026","0.057417362063","0.993866319748","0.756725425267","0.859233931168","0.859233931168"
|
||||
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","718633","148463","47859","41824873","379567270251","47859","379567318110","0.000013","41824873","41973336","99.646292","0.756221921130","0.003537078873","0.999999873912","0.999889695187","0.501768476392","0.017360591398","0.007041223811","0.004416184633","0.003533050405","0.051718628951","0.051713939443","0.000110304813","0.007040201537","0.995635307295","0.710609719902","0.829315972283","0.829315972283"
|
||||
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","717093","148226","47148","41715144","377860622617","47148","377860669765","0.000012","41715144","41863370","99.645929","0.758678227400","0.003540708739","0.999999875224","0.999889489251","0.501770291981","0.017379114288","0.007048522419","0.004420728101","0.003536725554","0.051829129163","0.051824445330","0.000110510749","0.007047500484","0.995675746140","0.710626433954","0.829341382913","0.829341382913"
|
||||
"SPANISH_LUCENE_SPANISH_PLURAL_STEM_FILTER","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","578805","325245","58578","41648091","379567259532","58578","379567318110","0.000015","41648091","41973336","99.225115","0.847382777999","0.007748847983","0.999999845672","0.999890132644","0.503874346827","0.037377069210","0.015357262275","0.009663967067","0.007738048760","0.081032341260","0.081026288458","0.000109867356","0.015355289170","0.995442321769","0.723731297627","0.838115191065","0.838115191065"
|
||||
"SPANISH_LUCENE_SPANISH_PLURAL_STEM_FILTER","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","577533","324656","57716","41538714","377860612049","57716","377860669765","0.000015","41538714","41863370","99.224487","0.849057985417","0.007755132948","0.999999847256","0.999889928152","0.503877490102","0.037408921072","0.015369880354","0.009671831022","0.007744455857","0.081145286724","0.081139234944","0.000110071848","0.015367905834","0.995484117647","0.723752971494","0.838144538380","0.838144538380"
|
||||
"SPANISH_RADIXOR","ES_ES","ALL_WORDS","PRIMARY_OUTPUT","65059","871332","3589","871332","0","1","871332","64995","41074684","288483","898652","379567029627","288483","379567318110","0.000076","898652","41973336","2.141007","0.993025606574","0.978589931475","0.999999239969","0.999996872745","0.989294585722","0.990104500109","0.985754921826","0.981443392220","0.971909988067","0.985781345071","0.985779787115","0.000003127255","0.985753358111","0.995417814373","0.993266303762","0.994340895233","0.994340895233"
|
||||
"SPANISH_RADIXOR","ES_ES","ALL_WORDS","ANY_CANDIDATE","65059","871332","3589","828695","42637","21","916797","65118","41972710","2","626","379567318108","2","379567318110","0.000000","626","41973336","0.001491","0.999999952350","0.999985085770","0.999999999995","0.999999998346","0.999992542882","0.999996978999","0.999992519005","0.999988059050","0.999985038121","0.999992519032","0.999992518205","0.000000001654","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","ALL_WORDS","ALL_CANDIDATES","65059","871332","3589","828695","42637","21","916797","65118","41972710","1349800","626","379565968310","1349800","379567318110","0.000356","626","41973336","0.001491","0.968842987168","0.999985085770","0.999996443846","0.999996442590","0.999990764808","0.974915259157","0.984167740127","0.993597526064","0.968828987818","0.984290880594","0.984289129583","0.000003557410","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","869371","3525","869371","0","1","869371","64814","40978337","276044","885033","377860393721","276044","377860669765","0.000073","885033","41863370","2.114099","0.993308734895","0.978859012067","0.999999269456","0.999996927576","0.989429140761","0.990384762162","0.986030938205","0.981715226337","0.972446769193","0.986057405488","0.986055874970","0.000003072424","0.986029401906","0.995463637710","0.993323040564","0.994392187139","0.994392187139"
|
||||
"SPANISH_RADIXOR","ES_ES","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","64918","869371","3525","826968","42403","21","914127","64933","41863370","0","0","377860669765","0","377860669765","0.000000","0","41863370","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","64918","869371","3525","826968","42403","21","914127","64933","41863370","1255381","0","377859414384","1255381","377860669765","0.000332","0","41863370","0.000000","0.970885497124","1.000000000000","0.999996677662","0.999996678030","0.999998338831","0.976571978660","0.985227704543","0.994038240493","0.970885497124","0.985335220686","0.985333583876","0.000003321970","","","","",""
|
||||
"SWEDISH_LUCENE_SWEDISH_LIGHT_STEM_FILTER","SV_SE","ALL_WORDS","PRIMARY_OUTPUT","12371","98108","68","98108","0","1","98108","22392","218635","45941","166707","4812109495","45941","4812155436","0.000955","166707","385342","43.262089","0.826359911708","0.567379107390","0.999990453135","0.999955813777","0.783684780262","0.757232036109","0.672807954234","0.605320541501","0.506940918144","0.684733049508","0.684712936280","0.000044186223","0.672786622564","0.986795482859","0.942302523776","0.964035907715","0.964035907715"
|
||||
"SWEDISH_LUCENE_SWEDISH_LIGHT_STEM_FILTER","SV_SE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","97881","68","97881","0","1","97881","22338","218126","45941","166437","4789865636","45941","4789911577","0.000959","166437","384563","43.279515","0.826025213298","0.567204853301","0.999990408800","0.999955664954","0.783597631051","0.756945124029","0.672574503184","0.605125951621","0.506675896159","0.684489232882","0.684469049184","0.000044335046","0.672553099274","0.986761366955","0.942264620928","0.963999791833","0.963999791833"
|
||||
"SWEDISH_LUCENE_SWEDISH_MINIMAL_STEM_FILTER","SV_SE","ALL_WORDS","PRIMARY_OUTPUT","12371","98108","68","98108","0","1","98108","23360","228181","40227","157161","4812115209","40227","4812155436","0.000836","157161","385342","40.784809","0.850127417961","0.592151906618","0.999991640544","0.999958984659","0.796071773581","0.781991317186","0.698068068834","0.630412272016","0.536178622033","0.709510092538","0.709491456160","0.000041015341","0.698048215965","0.988492665376","0.944581755622","0.966038479572","0.966038479572"
|
||||
"SWEDISH_LUCENE_SWEDISH_MINIMAL_STEM_FILTER","SV_SE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","97881","68","97881","0","1","97881","23312","227624","40227","156939","4789871350","40227","4789911577","0.000840","156939","384563","40.809698","0.849815755775","0.591903017191","0.999991601724","0.999958840540","0.795947309457","0.781693541131","0.697790053555","0.630152322431","0.535850655618","0.709230928471","0.709212225121","0.000041159460","0.697770131116","0.988462934404","0.944527865197","0.965996098328","0.965996098328"
|
||||
"SWEDISH_RADIXOR","SV_SE","ALL_WORDS","PRIMARY_OUTPUT","12371","98108","68","98108","0","1","98108","12330","365796","24473","19546","4812130963","24473","4812155436","0.000509","19546","385342","5.072377","0.937291970410","0.949276227351","0.999994914337","0.999990853272","0.974635570844","0.939664553041","0.943246034417","0.946854921499","0.892588119029","0.943265066457","0.943260495884","0.000009146728","0.943241460869","0.992630770222","0.993394969179","0.993012722673","0.993012722673"
|
||||
"SWEDISH_RADIXOR","SV_SE","ALL_WORDS","ANY_CANDIDATE","12371","98108","68","92341","5767","5","104148","12371","385342","0","0","4812155436","0","4812155436","0.000000","0","385342","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","ALL_WORDS","ALL_CANDIDATES","12371","98108","68","92341","5767","5","104148","12371","385342","47848","0","4812107588","47848","4812155436","0.000994","0","385342","0.000000","0.889545003347","1.000000000000","0.999990056847","0.999990057643","0.999995028423","0.909639856815","0.941544130223","0.975767741439","0.889545003347","0.943156934634","0.943152245645","0.000009942357","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","97881","68","97881","0","1","97881","12301","365017","24473","19546","4789887104","24473","4789911577","0.000511","19546","384563","5.082652","0.937166551131","0.949173477428","0.999994890720","0.999990810798","0.974584184074","0.939543572972","0.943131801052","0.946747541943","0.892383555482","0.943150907472","0.943146315681","0.000009189202","0.943127206266","0.992611730682","0.993377890892","0.992994663001","0.992994663001"
|
||||
"SWEDISH_RADIXOR","SV_SE","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","12342","97881","68","92114","5767","5","103921","12342","384563","0","0","4789911577","0","4789911577","0.000000","0","384563","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","12342","97881","68","92114","5767","5","103921","12342","384563","47848","0","4789863729","47848","4789911577","0.000999","0","384563","0.000000","0.889346015712","1.000000000000","0.999990010672","0.999990011473","0.999995005336","0.909473386475","0.941432652692","0.975719846569","0.889346015712","0.943051438529","0.943046728292","0.000009988527","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","ALL_WORDS","PRIMARY_OUTPUT","1493","14245","4","14245","0","1","14245","2358","56032","828","9308","101386722","828","101387550","0.000817","9308","65340","14.245485","0.985437917693","0.857545148454","0.999991833317","0.999900091560","0.928768490886","0.956895962839","0.917054009820","0.880397209478","0.846814169992","0.919270093835","0.919222898475","0.000099908440","0.917004266345","0.997989675681","0.970999849348","0.984309781661","0.984309781661"
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","ALL_WORDS","ANY_CANDIDATE","1493","14245","4","12038","2207","6","16937","2912","60394","122","4946","101387428","122","101387550","0.000120","4946","65340","7.569636","0.997984004230","0.924303642485","0.999998796696","0.999950045780","0.962151219591","0.982322936592","0.959731756929","0.938156308641","0.922581039382","0.960437530635","0.960413432420","0.000049954220","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","ALL_WORDS","ALL_CANDIDATES","1493","14245","4","12038","2207","6","16937","2912","60394","1368","4946","101386182","1368","101387550","0.001349","4946","65340","7.569636","0.977850458211","0.924303642485","0.999986507219","0.999937764217","0.962145074852","0.966650447520","0.950323362339","0.934538657225","0.905348683816","0.950700131656","0.950669478973","0.000062235783","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14236","4","14236","0","1","14236","2356","56016","828","9308","101258578","828","101259406","0.000818","9308","65324","14.248974","0.985433818873","0.857510256567","0.999991822982","0.999899965191","0.928751039775","0.956884181756","0.917032283413","0.880367133966","0.846777119361","0.919249480202","0.919202225823","0.000100034809","0.916982477117","0.997988093697","0.970977645923","0.984297603943","0.984297603943"
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14236","4","12029","2207","6","16928","2910","60378","122","4946","101259284","122","101259406","0.000120","4946","65324","7.571490","0.997983471074","0.924285101953","0.999998795174","0.999949982596","0.962141948563","0.982318335047","0.959721515768","0.938140934008","0.922562112276","0.960427641371","0.960403512884","0.000050017404","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14236","4","12029","2207","6","16928","2910","60378","1368","4946","101258038","1368","101259406","0.001351","4946","65324","7.571490","0.977844718686","0.924285101953","0.999986490144","0.999937685499","0.962135796049","0.966641904786","0.950310852286","0.934522445998","0.905325976129","0.950687806541","0.950657115187","0.000062314501","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","ALL_WORDS","PRIMARY_OUTPUT","1493","14245","4","14245","0","1","14245","2365","56016","828","9324","101386722","828","101387550","0.000817","9324","65340","14.269972","0.985433818873","0.857300275482","0.999991833317","0.999899933851","0.928646054399","0.956831877998","0.916912197996","0.880190066750","0.846572361262","0.919136923635","0.919089660097","0.000100066149","0.916862376978","0.997989675681","0.970875945953","0.984246115917","0.984246115917"
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","ALL_WORDS","ANY_CANDIDATE","1493","14245","4","12038","2207","6","16937","2919","60378","122","4962","101387428","122","101387550","0.000120","4962","65340","7.594123","0.997983471074","0.924058769513","0.999998796696","0.999949888071","0.962028783105","0.982267195939","0.959599491418","0.937954390108","0.922336622774","0.960310042786","0.960285871670","0.000050111929","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","ALL_WORDS","ALL_CANDIDATES","1493","14245","4","12038","2207","6","16937","2919","60378","1368","4962","101386182","1368","101387550","0.001349","4962","65340","7.594123","0.977844718686","0.924058769513","0.999986507219","0.999937606509","0.962022638366","0.966592384831","0.950191209102","0.934337338211","0.905108832524","0.950571400540","0.950540673331","0.000062393491","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14236","4","14236","0","1","14236","2356","56016","828","9308","101258578","828","101259406","0.000818","9308","65324","14.248974","0.985433818873","0.857510256567","0.999991822982","0.999899965191","0.928751039775","0.956884181756","0.917032283413","0.880367133966","0.846777119361","0.919249480202","0.919202225823","0.000100034809","0.916982477117","0.997988093697","0.970977645923","0.984297603943","0.984297603943"
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14236","4","12029","2207","6","16928","2910","60378","122","4946","101259284","122","101259406","0.000120","4946","65324","7.571490","0.997983471074","0.924285101953","0.999998795174","0.999949982596","0.962141948563","0.982318335047","0.959721515768","0.938140934008","0.922562112276","0.960427641371","0.960403512884","0.000050017404","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14236","4","12029","2207","6","16928","2910","60378","1368","4946","101258038","1368","101259406","0.001351","4946","65324","7.571490","0.977844718686","0.924285101953","0.999986490144","0.999937685499","0.962135796049","0.966641904786","0.950310852286","0.934522445998","0.905325976129","0.950687806541","0.950657115187","0.000062314501","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","ALL_WORDS","PRIMARY_OUTPUT","1493","14245","4","14245","0","1","14245","1493","64732","880","608","101386670","880","101387550","0.000868","608","65340","0.930517","0.986587819301","0.990694827058","0.999991320433","0.999985333094","0.995343073746","0.987406494442","0.988637057853","0.989870692292","0.977529447297","0.988639190514","0.988631855097","0.000014666906","0.988629719696","0.997993591453","0.998265712624","0.998129633491","0.998129633491"
|
||||
"UKRAINIAN_RADIXOR","UK_UA","ALL_WORDS","ANY_CANDIDATE","1493","14245","4","14055","190","2","14435","1493","65340","0","0","101387550","0","101387550","0.000000","0","65340","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","ALL_WORDS","ALL_CANDIDATES","1493","14245","4","14055","190","2","14435","1493","65340","1490","0","101386060","1490","101387550","0.001470","0","65340","0.000000","0.977704623672","1.000000000000","0.999985303916","0.999985313380","0.999992651958","0.982083809295","0.988726639933","0.995459946982","0.977704623672","0.988789473888","0.988782208195","0.000014686620","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14236","4","14236","0","1","14236","1491","64716","880","608","101258526","880","101259406","0.000869","608","65324","0.930745","0.986584547838","0.990692547915","0.999991309449","0.999985314543","0.995341928682","0.987403420118","0.988634280477","0.989868213355","0.977524016676","0.988636414174","0.988629069474","0.000014685457","0.988626933033","0.997992012551","0.998264347489","0.998128161444","0.998128161444"
|
||||
"UKRAINIAN_RADIXOR","UK_UA","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14236","4","14046","190","2","14426","1491","65324","0","0","101259406","0","101259406","0.000000","0","65324","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14236","4","14046","190","2","14426","1491","65324","1490","0","101257916","1490","101259406","0.001471","0","65324","0.000000","0.977699284581","1.000000000000","0.999985285318","0.999985294804","0.999992642659","0.982079499669","0.988723909852","0.995458840023","0.977699284581","0.988786774073","0.988779499204","0.000014705196","","","","",""
|
||||
"YI_RADIXOR","YI","ALL_WORDS","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","802","6195","195","149","6392714","195","6392909","0.003050","149","6344","2.348676","0.969483568075","0.976513240858","0.999969497454","0.999946243726","0.988241369156","0.970881394183","0.972985707555","0.975099162627","0.947392567671","0.972992055990","0.972965163911","0.000053756274","0.972958803000","0.995691103897","0.996142223728","0.995916612726","0.995916612726"
|
||||
"YI_RADIXOR","YI","ALL_WORDS","ANY_CANDIDATE","802","3578","0","3489","89","3","3676","802","6344","0","0","6392909","0","6392909","0.000000","0","6344","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"YI_RADIXOR","YI","ALL_WORDS","ALL_CANDIDATES","802","3578","0","3489","89","3","3676","802","6344","389","0","6392520","389","6392909","0.006085","0","6344","0.000000","0.942224862617","1.000000000000","0.999939151332","0.999939211655","0.999969575666","0.953239572064","0.970253116158","0.987885016662","0.942224862617","0.970682678643","0.970653145819","0.000060788345","","","","",""
|
||||
"YI_RADIXOR","YI","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3578","0","3578","0","1","3578","802","6195","195","149","6392714","195","6392909","0.003050","149","6344","2.348676","0.969483568075","0.976513240858","0.999969497454","0.999946243726","0.988241369156","0.970881394183","0.972985707555","0.975099162627","0.947392567671","0.972992055990","0.972965163911","0.000053756274","0.972958803000","0.995691103897","0.996142223728","0.995916612726","0.995916612726"
|
||||
"YI_RADIXOR","YI","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","802","3578","0","3489","89","3","3676","802","6344","0","0","6392909","0","6392909","0.000000","0","6344","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"YI_RADIXOR","YI","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","802","3578","0","3489","89","3","3676","802","6344","389","0","6392520","389","6392909","0.006085","0","6344","0.000000","0.942224862617","1.000000000000","0.999939151332","0.999939211655","0.999969575666","0.953239572064","0.970253116158","0.987885016662","0.942224862617","0.970682678643","0.970653145819","0.000060788345","","","","",""
|
||||
Stemmer,Language,Dictionary model ID,Dictionary model version,Dictionary model SHA-256,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,Forms with multiple candidates,Maximum candidates for one form,Total candidate assignments,Distinct output stems,True-positive pairs,False-positive pairs,False-negative pairs,True-negative pairs,Over-stemming error pairs,Over-stemming possible pairs,Over-stemming percentage,Under-stemming error pairs,Under-stemming possible pairs,Under-stemming percentage,Pairwise precision,Pairwise recall,Pairwise specificity,Pairwise accuracy,Balanced accuracy,Pairwise F0.5,Pairwise F1,Pairwise F2,Jaccard index,Fowlkes-Mallows index,Matthews correlation coefficient,Pairwise error rate,Adjusted Rand Index,Homogeneity,Completeness,V-measure,Normalized mutual information
|
||||
"CZECH_LUCENE_CZECH_STEM_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","PRIMARY_OUTPUT","5113","51401","2","51401","0","1","51401","9647","176908","12256","123601","1320692935","12256","1320705191","0.000928","123601","300509","41.130549","0.935209659343","0.588694514973","0.999990720109","0.999897156386","0.794342617541","0.836709501355","0.722555664699","0.635810810811","0.565625949195","0.741992450639","0.741949479040","0.000102843614","","","","",""
|
||||
"CZECH_LUCENE_CZECH_STEM_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50697","2","50697","0","1","50697","9558","174046","11790","123441","1284758279","11790","1284770069","0.000918","123441","297487","41.494586","0.936556964205","0.585054136819","0.999990823261","0.999894767400","0.792522480040","0.836091546082","0.720205742330","0.632533886133","0.562751190680","0.740227347695","0.740183575451","0.000105232600","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","PRIMARY_OUTPUT","5113","51401","2","51401","0","1","51401","5162","298476","0","2033","1320705191","0","1320705191","0.000000","2033","300509","0.676519","1.000000000000","0.993234811603","1.000000000000","0.999998461021","0.996617405801","0.998639599629","0.996605925023","0.994580516517","0.993234811603","0.996611665396","0.996610898340","0.000001538979","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","ANY_CANDIDATE","5113","51401","2","51080","321","4","51739","5166","","","","","0","1320705191","0.000000","0","300509","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","ALL_CANDIDATES","5113","51401","2","51080","321","4","51739","5166","300509","0","0","1320705191","0","1320705191","0.000000","0","300509","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50697","2","50697","0","1","50697","5037","295818","0","1669","1284770069","0","1284770069","0.000000","1669","297487","0.561033","1.000000000000","0.994389670809","1.000000000000","0.999998701236","0.997194835405","0.998872875329","0.997186944320","0.995506694863","0.994389670809","0.997190889855","0.997190242147","0.000001298764","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","5038","50697","2","50428","269","4","50975","5040","","","","","0","1284770069","0.000000","0","297487","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"CZECH_RADIXOR","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","5038","50697","2","50428","269","4","50975","5040","297487","0","0","1284770069","0","1284770069","0.000000","0","297487","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","ALL_WORDS","PRIMARY_OUTPUT","4179","27921","32","27921","0","1","27921","4184","89021","0","674","389687465","0","389687465","0.000000","674","89695","0.751435","1.000000000000","0.992485645800","1.000000000000","0.999998270807","0.996242822900","0.998488040038","0.996228653282","0.993979468559","0.992485645800","0.996235738066","0.996234876527","0.000001729193","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","ALL_WORDS","ANY_CANDIDATE","4179","27921","32","27756","165","3","28087","4187","","","","","0","389687465","0.000000","0","89695","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","ALL_WORDS","ALL_CANDIDATES","4179","27921","32","27756","165","3","28087","4187","89695","0","0","389687465","0","389687465","0.000000","0","89695","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","27875","32","27875","0","1","27875","4170","88910","0","630","388404335","0","388404335","0.000000","630","89540","0.703596","1.000000000000","0.992964038419","1.000000000000","0.999998378353","0.996482019209","0.998584842086","0.996469599328","0.994363298812","0.992964038419","0.996475809249","0.996475001098","0.000001621647","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4173","27875","32","27718","157","3","28033","4173","","","","","0","388404335","0.000000","0","89540","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"DA_DK_RADIXOR","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4173","27875","32","27718","157","3","28033","4173","89540","0","0","388404335","0","388404335","0.000000","0","89540","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"ENGLISH_LUCENE_KSTEM_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","371125","237301","193361","76054","175199230769","193361","175199424130","0.000110","76054","313355","24.270875","0.551014484677","0.757291251137","0.999998896338","0.999998462241","0.878645073737","0.582761911451","0.637891338504","0.704541109043","0.468311638077","0.645970934714","0.645970209547","0.000001537759","","","","",""
|
||||
"ENGLISH_LUCENE_KSTEM_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","347624","237291","193354","74091","161561796284","193354","161561989638","0.000120","74091","311382","23.794246","0.551013015361","0.762057537045","0.999998803221","0.999998344632","0.881028170133","0.583322107296","0.639575109801","0.707835647036","0.470128938693","0.647999707843","0.647998929360","0.000001655368","","","","",""
|
||||
"ENGLISH_LUCENE_MINIMAL_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","453328","137223","1401","176132","175199422729","1401","175199424130","0.000001","176132","313355","56.208454","0.989893524931","0.437915463292","0.999999992003","0.999998986682","0.718957727648","0.790590781136","0.607209626996","0.492883127257","0.435966272287","0.658399332913","0.658398992571","0.000001013318","","","","",""
|
||||
"ENGLISH_LUCENE_MINIMAL_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","430129","136930","1388","174452","161561988250","1388","161561989638","0.000001","174452","311382","56.025075","0.989965152764","0.439749246906","0.999999991409","0.999998911627","0.719874619157","0.791819618021","0.608983766956","0.494744357392","0.437797742750","0.659800295840","0.659799929564","0.000001088373","","","","",""
|
||||
"ENGLISH_LUCENE_PORTER_COPIED","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","319968","285026","362583","28329","175199061547","362583","175199424130","0.000207","28329","313355","9.040545","0.440120504811","0.909594549313","0.999997930456","0.999997768764","0.954796239884","0.490782566652","0.593208486478","0.749662419669","0.421674768988","0.632717324100","0.632716488082","0.000002231236","","","","",""
|
||||
"ENGLISH_LUCENE_PORTER_COPIED","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","298779","283398","359344","27984","161561630294","359344","161561989638","0.000222","27984","311382","8.987032","0.440920307059","0.910129679943","0.999997775813","0.999997602609","0.955063727878","0.491609277152","0.594048572303","0.750417048409","0.422524249843","0.633478222155","0.633477323677","0.000002397391","","","","",""
|
||||
"ENGLISH_LUCENE_PORTER_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","319968","285026","362583","28329","175199061547","362583","175199424130","0.000207","28329","313355","9.040545","0.440120504811","0.909594549313","0.999997930456","0.999997768764","0.954796239884","0.490782566652","0.593208486478","0.749662419669","0.421674768988","0.632717324100","0.632716488082","0.000002231236","","","","",""
|
||||
"ENGLISH_LUCENE_PORTER_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","298779","283398","359344","27984","161561630294","359344","161561989638","0.000222","27984","311382","8.987032","0.440920307059","0.910129679943","0.999997775813","0.999997602609","0.955063727878","0.491609277152","0.594048572303","0.750417048409","0.422524249843","0.633478222155","0.633477323677","0.000002397391","","","","",""
|
||||
"ENGLISH_LUCENE_POSSESSIVE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","591899","7","40","313348","175199424090","40","175199424130","0.000000","313348","313355","99.997766","0.148936170213","0.000022338881","0.999999999772","0.999998211253","0.500011169326","0.000111627432","0.000044671061","0.000027922554","0.000022336030","0.001824025043","0.001824004770","0.000001788747","","","","",""
|
||||
"ENGLISH_LUCENE_POSSESSIVE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","568400","5","36","311377","161561989602","36","161561989638","0.000000","311377","311382","99.998394","0.121951219512","0.000016057447","0.999999999777","0.999998072490","0.500008028612","0.000080244972","0.000032110666","0.000020071148","0.000016055591","0.001399366020","0.001399345253","0.000001927510","","","","",""
|
||||
"ENGLISH_OPENNLP_PORTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","319968","285026","362583","28329","175199061547","362583","175199424130","0.000207","28329","313355","9.040545","0.440120504811","0.909594549313","0.999997930456","0.999997768764","0.954796239884","0.490782566652","0.593208486478","0.749662419669","0.421674768988","0.632717324100","0.632716488082","0.000002231236","","","","",""
|
||||
"ENGLISH_OPENNLP_PORTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","298779","283398","359344","27984","161561630294","359344","161561989638","0.000222","27984","311382","8.987032","0.440920307059","0.910129679943","0.999997775813","0.999997602609","0.955063727878","0.491609277152","0.594048572303","0.750417048409","0.422524249843","0.633478222155","0.633477323677","0.000002397391","","","","",""
|
||||
"ENGLISH_PAICE_HUSK_LANCASTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","268169","283611","1682034","29744","175197742096","1682034","175199424130","0.000960","29744","313355","9.492110","0.144283937334","0.905078904118","0.999990399318","0.999990229563","0.952534651718","0.173442548161","0.248890741553","0.440517665844","0.142133188065","0.361370098215","0.361367968977","0.000009770437","","","","",""
|
||||
"ENGLISH_PAICE_HUSK_LANCASTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","249411","282022","1666990","29360","161560322648","1666990","161561989638","0.001032","29360","311382","9.428933","0.144699981324","0.905710670495","0.999989682041","0.999989500335","0.952850176268","0.173928112855","0.249533488410","0.441412535138","0.142552563421","0.362017012177","0.362014722775","0.000010499665","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","390361","291757","3","21598","175199424127","3","175199424130","0.000000","21598","313355","6.892502","0.999989717576","0.931074978858","0.999999999983","0.999999876706","0.965537489420","0.985402544591","0.964302653215","0.944087420236","0.931066065012","0.964917304825","0.964917245339","0.000000123294","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","ANY_CANDIDATE","396939","591946","250964","578228","13718","1355","607918","397392","","","","","0","175199424130","0.000000","15","313355","0.004787","","","","","","","","","","","","","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","ALL_CANDIDATES","396939","591946","250964","578228","13718","1355","607918","397392","313340","55","15","175199424075","55","175199424130","0.000000","15","313355","0.004787","0.999824502624","0.999952130970","0.999999999686","0.999999999600","0.999976065328","0.999850025687","0.999888312724","0.999926602694","0.999776650394","0.999888314761","0.999888314561","0.000000000400","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","367590","290334","3","21048","161561989635","3","161561989638","0.000000","21048","311382","6.759543","0.999989667180","0.932404570592","0.999999999981","0.999999869704","0.966202285287","0.985700026481","0.965015231362","0.945180728775","0.932395587456","0.965605994297","0.965605931388","0.000000130296","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","374384","568441","228735","555084","13357","1355","584042","374506","","","","","0","161561989638","0.000000","0","311382","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"ENGLISH_RADIXOR","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","374384","568441","228735","555084","13357","1355","584042","374506","311382","15","0","161561989623","15","161561989638","0.000000","0","311382","0.000000","0.999951829979","1.000000000000","0.999999999907","0.999999999907","0.999999999954","0.999961463612","0.999975914409","0.999990365625","0.999951829979","0.999975914699","0.999975914653","0.000000000093","","","","",""
|
||||
"ENGLISH_SNOWBALL_ORIGINAL_PORTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","321092","284940","360538","28415","175199063592","360538","175199424130","0.000206","28415","313355","9.067990","0.441440296958","0.909320100206","0.999997942128","0.999997779945","0.954659021167","0.492078968883","0.594347503684","0.750277266077","0.422826769235","0.633569676567","0.633568843266","0.000002220055","","","","",""
|
||||
"ENGLISH_SNOWBALL_ORIGINAL_PORTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","299877","283312","357325","28070","161561632313","357325","161561989638","0.000221","28070","311382","9.014651","0.442234838138","0.909853491852","0.999997788310","0.999997614573","0.954925640081","0.492899966248","0.595181398691","0.751026553880","0.423671353822","0.634325556555","0.634324660984","0.000002385427","","","","",""
|
||||
"ENGLISH_SNOWBALL_PORTER2","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","318385","284971","371381","28384","175199052749","371381","175199424130","0.000212","28384","313355","9.058097","0.434174040759","0.909419029535","0.999997880238","0.999997718233","0.954708454887","0.484848557029","0.587746607996","0.746086443827","0.416176453407","0.628367833992","0.628366984426","0.000002281767","","","","",""
|
||||
"ENGLISH_SNOWBALL_PORTER2","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","297220","283368","368027","28014","161561621611","368027","161561989638","0.000228","28014","311382","8.996666","0.435017155489","0.910033335260","0.999997722069","0.999997548679","0.955015528665","0.485724531207","0.588647215295","0.746914872138","0.417080138768","0.629190045142","0.629189132274","0.000002451321","","","","",""
|
||||
"FINNISH_LUCENE_FINNISH_LIGHT_STEM_FILTER","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","PRIMARY_OUTPUT","57027","1788784","292","1788784","0","1","1788784","439975","12317229","1508153","19148370","1599840231184","1508153","1599841739337","0.000094","19148370","31465599","60.854936","0.890914189568","0.391450644242","0.999999057311","0.999987088650","0.695724850776","0.709786610775","0.543915310644","0.440884276934","0.373546480243","0.590549687554","0.590545009664","0.000012911350","","","","",""
|
||||
"FINNISH_LUCENE_FINNISH_LIGHT_STEM_FILTER","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1734784","274","1734784","0","1","1734784","431848","11954192","1155011","18806691","1504704980042","1155011","1504706135053","0.000077","18806691","30760883","61.138333","0.911893118140","0.388616672675","0.999999232401","0.999986734091","0.694307952538","0.718420864905","0.544981470973","0.438999334093","0.374552942180","0.595295615141","0.595290947645","0.000013265909","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","PRIMARY_OUTPUT","57027","1788784","292","1788784","0","1","1788784","69091","30511413","804","954186","1599841738533","804","1599841739337","0.000000","954186","31465599","3.032474","0.999973649899","0.969675263452","0.999999999497","0.999999403084","0.984837631475","0.993763441201","0.984591422195","0.975587162604","0.969650487220","0.984707932542","0.984707638627","0.000000596916","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","ANY_CANDIDATE","57027","1788784","292","1754389","34395","6","1826768","69769","","","","","0","1599841739337","0.000000","0","31465599","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","ALL_CANDIDATES","57027","1788784","292","1754389","34395","6","1826768","69769","31465599","2327","0","1599841737010","2327","1599841739337","0.000000","0","31465599","0.000000","0.999926051688","1.000000000000","0.999999998545","0.999999998546","0.999999999273","0.999940840476","0.999963024477","0.999985209463","0.999926051688","0.999963025161","0.999963024433","0.000000001454","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1734784","274","1734784","0","1","1734784","54633","30037514","804","723369","1504706134249","804","1504706135053","0.000000","723369","30760883","2.351587","0.999973234187","0.976484127585","0.999999999466","0.999999518738","0.988242063525","0.995185441684","0.988089103342","0.981093251747","0.976458605798","0.988158889650","0.988158651850","0.000000481262","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","54762","1734784","274","1712724","22060","6","1758300","54984","","","","","0","1504706135053","0.000000","0","30760883","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"FINNISH_RADIXOR","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","54762","1734784","274","1712724","22060","6","1758300","54984","30760883","2235","0","1504706132818","2235","1504706135053","0.000000","0","30760883","0.000000","0.999927348067","1.000000000000","0.999999998515","0.999999998515","0.999999999257","0.999941877609","0.999963672714","0.999985468769","0.999927348067","0.999963673373","0.999963672631","0.000000001485","","","","",""
|
||||
"FRENCH_LUCENE_FRENCH_LIGHT_STEM_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","245918","198474","152794","5171725","81606719062","152794","81606871856","0.000187","5171725","5370199","96.304159","0.565021578965","0.036958406942","0.999998127682","0.999934758330","0.518478267312","0.146469417976","0.069378710041","0.045454703687","0.035935949946","0.144507084415","0.144495320409","0.000065241670","","","","",""
|
||||
"FRENCH_LUCENE_FRENCH_LIGHT_STEM_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","245182","196458","145140","5159793","80279351725","145140","80279496865","0.000181","5159793","5356251","96.332173","0.575114608399","0.036678266198","0.999998192066","0.999933923613","0.518338229132","0.146116638947","0.068958654397","0.045128311714","0.035710604827","0.145238447737","0.145226752971","0.000066076387","","","","",""
|
||||
"FRENCH_LUCENE_FRENCH_MINIMAL_STEM_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","269236","180266","67902","5189933","81606803954","67902","81606871856","0.000083","5189933","5370199","96.643216","0.726386963670","0.033567843575","0.999999167938","0.999935575413","0.516783505756","0.141654608431","0.064170247333","0.041480578641","0.033148703932","0.156151349567","0.156142578924","0.000064424587","","","","",""
|
||||
"FRENCH_LUCENE_FRENCH_MINIMAL_STEM_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","268411","178414","60724","5177837","80279436141","60724","80279496865","0.000076","5177837","5356251","96.669051","0.746071306108","0.033309492031","0.999999243593","0.999934750320","0.516654367812","0.141311236863","0.063771794955","0.041177259639","0.032936094407","0.157642812159","0.157634208515","0.000065249680","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","60225","4925833","29","444366","81606871827","29","81606871856","0.000000","444366","5370199","8.274665","0.999994112706","0.917253345733","0.999999999645","0.999994554800","0.958626672689","0.982272941786","0.956838348180","0.932687695656","0.917248392433","0.957730622666","0.957728014957","0.000005445200","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","ANY_CANDIDATE","59240","404011","2301","382167","21844","56","427440","60383","","","","","0","81606871856","0.000000","232","5370199","0.004320","","","","","","","","","","","","","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","ALL_CANDIDATES","59240","404011","2301","382167","21844","56","427440","60383","5369967","2303","232","81606869553","2303","81606871856","0.000003","232","5370199","0.004320","0.999571317153","0.999956798621","0.999999971779","0.999999968938","0.999978385200","0.999648389668","0.999764020729","0.999879678544","0.999528152805","0.999764039308","0.999764023779","0.000000031062","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","58069","4915501","1","440750","80279496864","1","80279496865","0.000000","440750","5356251","8.228703","0.999999796562","0.917712967521","0.999999999988","0.999994510160","0.958856483755","0.982382706966","0.957090965875","0.933068825633","0.917712796187","0.957973267280","0.957970637555","0.000005489840","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","57698","400712","2133","380101","20611","56","422336","58208","","","","","0","80279496865","0.000000","0","5356251","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"FRENCH_RADIXOR","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","57698","400712","2133","380101","20611","56","422336","58208","5356251","75","0","80279496790","75","80279496865","0.000000","0","5356251","0.000000","0.999985997865","1.000000000000","0.999999999066","0.999999999066","0.999999999533","0.999988798261","0.999992998883","0.999997199542","0.999985997865","0.999992998908","0.999992998441","0.000000000934","","","","",""
|
||||
"GERMAN_CISTEM","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","59097","1018135","258954","326717","38436474939","258954","38436733893","0.000674","326717","1344852","24.293900","0.797231046544","0.757061000021","0.999993262851","0.999984763260","0.878527131436","0.788859587356","0.776626934016","0.764767865140","0.634824286728","0.776886435294","0.776878836909","0.000015236740","","","","",""
|
||||
"GERMAN_CISTEM","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","23023","712025","86055","146392","10594877479","86055","10594963534","0.000812","146392","858417","17.053716","0.892172463913","0.829462836826","0.999991877745","0.999978062391","0.914727357285","0.878883274821","0.859675568383","0.841289462416","0.753886827773","0.860246419845","0.860235546687","0.000021937609","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","98357","682737","73514","662115","38436660379","73514","38436733893","0.000191","662115","1344852","49.233299","0.902791533499","0.507667014660","0.999998087403","0.999980861973","0.753832551031","0.781189357269","0.649884370257","0.556368109766","0.481354600999","0.676991493796","0.676983759805","0.000019138027","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","50335","461774","13791","396643","10594949743","13791","10594963534","0.000130","396643","858417","46.206331","0.971000809563","0.537936690443","0.999998698344","0.999961264544","0.768967694393","0.836341955252","0.692324184284","0.590619694452","0.529431053143","0.722728830139","0.722714023718","0.000038735456","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","140505","257534","33845","1087318","38436700048","33845","38436733893","0.000088","1087318","1344852","80.850384","0.883845438415","0.191496164634","0.999999119462","0.999970831971","0.595747642048","0.512940732195","0.314789293199","0.227070775185","0.186795213161","0.411403708765","0.411396179319","0.000029168029","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","80363","128762","4742","729655","10594958792","4742","10594963534","0.000045","729655","858417","85.000064","0.964480465005","0.149999359286","0.999999552429","0.999930689945","0.574999455857","0.462363359673","0.259621481953","0.180481905554","0.149175296788","0.380357005712","0.380342859388","0.000069310055","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","81085","594410","170297","750442","38436563596","170297","38436733893","0.000443","750442","1344852","55.801084","0.777304248555","0.441989155684","0.999995569421","0.999976046175","0.720992362552","0.674901446063","0.563539583392","0.483723042293","0.392311251237","0.586139956434","0.586129657503","0.000023953825","","","","",""
|
||||
"GERMAN_LUCENE_GERMAN_STEM_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","41574","371092","37962","487325","10594925572","37962","10594963534","0.000358","487325","858417","56.770194","0.907195626983","0.432298055607","0.999996416977","0.999950425082","0.716147236292","0.743780748511","0.585562904398","0.482850437789","0.413990064470","0.626241890646","0.626223420324","0.000049574918","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","68104","1103976","663","240876","38436733230","663","38436733893","0.000002","240876","1344852","17.910967","0.999399803918","0.820890328452","0.999999982751","0.999993716153","0.910445155602","0.957745833715","0.901392166781","0.851301663915","0.820485836278","0.905758043461","0.905755194143","0.000006283847","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","ANY_CANDIDATE","54092","277266","1474","248231","29035","8","313927","70717","","","","","502","38436733893","0.000001","111107","1344852","8.261653","","","","","","","","","","","","","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","ALL_CANDIDATES","54092","277266","1474","248231","29035","8","313927","70717","1233745","6862","111107","38436727031","6862","38436733893","0.000018","111107","1344852","8.261653","0.994468836626","0.917383474167","0.999999821473","0.999996930934","0.958691647820","0.978032527492","0.954372125027","0.931829459601","0.912726360754","0.955148824160","0.955147343048","0.000003069066","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","17264","801691","80","56726","10594963454","80","10594963534","0.000001","56726","858417","6.608210","0.999900220886","0.933917897712","0.999999992449","0.999994638830","0.966958945080","0.985968273037","0.965783393206","0.946408388835","0.933830869531","0.966346062346","0.966343471257","0.000005361170","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","16007","145574","228","135120","10454","8","157137","18366","","","","","0","10594963534","0.000000","0","858417","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"GERMAN_RADIXOR","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","16007","145574","228","135120","10454","8","157137","18366","858417","1490","0","10594962044","1490","10594963534","0.000014","0","858417","0.000000","0.998267254482","1.000000000000","0.999999859367","0.999999859379","0.999999929684","0.998613323034","0.999132875988","0.999652969844","0.998267254482","0.999133251615","0.999133181359","0.000000140621","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","ALL_WORDS","PRIMARY_OUTPUT","2358","57658","0","57658","0","1","57658","2358","685765","0","19645","1661488243","0","1661488243","0.000000","19645","705410","2.784905","1.000000000000","0.972150947676","1.000000000000","0.999988181281","0.986075473838","0.994303270726","0.985878843424","0.977595971951","0.972150947676","0.985977153729","0.985971324814","0.000011818719","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","ALL_WORDS","ANY_CANDIDATE","2358","57658","0","56674","984","40","58714","2358","","","","","0","1661488243","0.000000","0","705410","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","ALL_WORDS","ALL_CANDIDATES","2358","57658","0","56674","984","40","58714","2358","705410","0","0","1661488243","0","1661488243","0.000000","0","705410","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","2358","57658","0","57658","0","1","57658","2358","685765","0","19645","1661488243","0","1661488243","0.000000","19645","705410","2.784905","1.000000000000","0.972150947676","1.000000000000","0.999988181281","0.986075473838","0.994303270726","0.985878843424","0.977595971951","0.972150947676","0.985977153729","0.985971324814","0.000011818719","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","2358","57658","0","56674","984","40","58714","2358","","","","","0","1661488243","0.000000","0","705410","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"HE_IL_RADIXOR","HE_IL","he-il-default","1.0.0","9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","2358","57658","0","56674","984","40","58714","2358","705410","0","0","1661488243","0","1661488243","0.000000","0","705410","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"HUNGARIAN_LUCENE_HUNGARIAN_LIGHT_STEM_FILTER","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","PRIMARY_OUTPUT","19406","910688","1","910688","0","1","910688","94328","14021483","3795942","8096372","414649947531","3795942","414653743473","0.000915","8096372","22117855","36.605593","0.786953389729","0.633944069169","0.999990845514","0.999971321422","0.816967457342","0.750714749947","0.702210326308","0.659593346790","0.541081764282","0.706317516512","0.706303654339","0.000028678578","","","","",""
|
||||
"HUNGARIAN_LUCENE_HUNGARIAN_LIGHT_STEM_FILTER","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","872878","1","872878","0","1","872878","91516","13478053","3311675","7889264","380932886011","3311675","380936197686","0.000869","7889264","21367317","36.922109","0.802755887409","0.630778913422","0.999991306484","0.999970597935","0.815385109953","0.761246308142","0.706451613326","0.659015515857","0.546134663847","0.711590813883","0.711576731219","0.000029402065","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","PRIMARY_OUTPUT","19406","910688","1","910688","0","1","910688","20535","21921219","39","196636","414653743434","39","414653743473","0.000000","196636","22117855","0.889037","0.999998220905","0.991109626137","0.999999999906","0.999999525714","0.995554813021","0.998207770257","0.995534083532","0.992874681417","0.991107878535","0.995544003477","0.995543767377","0.000000474286","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","ANY_CANDIDATE","19406","910688","1","904024","6664","5","917595","20567","","","","","0","414653743473","0.000000","0","22117855","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","ALL_CANDIDATES","19406","910688","1","904024","6664","5","917595","20567","22117855","192","0","414653743281","192","414653743473","0.000000","0","22117855","0.000000","0.999991319306","1.000000000000","0.999999999537","0.999999999537","0.999999999768","0.999993055433","0.999995659634","0.999998263849","0.999991319306","0.999995659644","0.999995659412","0.000000000463","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","872878","1","872878","0","1","872878","18363","21206087","39","161230","380936197647","39","380936197686","0.000000","161230","21367317","0.754564","0.999998160909","0.992454363831","0.999999999898","0.999999576675","0.996227181864","0.998480240771","0.996211981258","0.993954004051","0.992452552389","0.996219121788","0.996218910913","0.000000423325","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","18360","872878","1","867360","5518","5","878574","18375","","","","","0","380936197686","0.000000","0","21367317","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"HUNGARIAN_RADIXOR","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","18360","872878","1","867360","5518","5","878574","18375","21367317","192","0","380936197494","192","380936197686","0.000000","0","21367317","0.000000","0.999991014395","1.000000000000","0.999999999496","0.999999999496","0.999999999748","0.999992811503","0.999995507177","0.999998202866","0.999991014395","0.999995507187","0.999995506935","0.000000000504","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","PRIMARY_OUTPUT","5113","51401","2","51401","0","1","51401","10920","212842","9128","87667","1320696063","9128","1320705191","0.000691","87667","300509","29.172837","0.958877325765","0.708271632464","0.999993088541","0.999926726281","0.854132360502","0.895506437707","0.814738965585","0.747335334261","0.687392010645","0.824102911566","0.824070367475","0.000073273719","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","ANY_CANDIDATE","5113","51401","2","48207","3194","5","55179","11359","","","","","8582","1320705191","0.000650","76964","300509","25.611213","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","ALL_WORDS","ALL_CANDIDATES","5113","51401","2","48207","3194","5","55179","11359","223545","10775","76964","1320694416","10775","1320705191","0.000816","76964","300509","25.611213","0.954015875726","0.743887870247","0.999991841480","0.999933581664","0.871939855863","0.903001238499","0.835949434305","0.778167111775","0.718138420221","0.842425568211","0.842395220341","0.000066418336","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","5038","50697","2","50697","0","1","50697","10816","210117","8993","87370","1284761076","8993","1284770069","0.000700","87370","297487","29.369351","0.958956688421","0.706306494065","0.999993000304","0.999925013281","0.853149747185","0.894932138029","0.813465815713","0.745593864837","0.685581440877","0.822992914040","0.822959656132","0.000074986719","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","5038","50697","2","47580","3117","5","54394","11240","","","","","8518","1284770069","0.000663","76872","297487","25.840457","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_CZECH_LUCENE_FILTER","CS_CZ","cs-cz-default","1.0.0","62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","5038","50697","2","47580","3117","5","54394","11240","220615","10523","76872","1284759546","10523","1284770069","0.000819","76872","297487","25.840457","0.954473085343","0.741595431061","0.999991809429","0.999931991902","0.870793620245","0.902651224715","0.834674864034","0.776219736174","0.716259212363","0.841328044915","0.841297029699","0.000068008098","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","PRIMARY_OUTPUT","4992","26201","85","26201","0","1","26201","15909","18409","356","46028","343168307","356","343168663","0.000104","46028","64437","71.431010","0.981028510525","0.285689898661","0.999998962609","0.999864861518","0.642844430635","0.659834978530","0.442513401120","0.332877658555","0.284120198170","0.529405266082","0.529368118333","0.000135138482","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","ANY_CANDIDATE","4992","26201","85","25002","1199","3","27429","16027","","","","","330","343168663","0.000096","43157","64437","66.975495","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","ALL_CANDIDATES","4992","26201","85","25002","1199","3","27429","16027","21280","503","43157","343168160","503","343168663","0.000147","43157","64437","66.975495","0.976908598448","0.330245045548","0.999998534248","0.999872797816","0.665121789898","0.701990512572","0.493620969613","0.380637567926","0.327687095781","0.567995796279","0.567957979352","0.000127202184","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25402","84","25402","0","1","25402","15258","18260","333","44758","322554750","333","322555083","0.000103","44758","63018","71.024152","0.982090033884","0.289758481704","0.999998967618","0.999860234129","0.644878724661","0.664531625300","0.447488696377","0.337317348013","0.288235386971","0.533450013698","0.533411381379","0.000139765871","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4796","25402","84","24271","1131","3","26562","15323","","","","","307","322555083","0.000095","41900","63018","66.488940","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4796","25402","84","24271","1131","3","26562","15323","21118","477","41900","322554606","477","322555083","0.000148","41900","63018","66.488940","0.977911553600","0.335110603320","0.999998521183","0.999868646552","0.667554562251","0.706769836276","0.499166794701","0.385833878400","0.332593117568","0.572458322256","0.572419041773","0.000131353448","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","PRIMARY_OUTPUT","396939","591946","250964","591946","0","1","591946","557518","45837","21444","267518","175199402686","21444","175199424130","0.000012","267518","313355","85.372182","0.681277032149","0.146278182892","0.999999877602","0.999998350671","0.573139030247","0.393464828775","0.240844271167","0.173532843543","0.136909011078","0.315683332326","0.315682846485","0.000001649329","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","ANY_CANDIDATE","396939","591946","250964","586210","5736","4","597698","557638","","","","","20367","175199424130","0.000012","262339","313355","83.719424","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","ALL_WORDS","ALL_CANDIDATES","396939","591946","250964","586210","5736","4","597698","557638","51016","38780","262339","175199385350","38780","175199424130","0.000022","262339","313355","83.719424","0.568132210789","0.162805763431","0.999999778652","0.999998281282","0.581402771042","0.379279119873","0.253086312573","0.189902443092","0.144876254845","0.304130232478","0.304129624950","0.000001718718","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","374384","568441","228735","568441","0","1","568441","535362","45763","19600","265619","161561970038","19600","161561989638","0.000012","265619","311382","85.303261","0.700136162661","0.146967390536","0.999999878684","0.999998234619","0.573483634610","0.399443817930","0.242938857848","0.174549218814","0.138264316489","0.320775910639","0.320775401621","0.000001765381","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","374384","568441","228735","562756","5685","4","574142","535485","","","","","18564","161561989638","0.000011","260443","311382","83.640994","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_ENGLISH_LUCENE_FILTER","US_UK","us-uk-default","1.0.0","8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","374384","568441","228735","562756","5685","4","574142","535485","50939","36611","260443","161561953027","36611","161561989638","0.000023","260443","311382","83.640994","0.581827527127","0.163590059798","0.999999773393","0.999998161366","0.581794916596","0.384978732795","0.255376856206","0.191057837576","0.146379381194","0.308514505258","0.308513864190","0.000001838634","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","154336","3402849","440809","1967350","81606431047","440809","81606871856","0.000540","1967350","5370199","36.634583","0.885315238765","0.633654171847","0.999994598384","0.999970492674","0.816824385116","0.820167925205","0.738637250394","0.671850417782","0.585586700276","0.748988447470","0.748975057537","0.000029507326","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","ANY_CANDIDATE","59240","404011","2301","395781","8230","4","412364","154718","","","","","439665","81606871856","0.000539","1782362","5370199","33.189869","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","ALL_CANDIDATES","59240","404011","2301","395781","8230","4","412364","154718","3587837","500695","1782362","81606371161","500695","81606871856","0.000614","1782362","5370199","33.189869","0.877536729565","0.668101312447","0.999993864549","0.999972025557","0.834047588498","0.825764821161","0.758629672416","0.701590006589","0.611122769377","0.765691478823","0.765678434549","0.000027974443","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","153822","3392703","433354","1963548","80279063511","433354","80279496865","0.000540","1963548","5356251","36.658999","0.886736135923","0.633410010098","0.999994601934","0.999970145029","0.816702306016","0.821061070269","0.738965192629","0.671794147581","0.585999044840","0.749444824393","0.749431295094","0.000029854971","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","57698","400712","2133","392518","8194","4","409028","154205","","","","","432307","80279496865","0.000539","1778903","5356251","33.211718","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","57698","400712","2133","392518","8194","4","409028","154205","3577348","492522","1778903","80279004343","492522","80279496865","0.000614","1778903","5356251","33.211718","0.878983358191","0.667882815798","0.999993864909","0.999971707926","0.833938340354","0.826722240168","0.759028660888","0.701581816015","0.611640766362","0.766197024471","0.766183847721","0.000028292074","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","182774","377391","111635","967461","38436622258","111635","38436733893","0.000290","967461","1344852","71.938102","0.771719704065","0.280618982609","0.999997095617","0.999971926380","0.640308039113","0.571638943385","0.411576996943","0.321543191932","0.259110448634","0.465359214171","0.465349217076","0.000028073620","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","ANY_CANDIDATE","54092","277266","1474","270784","6482","3","283881","183111","","","","","83073","38436733893","0.000216","952309","1344852","70.811435","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","ALL_CANDIDATES","54092","277266","1474","270784","6482","3","283881","183111","392543","135961","952309","38436597932","135961","38436733893","0.000354","952309","1344852","70.811435","0.742743668922","0.291885649871","0.999996462733","0.999971687711","0.645941056302","0.567444319934","0.419079982662","0.332218049287","0.265086138493","0.465613808312","0.465603221302","0.000028312289","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","86983","273967","58904","584450","10594904630","58904","10594963534","0.000556","584450","858417","68.084626","0.823042560031","0.319153744625","0.999994440377","0.999939282294","0.659574092501","0.625523710889","0.459950910275","0.363685335530","0.298659902041","0.512520355713","0.512498716988","0.000060717706","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","16007","145574","228","141036","4538","3","150205","87393","","","","","40608","10594963534","0.000383","573996","858417","66.866802","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","16007","145574","228","141036","4538","3","150205","87393","284421","66639","573996","10594896895","66639","10594963534","0.000629","573996","858417","66.866802","0.810177747394","0.331331975019","0.999993710313","0.999939538905","0.665662842666","0.628511082325","0.470320642724","0.375748270417","0.307463548153","0.518109827315","0.518087587435","0.000060461095","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","PRIMARY_OUTPUT","9990","120867","1","120867","0","1","120867","18419","968411","27967","148662","7303210371","27967","7303238338","0.000383","148662","1117073","13.308172","0.971931335296","0.866918276603","0.999996170603","0.999975818674","0.933457223603","0.948941565893","0.916426262071","0.886065398277","0.845744253476","0.917924309609","0.917912670119","0.000024181326","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ANY_CANDIDATE","9990","120867","1","110382","10485","6","132492","19068","","","","","25967","7303238338","0.000356","80738","1117073","7.227639","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ALL_CANDIDATES","9990","120867","1","110382","10485","6","132492","19068","1036335","44498","80738","7303193840","44498","7303238338","0.000609","80738","1117073","7.227639","0.958829902492","0.927723613408","0.999993907086","0.999982854613","0.963858760247","0.952442878793","0.943020311151","0.933782353074","0.892183947430","0.943148525834","0.943139991603","0.000017145387","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","119451","1","119451","0","1","119451","18149","963133","27267","147624","7133072951","27267","7133100218","0.000382","147624","1110757","13.290396","0.972468699515","0.867096043509","0.999996177398","0.999975485586","0.933546110454","0.949393940529","0.916764430264","0.886303269317","0.846320464243","0.918272161065","0.918260365969","0.000024514414","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","119451","1","109148","10303","6","130856","18789","","","","","25425","7133100218","0.000356","80363","1110757","7.234976","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_POLISH_LUCENE_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","119451","1","109148","10303","6","130856","18789","1030394","43630","80363","7133056588","43630","7133100218","0.000612","80363","1110757","7.234976","0.959377071648","0.927650242132","0.999993883445","0.999982619942","0.963822062789","0.952859269523","0.943246943286","0.933826616099","0.892589746766","0.943380290663","0.943371641374","0.000017380058","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","495840","9648381","244539","32290659","360919299051","244539","360919543590","0.000068","32290659","41939040","76.994273","0.975281413374","0.230057268836","0.999999322456","0.999909865181","0.615028295646","0.591847366825","0.372294661441","0.271557302745","0.228723622526","0.473677715654","0.473655293112","0.000090134819","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","ANY_CANDIDATE","65059","849661","3589","838352","11309","5","861853","496361","","","","","223500","360919543590","0.000062","31877837","41939040","76.009935","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","ALL_CANDIDATES","65059","849661","3589","838352","11309","5","861853","496361","10061203","263629","31877837","360919279961","263629","360919543590","0.000073","31877837","41939040","76.009935","0.974466509479","0.239900651040","0.999999269563","0.999910955967","0.619949960302","0.604360900012","0.385015599303","0.282489525889","0.238402054619","0.483502998999","0.483480352448","0.000089044033","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","495045","9614637","244260","32215829","359406900655","244260","359407144915","0.000068","32215829","41830466","77.015229","0.975224408978","0.229847714343","0.999999320381","0.999909694863","0.614923517362","0.591553085622","0.372016076112","0.271322827200","0.228513359778","0.473448097868","0.473425641311","0.000090305137","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","64918","847879","3525","836592","11287","5","860048","495572","","","","","223274","359407144915","0.000062","31806834","41830466","76.037484","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","64918","847879","3525","836592","11287","5","860048","495572","10023632","263289","31806834","359406881626","263289","359407144915","0.000073","31806834","41830466","76.037484","0.974405461070","0.239625157415","0.999999267435","0.999910779762","0.619812212425","0.603992255793","0.384655969034","0.282182888645","0.238126344395","0.483210163382","0.483187483018","0.000089220238","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","PRIMARY_OUTPUT","1493","14150","4","14150","0","1","14150","3137","50249","6","14876","100039044","6","100039050","0.000006","14876","65125","22.842226","0.999880608895","0.771577735125","0.999999940023","0.999851334872","0.885788837574","0.944015480283","0.871017507367","0.808498656498","0.771506655817","0.878342538880","0.878277198610","0.000148665128","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ANY_CANDIDATE","1493","14150","4","12891","1259","6","15577","3311","","","","","0","100039050","0.000000","9465","65125","14.533589","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ALL_CANDIDATES","1493","14150","4","12891","1259","6","15577","3311","55660","47","9465","100039003","47","100039050","0.000047","9465","65125","14.533589","0.999156299926","0.854664107486","0.999999530183","0.999904978988","0.927331818835","0.966477168149","0.921279131356","0.880119668445","0.854047750568","0.924090378326","0.924046375011","0.000095021012","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14141","4","14141","0","1","14141","3134","50237","6","14872","99911755","6","99911761","0.000006","14872","65109","22.841696","0.999880580379","0.771583037675","0.999999939947","0.999851185579","0.885791488811","0.944017047440","0.871020875234","0.808503310491","0.771511940413","0.878345544488","0.878280138338","0.000148814421","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14141","4","12883","1258","6","15567","3308","","","","","0","99911761","0.000000","9465","65109","14.537161","","","","","","","","","","","","","","","","",""
|
||||
"HUNSPELL_UKRAINIAN_LUCENE_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14141","4","12883","1258","6","15567","3308","55644","47","9465","99911714","47","99911761","0.000047","9465","65109","14.537161","0.999156057532","0.854628392388","0.999999529585","0.999904857994","0.927313960987","0.966467852143","0.921258278146","0.880089331187","0.854011909878","0.924070957878","0.924026899410","0.000095142006","","","","",""
|
||||
"ITALIAN_LUCENE_ITALIAN_LIGHT_STEM_FILTER","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","PRIMARY_OUTPUT","10009","324366","0","324366","0","1","324366","244870","109427","2752","6024695","52600351921","2752","52600354673","0.000005","6024695","6134122","98.216094","0.975467779174","0.017839064825","0.999999947681","0.999885423887","0.508919506253","0.083115367566","0.035037376521","0.022197346412","0.017831065132","0.131914491042","0.131906553725","0.000114576113","","","","",""
|
||||
"ITALIAN_LUCENE_ITALIAN_LIGHT_STEM_FILTER","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","324285","0","324285","0","1","324285","244808","109401","2752","6023081","52574083236","2752","52574085988","0.000005","6023081","6132482","98.216040","0.975462091964","0.017839595779","0.999999947655","0.999885397338","0.508919771717","0.083117639712","0.035038396960","0.022198003488","0.017831593709","0.131916069597","0.131908130255","0.000114602662","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","PRIMARY_OUTPUT","10009","324366","0","324366","0","1","324366","10010","6093034","0","41088","52600354673","0","52600354673","0.000000","41088","6134122","0.669827","1.000000000000","0.993301730875","1.000000000000","0.999999218956","0.996650865438","0.998653128810","0.996639611043","0.994634196381","0.993301730875","0.996645238224","0.996644848967","0.000000781044","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","ANY_CANDIDATE","10009","324366","0","321297","3069","4","327552","10012","","","","","0","52600354673","0.000000","80","6134122","0.001304","","","","","","","","","","","","","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","ALL_CANDIDATES","10009","324366","0","321297","3069","4","327552","10012","6134042","0","80","52600354673","0","52600354673","0.000000","80","6134122","0.001304","1.000000000000","0.999986958199","1.000000000000","0.999999998479","0.999993479099","0.999997391613","0.999993479057","0.999989566532","0.999986958199","0.999993479078","0.999993478318","0.000000001521","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","324285","0","324285","0","1","324285","10007","6091474","0","41008","52574085988","0","52574085988","0.000000","41008","6132482","0.668702","1.000000000000","0.993312984857","1.000000000000","0.999999220087","0.996656492428","0.998655403904","0.996645275883","0.994643223717","0.993312984857","0.996650884140","0.996650495444","0.000000779913","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","10007","324285","0","321217","3068","4","327469","10007","","","","","0","52574085988","0.000000","0","6132482","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"ITALIAN_RADIXOR","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","10007","324285","0","321217","3068","4","327469","10007","6132482","0","0","52574085988","0","52574085988","0.000000","0","6132482","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","PRIMARY_OUTPUT","4992","26201","85","26201","0","1","26201","5015","62985","0","1452","343168663","0","343168663","0.000000","1452","64437","2.253364","1.000000000000","0.977466362494","1.000000000000","0.999995769639","0.988733181247","0.995410538693","0.988604793521","0.981891479829","0.977466362494","0.988668985300","0.988666893700","0.000004230361","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","ANY_CANDIDATE","4992","26201","85","25905","296","3","26501","5016","","","","","0","343168663","0.000000","0","64437","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","ALL_CANDIDATES","4992","26201","85","25905","296","3","26501","5016","64437","0","0","343168663","0","343168663","0.000000","0","64437","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25402","84","25402","0","1","25402","4797","61646","0","1372","322555083","0","322555083","0.000000","1372","63018","2.177156","1.000000000000","0.978228442667","1.000000000000","0.999995747294","0.989114221334","0.995568504079","0.988994416993","0.982506582345","0.978228442667","0.989054317349","0.989052213866","0.000004252706","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4796","25402","84","25129","273","3","25679","4797","","","","","0","322555083","0.000000","0","63018","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NL_NL_RADIXOR","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4796","25402","84","25129","273","3","25679","4797","63018","0","0","322555083","0","322555083","0.000000","0","63018","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","ALL_WORDS","PRIMARY_OUTPUT","4688","16937","23","16937","0","1","16937","4680","25582","0","2780","143394154","0","143394154","0.000000","2780","28362","9.801848","1.000000000000","0.901981524575","1.000000000000","0.999980616712","0.950990762288","0.978728288316","0.948465074892","0.920017262461","0.901981524575","0.949727078994","0.949717872891","0.000019383288","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","ALL_WORDS","ANY_CANDIDATE","4688","16937","23","15846","1091","5","18255","4693","","","","","0","143394154","0.000000","0","28362","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","ALL_WORDS","ALL_CANDIDATES","4688","16937","23","15846","1091","5","18255","4693","28362","0","0","143394154","0","143394154","0.000000","0","28362","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","16906","23","16906","0","1","16906","4668","25537","0","2768","142869660","0","142869660","0.000000","2768","28305","9.779191","1.000000000000","0.902208090443","1.000000000000","0.999980629535","0.951104045222","0.978781630166","0.948590319825","0.920205827454","0.902208090443","0.949846350966","0.949837149794","0.000019370465","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4681","16906","23","15820","1086","5","18219","4681","","","","","0","142869660","0.000000","0","28305","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NN_NO_RADIXOR","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4681","16906","23","15820","1086","5","18219","4681","28305","0","0","142869660","0","142869660","0.000000","0","28305","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_LIGHT_STEM_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","PRIMARY_OUTPUT","17929","73170","252","73170","0","1","73170","25999","98455","11122","42440","2676735848","11122","2676746970","0.000416","42440","140895","30.121722","0.898500597753","0.698782781504","0.999995844957","0.999979990944","0.849389313230","0.849917904431","0.786155737967","0.731292997027","0.647657827743","0.792374120527","0.792364773649","0.000020009056","","","","",""
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_LIGHT_STEM_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","73111","252","73111","0","1","73111","25985","98376","11069","42430","2672420730","11069","2672431799","0.000414","42430","140806","30.133659","0.898862442323","0.698663409230","0.999995858080","0.999979982209","0.849329633655","0.850141551991","0.786218636489","0.731236313848","0.647743209877","0.792465960393","0.792456612651","0.000020017791","","","","",""
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_MINIMAL_STEM_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","PRIMARY_OUTPUT","17929","73170","252","73170","0","1","73170","27457","93352","2948","47543","2676744022","2948","2676746970","0.000110","47543","140895","33.743568","0.969387331256","0.662564320948","0.999998898663","0.999981138171","0.831281609806","0.887216187191","0.787132949683","0.707340728617","0.648985352085","0.801424643288","0.801416635301","0.000018861829","","","","",""
|
||||
"NORWEGIAN_BOKMAL_LUCENE_NORWEGIAN_MINIMAL_STEM_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","73111","252","73111","0","1","73111","27443","93273","2895","47533","2672428904","2895","2672431799","0.000108","47533","140806","33.757794","0.969896431245","0.662422055878","0.999998916717","0.999981131289","0.831210486298","0.887506232421","0.787200283575","0.707265177618","0.649076902736","0.801548992872","0.801540987973","0.000018868711","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","PRIMARY_OUTPUT","17929","73170","252","73170","0","1","73170","17886","134138","0","6757","2676746970","0","2676746970","0.000000","6757","140895","4.795770","1.000000000000","0.952042301004","1.000000000000","0.999997475800","0.976021150502","0.990025787995","0.975432039064","0.961262286483","0.952042301004","0.975726550323","0.975725318796","0.000002524200","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","ANY_CANDIDATE","17929","73170","252","71073","2097","9","75343","17962","","","","","0","2676746970","0.000000","0","140895","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","ALL_CANDIDATES","17929","73170","252","71073","2097","9","75343","17962","140895","0","0","2676746970","0","2676746970","0.000000","0","140895","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","73111","252","73111","0","1","73111","17838","134115","0","6691","2672431799","0","2672431799","0.000000","6691","140806","4.751928","1.000000000000","0.952480718151","1.000000000000","0.999997496420","0.976240359076","0.990120573010","0.975662099294","0.961619814753","0.952480718151","0.975951186357","0.975949964609","0.000002503580","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","17914","73111","252","71047","2064","9","75251","17914","","","","","0","2672431799","0.000000","0","140806","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"NORWEGIAN_BOKMAL_RADIXOR","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","17914","73111","252","71047","2064","9","75251","17914","140806","0","0","2672431799","0","2672431799","0.000000","0","140806","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PERSIAN_LUCENE_PERSIAN_STEM_FILTER","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","ALL_WORDS","PRIMARY_OUTPUT","69","3544","0","3544","0","1","3544","3190","425","3","95619","6182149","3","6182152","0.000049","95619","96044","99.557494","0.992990654206","0.004425055183","0.999999514732","0.984769191660","0.502212284958","0.021737796146","0.008810846671","0.005525163545","0.004424916968","0.066287543635","0.065773583741","0.015230808340","","","","",""
|
||||
"PERSIAN_LUCENE_PERSIAN_STEM_FILTER","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","69","3544","0","3544","0","1","3544","3190","425","3","95619","6182149","3","6182152","0.000049","95619","96044","99.557494","0.992990654206","0.004425055183","0.999999514732","0.984769191660","0.502212284958","0.021737796146","0.008810846671","0.005525163545","0.004424916968","0.066287543635","0.065773583741","0.015230808340","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","ALL_WORDS","PRIMARY_OUTPUT","69","3544","0","3544","0","1","3544","69","91503","0","4541","6182152","0","6182152","0.000000","4541","96044","4.728041","1.000000000000","0.952719586856","1.000000000000","0.999276703053","0.976359793428","0.990172186921","0.975787402624","0.961814585046","0.952719586856","0.976073556068","0.975715273893","0.000723296947","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","ALL_WORDS","ANY_CANDIDATE","69","3544","0","3387","157","2","3701","69","","","","","0","6182152","0.000000","0","96044","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","ALL_WORDS","ALL_CANDIDATES","69","3544","0","3387","157","2","3701","69","96044","0","0","6182152","0","6182152","0.000000","0","96044","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","69","3544","0","3544","0","1","3544","69","91503","0","4541","6182152","0","6182152","0.000000","4541","96044","4.728041","1.000000000000","0.952719586856","1.000000000000","0.999276703053","0.976359793428","0.990172186921","0.975787402624","0.961814585046","0.952719586856","0.976073556068","0.975715273893","0.000723296947","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","69","3544","0","3387","157","2","3701","69","","","","","0","6182152","0.000000","0","96044","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"PERSIAN_RADIXOR","FA_IR","fa-ir-default","1.0.0","b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","69","3544","0","3387","157","2","3701","69","96044","0","0","6182152","0","6182152","0.000000","0","96044","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","PRIMARY_OUTPUT","9990","120867","1","120867","0","1","120867","15519","1001785","76101","115288","7303162237","76101","7303238338","0.001042","115288","1117073","10.320543","0.929397914065","0.896794569379","0.999989579828","0.999973797962","0.948392074604","0.922688964795","0.912805205017","0.903130948917","0.839596739453","0.912950711772","0.912937654604","0.000026202038","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ANY_CANDIDATE","9990","120867","1","109091","11776","5","133810","16295","","","","","73019","7303238338","0.001000","27850","1117073","2.493123","","","","","","","","","","","","","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ALL_CANDIDATES","9990","120867","1","109091","11776","5","133810","16295","1089223","100514","27850","7303137824","100514","7303238338","0.001376","27850","1117073","2.493123","0.915515782059","0.975068773482","0.999986237064","0.999982426375","0.987527505273","0.926837225395","0.944354324804","0.962546321343","0.894575089911","0.944823184896","0.944814549127","0.000017573625","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","119451","1","119451","0","1","119451","15277","996176","76097","114581","7133024121","76097","7133100218","0.001067","114581","1110757","10.315578","0.929032065528","0.896844224254","0.999989331848","0.999973272728","0.948416778051","0.922410978529","0.912654429852","0.903102115370","0.839341654492","0.912796276349","0.912782956155","0.000026727272","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","119451","1","107785","11666","5","132279","16044","","","","","73019","7133100218","0.001024","27489","1110757","2.474799","","","","","","","","","","","","","","","","",""
|
||||
"POLISH_LUCENE_MORFOLOGIK_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","119451","1","107785","11666","5","132279","16044","1083268","100503","27489","7132999715","100503","7133100218","0.001409","27489","1110757","2.474799","0.915099288629","0.975252012816","0.999985910334","0.999982059404","0.987618961575","0.926528792008","0.944218593105","0.962597028968","0.894331522547","0.944697000716","0.944688187384","0.000017940596","","","","",""
|
||||
"POLISH_LUCENE_STEMPEL_DIRECT","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","PRIMARY_OUTPUT","9990","120867","1","120867","0","1","120867","31432","794690","43990","322383","7303194348","43990","7303238338","0.000602","322383","1117073","28.859618","0.947548528640","0.711403820520","0.999993976645","0.999949841844","0.855698898582","0.888558571472","0.812669084491","0.748722623749","0.684450370049","0.821029623950","0.821007024526","0.000050158156","","","","",""
|
||||
"POLISH_LUCENE_STEMPEL_DIRECT","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","119451","1","119451","0","1","119451","30830","791610","43601","319147","7133056617","43601","7133100218","0.000611","319147","1110757","28.732387","0.947796425095","0.712676129883","0.999993887511","0.999949153732","0.856335008697","0.889129551368","0.813589945981","0.749880784102","0.685757797841","0.821870968068","0.821848045519","0.000050846268","","","","",""
|
||||
"POLISH_LUCENE_STEMPEL_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","PRIMARY_OUTPUT","9990","120867","1","120867","0","1","120867","31432","794690","43990","322383","7303194348","43990","7303238338","0.000602","322383","1117073","28.859618","0.947548528640","0.711403820520","0.999993976645","0.999949841844","0.855698898582","0.888558571472","0.812669084491","0.748722623749","0.684450370049","0.821029623950","0.821007024526","0.000050158156","","","","",""
|
||||
"POLISH_LUCENE_STEMPEL_FILTER","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","119451","1","119451","0","1","119451","30830","791610","43601","319147","7133056617","43601","7133100218","0.000611","319147","1110757","28.732387","0.947796425095","0.712676129883","0.999993887511","0.999949153732","0.856335008697","0.889129551368","0.813589945981","0.749880784102","0.685757797841","0.821870968068","0.821848045519","0.000050846268","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","PRIMARY_OUTPUT","9990","120867","1","120867","0","1","120867","10074","1097200","0","19873","7303238338","0","7303238338","0.000000","19873","1117073","1.779024","1.000000000000","0.982209757106","1.000000000000","0.999997279294","0.991104878553","0.996390581475","0.991025045241","0.985716986027","0.982209757106","0.991064961093","0.991063612692","0.000002720706","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ANY_CANDIDATE","9990","120867","1","119475","1392","4","122430","10079","","","","","0","7303238338","0.000000","0","1117073","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","ALL_WORDS","ALL_CANDIDATES","9990","120867","1","119475","1392","4","122430","10079","1117073","0","0","7303238338","0","7303238338","0.000000","0","1117073","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","9846","119451","1","119451","0","1","119451","9844","1091431","0","19326","7133100218","0","7133100218","0.000000","19326","1110757","1.739895","1.000000000000","0.982601054956","1.000000000000","0.999997291081","0.991300527478","0.996471091564","0.991224182495","0.986032239104","0.982601054956","0.991262354251","0.991261011420","0.000002708919","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","9846","119451","1","118145","1306","4","120926","9847","","","","","0","7133100218","0.000000","0","1110757","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"POLISH_RADIXOR","PL_PL","pl-pl-unimorph","1.0.0","8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","9846","119451","1","118145","1306","4","120926","9847","1110757","0","0","7133100218","0","7133100218","0.000000","0","1110757","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_LIGHT_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","112814","149580","1249","5336772","22274111994","1249","22274113243","0.000006","5336772","5486352","97.273598","0.991719099112","0.027264018058","0.999999943926","0.999760407678","0.513631980992","0.122814577084","0.053069078321","0.033847392205","0.027257812658","0.164433109277","0.164413080578","0.000239592322","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_LIGHT_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","112814","149580","1249","5336772","22274111994","1249","22274113243","0.000006","5336772","5486352","97.273598","0.991719099112","0.027264018058","0.999999943926","0.999760407678","0.513631980992","0.122814577084","0.053069078321","0.033847392205","0.027257812658","0.164433109277","0.164413080578","0.000239592322","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_MINIMAL_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","167745","43329","17","5443023","22274113226","17","22274113243","0.000000","5443023","5486352","99.210240","0.999607806949","0.007897597529","0.999999999237","0.999755693994","0.503948798383","0.038278287185","0.015671380245","0.009852536437","0.007897573058","0.088850999693","0.088840137075","0.000244306006","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_MINIMAL_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","167745","43329","17","5443023","22274113226","17","22274113243","0.000000","5443023","5486352","99.210240","0.999607806949","0.007897597529","0.999999999237","0.999755693994","0.503948798383","0.038278287185","0.015671380245","0.009852536437","0.007897573058","0.088850999693","0.088840137075","0.000244306006","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","27586","3802658","80995","1683694","22274032248","80995","22274113243","0.000364","1683694","5486352","30.688771","0.979144635218","0.693112290280","0.999996363716","0.999920793505","0.846554326998","0.904491820642","0.811666162398","0.736120062592","0.683028738823","0.823806518930","0.823772560883","0.000079206495","","","","",""
|
||||
"PORTUGUESE_LUCENE_PORTUGUESE_STEM_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","27586","3802658","80995","1683694","22274032248","80995","22274113243","0.000364","1683694","5486352","30.688771","0.979144635218","0.693112290280","0.999996363716","0.999920793505","0.846554326998","0.904491820642","0.811666162398","0.736120062592","0.683028738823","0.823806518930","0.823772560883","0.000079206495","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","4001","5470353","0","15999","22274113243","0","22274113243","0.000000","15999","5486352","0.291615","1.000000000000","0.997083854627","1.000000000000","0.999999281899","0.998541927313","0.999415407119","0.998539798233","0.997665722283","0.997083854627","0.998540862773","0.998540504158","0.000000718101","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","ANY_CANDIDATE","4001","211091","0","210699","392","3","211489","4001","","","","","0","22274113243","0.000000","0","5486352","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","ALL_CANDIDATES","4001","211091","0","210699","392","3","211489","4001","5486352","0","0","22274113243","0","22274113243","0.000000","0","5486352","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","4001","5470353","0","15999","22274113243","0","22274113243","0.000000","15999","5486352","0.291615","1.000000000000","0.997083854627","1.000000000000","0.999999281899","0.998541927313","0.999415407119","0.998539798233","0.997665722283","0.997083854627","0.998540862773","0.998540504158","0.000000718101","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","4001","211091","0","210699","392","3","211489","4001","","","","","0","22274113243","0.000000","0","5486352","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"PORTUGUESE_RADIXOR","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","4001","211091","0","210699","392","3","211489","4001","5486352","0","0","22274113243","0","22274113243","0.000000","0","5486352","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"RUSSIAN_LUCENE_RUSSIAN_LIGHT_STEM_FILTER","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","PRIMARY_OUTPUT","37410","759333","10","759333","0","1","759333","232250","3036212","170067","10001394","288279715105","170067","288279885172","0.000059","10001394","13037606","76.711890","0.946958140574","0.232881097956","0.999999410063","0.999964718312","0.616440254010","0.586986164875","0.373828305236","0.274240682289","0.229882432734","0.469604782232","0.469595585287","0.000035281688","","","","",""
|
||||
"RUSSIAN_LUCENE_RUSSIAN_LIGHT_STEM_FILTER","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","758584","10","758584","0","1","758584","232143","3034033","167825","10001194","287711260184","167825","287711428009","0.000058","10001194","13035227","76.724356","0.947585120889","0.232756437613","0.999999416690","0.999964657093","0.616377927152","0.587020283013","0.373716464501","0.274112880444","0.229797852799","0.469634471769","0.469625269488","0.000035342907","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","PRIMARY_OUTPUT","37410","759333","10","759333","0","1","759333","37561","12781761","0","255845","288279885172","0","288279885172","0.000000","255845","13037606","1.962362","1.000000000000","0.980376381983","1.000000000000","0.999999112552","0.990188190992","0.996012679879","0.990090965437","0.984239248995","0.980376381983","0.990139577021","0.990139137653","0.000000887448","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","ANY_CANDIDATE","37410","759333","10","749720","9613","4","769106","37593","","","","","0","288279885172","0.000000","13","13037606","0.000100","","","","","","","","","","","","","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","ALL_CANDIDATES","37410","759333","10","749720","9613","4","769106","37593","13037593","0","13","288279885172","0","288279885172","0.000000","13","13037606","0.000100","1.000000000000","0.999999002884","1.000000000000","0.999999999955","0.999999501442","0.999999800577","0.999999501442","0.999999202307","0.999999002884","0.999999501442","0.999999501420","0.000000000045","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","758584","10","758584","0","1","758584","37282","12780071","0","255156","287711428009","0","287711428009","0.000000","255156","13035227","1.957434","1.000000000000","0.980425657336","1.000000000000","0.999999113193","0.990212828668","0.996022851412","0.990116093179","0.984278980143","0.980425657336","0.990164459742","0.990164020680","0.000000886807","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","37297","758584","10","749142","9442","4","768163","37306","","","","","0","287711428009","0.000000","0","13035227","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"RUSSIAN_RADIXOR","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","37297","758584","10","749142","9442","4","768163","37306","13035227","0","0","287711428009","0","287711428009","0.000000","0","13035227","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"SNOWBALL_DANISH_DIRECT","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","ALL_WORDS","PRIMARY_OUTPUT","4179","27921","32","27921","0","1","27921","5553","78545","4795","11150","389682670","4795","389687465","0.001230","11150","89695","12.431016","0.942464602832","0.875689837784","0.999987695268","0.999959092010","0.937838766526","0.928307194100","0.907851012801","0.888276938388","0.831251984337","0.908463909669","0.908443737019","0.000040907990","","","","",""
|
||||
"SNOWBALL_DANISH_DIRECT","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","27875","32","27875","0","1","27875","5539","78440","4795","11100","388399540","4795","388404335","0.001235","11100","89540","12.396694","0.942392022587","0.876033057851","0.999987654618","0.999959085584","0.938010356234","0.928327968188","0.908001736362","0.888546539947","0.831504743732","0.908606936602","0.908586757624","0.000040914416","","","","",""
|
||||
"SNOWBALL_DANISH_LUCENE_FILTER","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","ALL_WORDS","PRIMARY_OUTPUT","4179","27921","32","27921","0","1","27921","5546","78557","4961","11138","389682504","4961","389687465","0.001273","11138","89695","12.417638","0.940599631217","0.875823624505","0.999987269285","0.999958696913","0.937905446895","0.926889068757","0.907056629699","0.888055112164","0.829920977011","0.907633945058","0.907613558620","0.000041303087","","","","",""
|
||||
"SNOWBALL_DANISH_LUCENE_FILTER","DA_DK","da-dk-default","1.0.0","3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4173","27875","32","27875","0","1","27875","5539","78440","4795","11100","388399540","4795","388404335","0.001235","11100","89540","12.396694","0.942392022587","0.876033057851","0.999987654618","0.999959085584","0.938010356234","0.928327968188","0.908001736362","0.888546539947","0.831504743732","0.908606936602","0.908586757624","0.000040914416","","","","",""
|
||||
"SNOWBALL_DUTCH_DIRECT","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","PRIMARY_OUTPUT","4992","26201","85","26201","0","1","26201","12051","29267","2987","35170","343165676","2987","343168663","0.000870","35170","64437","54.580443","0.907391331308","0.454195570868","0.999991295825","0.999888830652","0.727093433346","0.756436964017","0.605371751249","0.504599968276","0.434073920266","0.641975952605","0.641933549660","0.000111169348","","","","",""
|
||||
"SNOWBALL_DUTCH_DIRECT","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25402","84","25402","0","1","25402","11466","29053","2987","33965","322552096","2987","322555083","0.000926","33965","63018","53.897299","0.906772784020","0.461027008156","0.999990739566","0.999885462099","0.730508873861","0.759841613575","0.611268909508","0.511294841471","0.440163623968","0.646565343716","0.646521311443","0.000114537901","","","","",""
|
||||
"SNOWBALL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","ALL_WORDS","PRIMARY_OUTPUT","4992","26201","85","26201","0","1","26201","14573","15204","759","49233","343167904","759","343168663","0.000221","49233","64437","76.404861","0.952452546514","0.235951394385","0.999997788260","0.999854349712","0.617974591322","0.592568341791","0.378208955224","0.277738198319","0.233204491073","0.474059602198","0.474021680915","0.000145650288","","","","",""
|
||||
"SNOWBALL_DUTCH_LUCENE_FILTER","NL_NL","nl-nl-default","1.0.0","c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4796","25402","84","25402","0","1","25402","14116","14874","715","48144","322554368","715","322555083","0.000222","48144","63018","76.397220","0.954134325486","0.236027801581","0.999997783324","0.999848554685","0.618012792452","0.593185189912","0.378439579172","0.277851461363","0.233379881694","0.474554767395","0.474515425112","0.000151445315","","","","",""
|
||||
"SNOWBALL_FINNISH_DIRECT","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","PRIMARY_OUTPUT","57027","1788784","292","1788784","0","1","1788784","381483","15082807","952306","16382792","1599840787031","952306","1599841739337","0.000060","16382792","31465599","52.065724","0.940611207417","0.479342757784","0.999999404750","0.999989164705","0.739671081267","0.788799811426","0.635056038739","0.531468350160","0.465261620083","0.671472389727","0.671468317739","0.000010835295","","","","",""
|
||||
"SNOWBALL_FINNISH_DIRECT","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1734784","274","1734784","0","1","1734784","372232","14663371","936765","16097512","1504705198288","936765","1504706135053","0.000062","16097512","30760883","52.331112","0.939951485038","0.476688884386","0.999999377443","0.999988679564","0.738344130915","0.786987247415","0.632573283171","0.528815026735","0.462601231486","0.669376145960","0.669371899615","0.000011320436","","","","",""
|
||||
"SNOWBALL_FINNISH_LUCENE_FILTER","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","ALL_WORDS","PRIMARY_OUTPUT","57027","1788784","292","1788784","0","1","1788784","377778","15121052","1288634","16344547","1599840450703","1288634","1599841739337","0.000081","16344547","31465599","51.944179","0.921471136011","0.480558212161","0.999999194524","0.999988978388","0.740278703342","0.778598131291","0.631685095974","0.531413183368","0.461651842069","0.665447610018","0.665443363449","0.000011021612","","","","",""
|
||||
"SNOWBALL_FINNISH_LUCENE_FILTER","FI_FI","fi-fi-default","1.0.0","ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","54762","1734784","274","1734784","0","1","1734784","372232","14663371","936765","16097512","1504705198288","936765","1504706135053","0.000062","16097512","30760883","52.331112","0.939951485038","0.476688884386","0.999999377443","0.999988679564","0.738344130915","0.786987247415","0.632573283171","0.528815026735","0.462601231486","0.669376145960","0.669371899615","0.000011320436","","","","",""
|
||||
"SNOWBALL_FRENCH_DIRECT","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","85627","3744838","1092238","1625361","81605779618","1092238","81606871856","0.001338","1625361","5370199","30.266309","0.774194575401","0.697336914330","0.999986615858","0.999966701086","0.848661765094","0.757496924470","0.733758618240","0.711462917671","0.579477680015","0.734761496202","0.734744993787","0.000033298914","","","","",""
|
||||
"SNOWBALL_FRENCH_DIRECT","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","84526","3736871","1088903","1619380","80278407962","1088903","80279496865","0.001356","1619380","5356251","30.233460","0.774356818202","0.697665400669","0.999986436101","0.999966266576","0.848825918385","0.757698693319","0.734013322497","0.711763857056","0.579795455624","0.735011537210","0.734994818860","0.000033733424","","","","",""
|
||||
"SNOWBALL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","ALL_WORDS","PRIMARY_OUTPUT","59240","404011","2301","404011","0","1","404011","85202","3742072","1097843","1628127","81605774013","1097843","81606871856","0.001345","1628127","5370199","30.317815","0.773168950281","0.696821849619","0.999986547175","0.999966598516","0.848404198397","0.756589837411","0.733012775372","0.710860736247","0.578547882033","0.734003418250","0.733986862867","0.000033401484","","","","",""
|
||||
"SNOWBALL_FRENCH_LUCENE_FILTER","FR_FR","fr-fr-default","1.0.0","a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","57698","400712","2133","400712","0","1","400712","84810","3734232","1086494","1622019","80278410371","1086494","80279496865","0.001353","1622019","5356251","30.282729","0.774620254294","0.697172705312","0.999986466109","0.999966263711","0.848579585710","0.757784104203","0.733858787339","0.711398006457","0.579602638316","0.734876927298","0.734860210439","0.000033736289","","","","",""
|
||||
"SNOWBALL_GERMAN_DIRECT","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","81649","742376","65811","602476","38436668082","65811","38436733893","0.000171","602476","1344852","44.798684","0.918569588474","0.552013158325","0.999998287810","0.999982613933","0.776005723068","0.810879063265","0.689607573295","0.599890587538","0.526260347085","0.712083211201","0.712076031428","0.000017386067","","","","",""
|
||||
"SNOWBALL_GERMAN_DIRECT","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","37843","506459","41477","351958","10594922057","41477","10594963534","0.000391","351958","858417","41.000819","0.924303203294","0.589991810507","0.999996085215","0.999962868855","0.794993947861","0.830216831177","0.720244490537","0.635998708058","0.562798507380","0.738465517386","0.738449797528","0.000037131145","","","","",""
|
||||
"SNOWBALL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","ALL_WORDS","PRIMARY_OUTPUT","54092","277266","1474","277266","0","1","277266","86669","723725","142783","621127","38436591110","142783","38436733893","0.000371","621127","1344852","46.185528","0.835220217240","0.538144717783","0.999996285246","0.999980126218","0.769070501515","0.752174652309","0.654551949931","0.579358576068","0.486493662760","0.670424751999","0.670415952491","0.000019873782","","","","",""
|
||||
"SNOWBALL_GERMAN_LUCENE_FILTER","DE_DE","de-de-default","1.0.0","cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","16007","145574","228","145574","0","1","145574","46077","471644","34482","386773","10594929052","34482","10594963534","0.000325","386773","858417","45.056540","0.931870719939","0.549434598802","0.999996745435","0.999960243292","0.774715672118","0.817996747049","0.691284921032","0.598564290417","0.528216517210","0.715543160924","0.715527026594","0.000039756708","","","","",""
|
||||
"SNOWBALL_HUNGARIAN_DIRECT","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","PRIMARY_OUTPUT","19406","910688","1","910688","0","1","910688","116105","14275129","1281527","7842726","414652461946","1281527","414653743473","0.000309","7842726","22117855","35.458800","0.917621949087","0.645411998587","0.999996909404","0.999977996662","0.822704453996","0.846239680964","0.757813631609","0.686119053091","0.610064359819","0.769574048489","0.769564274829","0.000022003338","","","","",""
|
||||
"SNOWBALL_HUNGARIAN_DIRECT","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","872878","1","872878","0","1","872878","111379","13763897","1273370","7603420","380934924316","1273370","380936197686","0.000334","7603420","21367317","35.584346","0.915319053655","0.644156540571","0.999996657262","0.999976698743","0.822076598916","0.844241130173","0.756162850261","0.684726470771","0.607927533294","0.767859853828","0.767849489469","0.000023301257","","","","",""
|
||||
"SNOWBALL_HUNGARIAN_LUCENE_FILTER","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","ALL_WORDS","PRIMARY_OUTPUT","19406","910688","1","910688","0","1","910688","114867","14286575","1565633","7831280","414652177840","1565633","414653743473","0.000378","7831280","22117855","35.407050","0.901235651210","0.645929499040","0.999996224240","0.999977339137","0.822962861640","0.835211528771","0.752517845440","0.684723838400","0.603229346961","0.762977517823","0.762967293456","0.000022660863","","","","",""
|
||||
"SNOWBALL_HUNGARIAN_LUCENE_FILTER","HU_HU","hu-hu-default","1.0.0","359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","18360","872878","1","872878","0","1","872878","111379","13763897","1273370","7603420","380934924316","1273370","380936197686","0.000334","7603420","21367317","35.584346","0.915319053655","0.644156540571","0.999996657262","0.999976698743","0.822076598916","0.844241130173","0.756162850261","0.684726470771","0.607927533294","0.767859853828","0.767849489469","0.000023301257","","","","",""
|
||||
"SNOWBALL_ITALIAN_DIRECT","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","PRIMARY_OUTPUT","10009","324366","0","324366","0","1","324366","46828","4493783","388246","1640339","52599966427","388246","52600354673","0.000738","1640339","6134122","26.741219","0.920474458468","0.732587809633","0.999992618947","0.999961438502","0.866290214290","0.875563347203","0.815853559015","0.763767765724","0.688980290594","0.821174991918","0.821156945015","0.000038561498","","","","",""
|
||||
"SNOWBALL_ITALIAN_DIRECT","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","324285","0","324285","0","1","324285","46814","4492785","388246","1639697","52573697742","388246","52574085988","0.000738","1639697","6132482","26.737902","0.920458198278","0.732620984456","0.999992615259","0.999961431446","0.866306799858","0.875561054334","0.815867743562","0.763794373383","0.689000522641","0.821186331701","0.821168280921","0.000038568554","","","","",""
|
||||
"SNOWBALL_ITALIAN_LUCENE_FILTER","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","ALL_WORDS","PRIMARY_OUTPUT","10009","324366","0","324366","0","1","324366","46828","4493783","388246","1640339","52599966427","388246","52600354673","0.000738","1640339","6134122","26.741219","0.920474458468","0.732587809633","0.999992618947","0.999961438502","0.866290214290","0.875563347203","0.815853559015","0.763767765724","0.688980290594","0.821174991918","0.821156945015","0.000038561498","","","","",""
|
||||
"SNOWBALL_ITALIAN_LUCENE_FILTER","IT_IT","it-it-default","1.0.0","5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","10007","324285","0","324285","0","1","324285","46814","4492785","388246","1639697","52573697742","388246","52574085988","0.000738","1639697","6132482","26.737902","0.920458198278","0.732620984456","0.999992615259","0.999961431446","0.866306799858","0.875561054334","0.815867743562","0.763794373383","0.689000522641","0.821186331701","0.821168280921","0.000038568554","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","PRIMARY_OUTPUT","17929","73170","252","73170","0","1","73170","24394","105463","10337","35432","2676736633","10337","2676746970","0.000386","35432","140895","25.147805","0.910734024180","0.748521948969","0.999996138223","0.999982902160","0.874259043596","0.872900785472","0.821698903368","0.776170920545","0.697359024545","0.825653926758","0.825645796759","0.000017097840","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","73111","252","73111","0","1","73111","24367","105404","10337","35402","2672421462","10337","2672431799","0.000387","35402","140806","25.142395","0.910688520058","0.748576054998","0.999996131987","0.999982885778","0.874286093493","0.872882057737","0.821712980468","0.776210850338","0.697379303044","0.825663139347","0.825655001073","0.000017114222","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","ALL_WORDS","PRIMARY_OUTPUT","17929","73170","252","73170","0","1","73170","24396","105429","10403","35466","2676736567","10403","2676746970","0.000389","35466","140895","25.171937","0.910188894261","0.748280634515","0.999996113566","0.999982864803","0.874138374041","0.872434515071","0.821331609063","0.775884146880","0.696830096895","0.825273726303","0.825265576414","0.000017135197","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER","NB_NO","nb-no-default","1.0.0","f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","17914","73111","252","73111","0","1","73111","24381","105352","10350","35454","2672421449","10350","2672431799","0.000387","35454","140806","25.179325","0.910546057977","0.748206752553","0.999996127123","0.999982861457","0.874101439838","0.872676909416","0.821432469942","0.775872480948","0.696975310275","0.825394880702","0.825386731577","0.000017138543","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","ALL_WORDS","PRIMARY_OUTPUT","4688","16937","23","16937","0","1","16937","6138","20880","1201","7482","143392953","1201","143394154","0.000838","7482","28362","26.380368","0.945609347403","0.736196319018","0.999991624484","0.999939458599","0.868093971751","0.894708876815","0.827865115080","0.770314840366","0.706288265738","0.834358508549","0.834330743646","0.000060541401","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","16906","23","16906","0","1","16906","6120","20847","1201","7458","142868459","1201","142869660","0.000841","7458","28305","26.348702","0.945527939042","0.736512983572","0.999991593737","0.999939404316","0.868252288654","0.894744070663","0.828034079399","0.770581364403","0.706534264217","0.834502009245","0.834474211808","0.000060595684","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","ALL_WORDS","PRIMARY_OUTPUT","4688","16937","23","16937","0","1","16937","6144","20854","1222","7508","143392932","1222","143394154","0.000852","7508","28362","26.472040","0.944645769161","0.735279599464","0.999991478035","0.999939130896","0.867635538749","0.893747964274","0.826916213966","0.769384020542","0.704908058410","0.833413920441","0.833385994629","0.000060869104","","","","",""
|
||||
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER","NN_NO","nn-no-default","1.0.0","900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4681","16906","23","16906","0","1","16906","6130","20824","1201","7481","142868459","1201","142869660","0.000841","7481","28305","26.429959","0.945471055619","0.735700406289","0.999991593737","0.999939243362","0.867846000013","0.894463296250","0.827498509835","0.769862102111","0.705754761743","0.834016450529","0.833988591624","0.000060756638","","","","",""
|
||||
"SNOWBALL_PORTUGUESE_DIRECT","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","11315","4816198","146201","670154","22273967042","146201","22274113243","0.000656","670154","5486352","12.214929","0.970538241685","0.877850710272","0.999993436282","0.999963358632","0.938922073277","0.950467296507","0.921870566157","0.894944355740","0.855064834721","0.923031789707","0.923014032222","0.000036641368","","","","",""
|
||||
"SNOWBALL_PORTUGUESE_DIRECT","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","11315","4816198","146201","670154","22273967042","146201","22274113243","0.000656","670154","5486352","12.214929","0.970538241685","0.877850710272","0.999993436282","0.999963358632","0.938922073277","0.950467296507","0.921870566157","0.894944355740","0.855064834721","0.923031789707","0.923014032222","0.000036641368","","","","",""
|
||||
"SNOWBALL_PORTUGUESE_LUCENE_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","ALL_WORDS","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","11315","4816198","146201","670154","22273967042","146201","22274113243","0.000656","670154","5486352","12.214929","0.970538241685","0.877850710272","0.999993436282","0.999963358632","0.938922073277","0.950467296507","0.921870566157","0.894944355740","0.855064834721","0.923031789707","0.923014032222","0.000036641368","","","","",""
|
||||
"SNOWBALL_PORTUGUESE_LUCENE_FILTER","PT_PT","pt-pt-default","1.0.0","7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","4001","211091","0","211091","0","1","211091","11315","4816198","146201","670154","22273967042","146201","22274113243","0.000656","670154","5486352","12.214929","0.970538241685","0.877850710272","0.999993436282","0.999963358632","0.938922073277","0.950467296507","0.921870566157","0.894944355740","0.855064834721","0.923031789707","0.923014032222","0.000036641368","","","","",""
|
||||
"SNOWBALL_RUSSIAN_DIRECT","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","PRIMARY_OUTPUT","37410","759333","10","759333","0","1","759333","64358","8723768","3499880","4313838","288276385292","3499880","288279885172","0.001214","4313838","13037606","33.087654","0.713679582396","0.669123457175","0.999987859437","0.999972896601","0.834555658306","0.704299886143","0.690683684983","0.677583981327","0.527514762522","0.691042509176","0.691028995355","0.000027103399","","","","",""
|
||||
"SNOWBALL_RUSSIAN_DIRECT","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","758584","10","758584","0","1","758584","64159","8721831","3499880","4313396","287707928129","3499880","287711428009","0.001216","4313396","13035227","33.090302","0.713634203918","0.669096978518","0.999987835450","0.999972844589","0.834542406984","0.704258664088","0.690648327996","0.677554078401","0.527473514384","0.691006866542","0.690993326753","0.000027155411","","","","",""
|
||||
"SNOWBALL_RUSSIAN_LUCENE_FILTER","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","ALL_WORDS","PRIMARY_OUTPUT","37410","759333","10","759333","0","1","759333","64266","8724001","3502741","4313605","288276382431","3502741","288279885172","0.001215","4313605","13037606","33.085867","0.713518041028","0.669141328554","0.999987849513","0.999972887486","0.834564589033","0.704177980141","0.690617545325","0.677569512768","0.527437604544","0.690973523314","0.690960004637","0.000027112514","","","","",""
|
||||
"SNOWBALL_RUSSIAN_LUCENE_FILTER","RU_RU","ru-ru-default","1.0.0","df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","37297","758584","10","758584","0","1","758584","64159","8721831","3499880","4313396","287707928129","3499880","287711428009","0.001216","4313396","13035227","33.090302","0.713634203918","0.669096978518","0.999987835450","0.999972844589","0.834542406984","0.704258664088","0.690648327996","0.677554078401","0.527473514384","0.691006866542","0.690993326753","0.000027155411","","","","",""
|
||||
"SNOWBALL_SPANISH_DIRECT","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","195021","12786403","1491944","29152637","360918051646","1491944","360919543590","0.000413","29152637","41939040","69.511932","0.895510033479","0.304880679195","0.999995866270","0.999915102906","0.652438272733","0.645436121970","0.454891402192","0.351208219000","0.294407398184","0.522516705219","0.522489599380","0.000084897094","","","","",""
|
||||
"SNOWBALL_SPANISH_DIRECT","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","194444","12762004","1472547","29068462","359405672368","1472547","359407144915","0.000410","29068462","41830466","69.491126","0.896551215419","0.305088736042","0.999995902844","0.999915033813","0.652542319443","0.646055272385","0.455257295293","0.351461114717","0.294713995998","0.522998735285","0.522971634774","0.000084966187","","","","",""
|
||||
"SNOWBALL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","194971","12786409","1493087","29152631","360918050503","1493087","360919543590","0.000414","29152631","41939040","69.511918","0.895438396425","0.304880822260","0.999995863103","0.999915099756","0.652438342682","0.645406478192","0.454882318529","0.351206166994","0.294399788433","0.522495927817","0.522468818464","0.000084900244","","","","",""
|
||||
"SNOWBALL_SPANISH_LUCENE_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","194444","12762004","1472547","29068462","359405672368","1472547","359407144915","0.000410","29068462","41830466","69.491126","0.896551215419","0.305088736042","0.999995902844","0.999915033813","0.652542319443","0.646055272385","0.455257295293","0.351461114717","0.294713995998","0.522998735285","0.522971634774","0.000084966187","","","","",""
|
||||
"SNOWBALL_SWEDISH_DIRECT","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","PRIMARY_OUTPUT","12371","95181","68","95181","0","1","95181","25915","234278","37166","145369","4529246977","37166","4529284143","0.000821","145369","379647","38.290570","0.863080414376","0.617094300758","0.999991794288","0.999959702307","0.808543047523","0.799352814853","0.719647483992","0.654396122527","0.562069801086","0.729795865162","0.729777443809","0.000040297693","","","","",""
|
||||
"SNOWBALL_SWEDISH_DIRECT","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","94954","68","94954","0","1","94954","25840","233849","37166","145019","4507667547","37166","4507704713","0.000824","145019","378868","38.276920","0.862863679132","0.617230803340","0.999991755006","0.999959587040","0.808611279173","0.799249860554","0.719664924302","0.654493987362","0.562091079094","0.729784928485","0.729766449710","0.000040412960","","","","",""
|
||||
"SNOWBALL_SWEDISH_LUCENE_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","PRIMARY_OUTPUT","12371","95181","68","95181","0","1","95181","26781","227960","35082","151687","4529249061","35082","4529284143","0.000775","151687","379647","39.954747","0.866629663704","0.600452525636","0.999992254405","0.999958767580","0.800222390020","0.796052562656","0.709394434944","0.639751239034","0.549660139513","0.721366737771","0.721348147050","0.000041232420","","","","",""
|
||||
"SNOWBALL_SWEDISH_LUCENE_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","94954","68","94954","0","1","94954","26706","227531","35082","151337","4507669631","35082","4507704713","0.000778","151337","378868","39.944519","0.866411792257","0.600554810646","0.999992217325","0.999958647839","0.800273513986","0.795941426692","0.709392795734","0.639820368543","0.549658171277","0.721337486785","0.721318837420","0.000041352161","","","","",""
|
||||
"SNOWBALL_YIDDISH_DIRECT","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","ALL_WORDS","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","1087","4943","823","1375","6228605","823","6229428","0.013211","1375","6318","21.763216","0.857266736039","0.782367837923","0.999867885141","0.999647516111","0.891117861532","0.841161255190","0.818106587223","0.796281976932","0.692199971993","0.818961490425","0.818786919931","0.000352483889","","","","",""
|
||||
"SNOWBALL_YIDDISH_DIRECT","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","1087","4943","823","1375","6228605","823","6229428","0.013211","1375","6318","21.763216","0.857266736039","0.782367837923","0.999867885141","0.999647516111","0.891117861532","0.841161255190","0.818106587223","0.796281976932","0.692199971993","0.818961490425","0.818786919931","0.000352483889","","","","",""
|
||||
"SNOWBALL_YIDDISH_LUCENE_FILTER","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","ALL_WORDS","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","1087","4943","823","1375","6228605","823","6229428","0.013211","1375","6318","21.763216","0.857266736039","0.782367837923","0.999867885141","0.999647516111","0.891117861532","0.841161255190","0.818106587223","0.796281976932","0.692199971993","0.818961490425","0.818786919931","0.000352483889","","","","",""
|
||||
"SNOWBALL_YIDDISH_LUCENE_FILTER","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","1087","4943","823","1375","6228605","823","6229428","0.013211","1375","6318","21.763216","0.857266736039","0.782367837923","0.999867885141","0.999647516111","0.891117861532","0.841161255190","0.818106587223","0.796281976932","0.692199971993","0.818961490425","0.818786919931","0.000352483889","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","405552","1221659","32066","40717381","360919511524","32066","360919543590","0.000009","40717381","41939040","97.087060","0.974423418214","0.029129398289","0.999999911155","0.999887108600","0.514564654722","0.130091212793","0.056567760828","0.036141643431","0.029107143376","0.168476609210","0.168466600268","0.000112891400","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","404617","1219357","31857","40611109","359407113058","31857","359407144915","0.000009","40611109","41830466","97.085003","0.974539127599","0.029149974088","0.999999911362","0.999886929804","0.514574942725","0.130174935063","0.056606752569","0.036167014759","0.029127791117","0.168546107394","0.168536080782","0.000113070196","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","718633","140718","4263","41798322","360919539327","4263","360919543590","0.000001","41798322","41939040","99.664470","0.970596147081","0.003355298548","0.999999988189","0.999884191009","0.501677643368","0.016547675056","0.006687478841","0.004190501600","0.003354957524","0.057066976813","0.057063472077","0.000115808991","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","717093","140505","4162","41689961","359407140753","4162","359407144915","0.000001","41689961","41830466","99.664108","0.971230481036","0.003358915485","0.999999988420","0.999884005448","0.501679451953","0.016565417252","0.006694678013","0.004195017332","0.003358581317","0.057116382085","0.057112873151","0.000115994552","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_PLURAL_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","578805","315690","6721","41623350","360919536869","6721","360919543590","0.000002","41623350","41939040","99.247265","0.979153937055","0.007527353988","0.999999981378","0.999884668938","0.503763667683","0.036513949858","0.014939856182","0.009391143622","0.007526147875","0.085851256794","0.085846095212","0.000115331062","","","","",""
|
||||
"SPANISH_LUCENE_SPANISH_PLURAL_STEM_FILTER","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","577533","315155","6589","41515311","359407138326","6589","359407144915","0.000002","41515311","41830466","99.246590","0.979520985628","0.007534102059","0.999999981667","0.999884484578","0.503767041863","0.036546115143","0.014953189880","0.009399553128","0.007532915498","0.085905826777","0.085900657455","0.000115515422","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","PRIMARY_OUTPUT","65059","849661","3589","849661","0","1","849661","64995","41053986","0","885054","360919543590","0","360919543590","0.000000","885054","41939040","2.110334","1.000000000000","0.978896655717","1.000000000000","0.999997548065","0.989448327859","0.995706851308","0.989335802746","0.983045766368","0.978896655717","0.989392063702","0.989390850601","0.000002451935","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","ANY_CANDIDATE","65059","849661","3589","828694","20967","21","871404","65118","","","","","0","360919543590","0.000000","626","41939040","0.001493","","","","","","","","","","","","","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","ALL_WORDS","ALL_CANDIDATES","65059","849661","3589","828694","20967","21","871404","65118","41938414","1737","626","360919541853","1737","360919543590","0.000000","626","41939040","0.001493","0.999958583840","0.999985073573","0.999999995187","0.999999993454","0.999992534380","0.999963881674","0.999971828531","0.999979775515","0.999943658650","0.999971828619","0.999971825345","0.000000006546","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","64918","847879","3525","847879","0","1","847879","64814","40958710","34","871756","359407144881","34","359407144915","0.000000","871756","41830466","2.084022","0.999999169896","0.979159782729","0.999999999905","0.999997574649","0.989579891317","0.995760629537","0.989469763028","0.983257884479","0.979158986864","0.989524618150","0.989523418038","0.000002425351","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","64918","847879","3525","826968","20911","21","869542","64933","","","","","0","359407144915","0.000000","0","41830466","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"SPANISH_RADIXOR","ES_ES","es-es-default","1.0.0","7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","64918","847879","3525","826968","20911","21","869542","64933","41830466","560","0","359407144355","560","359407144915","0.000000","0","41830466","0.000000","0.999986612807","1.000000000000","0.999999998442","0.999999998442","0.999999999221","0.999989290217","0.999993306359","0.999997322533","0.999986612807","0.999993306381","0.999993305602","0.000000001558","","","","",""
|
||||
"SWEDISH_LUCENE_SWEDISH_LIGHT_STEM_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","PRIMARY_OUTPUT","12371","95181","68","95181","0","1","95181","22392","216573","24174","163074","4529259969","24174","4529284143","0.000534","163074","379647","42.954113","0.899587533801","0.570458873638","0.999994662733","0.999958661833","0.785226768185","0.806522249159","0.698178899216","0.615496764402","0.536309404414","0.716364216911","0.716346530489","0.000041338167","","","","",""
|
||||
"SWEDISH_LUCENE_SWEDISH_LIGHT_STEM_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","94954","68","94954","0","1","94954","22338","216064","24174","162804","4507680539","24174","4507704713","0.000536","162804","378868","42.971167","0.899374786670","0.570288332612","0.999994637182","0.999958523839","0.785141484897","0.806317266498","0.697987097524","0.615318019491","0.536083088115","0.716172428597","0.716154681700","0.000041476161","","","","",""
|
||||
"SWEDISH_LUCENE_SWEDISH_MINIMAL_STEM_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","PRIMARY_OUTPUT","12371","95181","68","95181","0","1","95181","23360","226201","19890","153446","4529264253","19890","4529284143","0.000439","153446","379647","40.418073","0.919176239684","0.595819274221","0.999995608578","0.999961733142","0.797907441399","0.829175864418","0.722989494005","0.640912596569","0.566157827686","0.740042512299","0.740026147317","0.000038266858","","","","",""
|
||||
"SWEDISH_LUCENE_SWEDISH_MINIMAL_STEM_FILTER","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","94954","68","94954","0","1","94954","23312","225644","19890","153224","4507684823","19890","4507704713","0.000441","153224","378868","40.442582","0.918992888969","0.595574184149","0.999995587555","0.999961599204","0.797784885852","0.828961560730","0.722752329429","0.640667890967","0.565867017088","0.739816490818","0.739800068479","0.000038400796","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","PRIMARY_OUTPUT","12371","95181","68","95181","0","1","95181","12330","362653","0","16994","4529284143","0","4529284143","0.000000","16994","379647","4.476263","1.000000000000","0.955237365237","1.000000000000","0.999996248287","0.977618682618","0.990714975312","0.977106291257","0.963866405208","0.955237365237","0.977362453359","0.977360619819","0.000003751713","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","ANY_CANDIDATE","12371","95181","68","92341","2840","5","98108","12371","","","","","0","4529284143","0.000000","0","379647","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","ALL_WORDS","ALL_CANDIDATES","12371","95181","68","92341","2840","5","98108","12371","379647","0","0","4529284143","0","4529284143","0.000000","0","379647","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","12342","94954","68","94954","0","1","94954","12301","361874","0","16994","4507704713","0","4507704713","0.000000","16994","378868","4.485467","1.000000000000","0.955145327660","1.000000000000","0.999996230327","0.977572663830","0.990695173580","0.977058139001","0.963791437487","0.955145327660","0.977315367556","0.977313525326","0.000003769673","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","12342","94954","68","92114","2840","5","97881","12342","","","","","0","4507704713","0.000000","0","378868","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"SWEDISH_RADIXOR","SV_SE","sv-se-default","1.0.0","d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","12342","94954","68","92114","2840","5","97881","12342","378868","0","0","4507704713","0","4507704713","0.000000","0","378868","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","PRIMARY_OUTPUT","1493","14150","4","14150","0","1","14150","2358","55865","28","9260","100039022","28","100039050","0.000028","9260","65125","14.218810","0.999499042814","0.857811900192","0.999999720109","0.999907216657","0.928905810151","0.967536898548","0.923251086615","0.882841908639","0.857443248968","0.925949336171","0.925906311882","0.000092783343","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ANY_CANDIDATE","1493","14150","4","12020","2130","6","16748","2912","","","","","0","100039050","0.000000","4946","65125","7.594626","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ALL_CANDIDATES","1493","14150","4","12020","2130","6","16748","2912","60179","59","4946","100038991","59","100039050","0.000059","4946","65125","7.594626","0.999020551811","0.924053742802","0.999999410230","0.999950002085","0.962026576516","0.983069619736","0.960075939472","0.938133305065","0.923217353952","0.960806265611","0.960782185122","0.000049997915","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14141","4","14141","0","1","14141","2356","55849","28","9260","99911733","28","99911761","0.000028","9260","65109","14.222304","0.999498899368","0.857776958639","0.999999719753","0.999907098512","0.928888339196","0.967527900297","0.923230787033","0.882812277712","0.857408231880","0.925930411026","0.925887332795","0.000092901488","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14141","4","12011","2130","6","16739","2910","","","","","0","99911761","0.000000","4946","65109","7.596492","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14141","4","12011","2130","6","16739","2910","60163","59","4946","99911702","59","99911761","0.000059","4946","65109","7.596492","0.999020291588","0.924035079636","0.999999409479","0.999949938421","0.962017244557","0.983065193450","0.960065745905","0.938117870130","0.923198502332","0.960796437699","0.960772326767","0.000050061579","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","PRIMARY_OUTPUT","1493","14150","4","14150","0","1","14150","2365","55849","28","9276","100039022","28","100039050","0.000028","9276","65125","14.243378","0.999498899368","0.857566218810","0.999999720109","0.999907056824","0.928782969460","0.967474266629","0.923108708947","0.882633693347","0.857197673169","0.925816662108","0.925773569870","0.000092943176","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ANY_CANDIDATE","1493","14150","4","12020","2130","6","16748","2919","","","","","0","100039050","0.000000","4962","65125","7.619194","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ALL_CANDIDATES","1493","14150","4","12020","2130","6","16748","2919","60163","59","4962","100038991","59","100039050","0.000059","4962","65125","7.619194","0.999020291588","0.923808061420","0.999999410230","0.999949842252","0.961903735825","0.983013793532","0.959943197683","0.937930668928","0.922971894944","0.960678405551","0.960654251285","0.000050157748","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14141","4","14141","0","1","14141","2356","55849","28","9260","99911733","28","99911761","0.000028","9260","65109","14.222304","0.999498899368","0.857776958639","0.999999719753","0.999907098512","0.928888339196","0.967527900297","0.923230787033","0.882812277712","0.857408231880","0.925930411026","0.925887332795","0.000092901488","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14141","4","12011","2130","6","16739","2910","","","","","0","99911761","0.000000","4946","65109","7.596492","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_MORFOLOGIK_DIRECT","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14141","4","12011","2130","6","16739","2910","60163","59","4946","99911702","59","99911761","0.000059","4946","65109","7.596492","0.999020291588","0.924035079636","0.999999409479","0.999949938421","0.962017244557","0.983065193450","0.960065745905","0.938117870130","0.923198502332","0.960796437699","0.960772326767","0.000050061579","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","PRIMARY_OUTPUT","1493","14150","4","14150","0","1","14150","1493","64580","0","545","100039050","0","100039050","0.000000","545","65125","0.836852","1.000000000000","0.991631477927","1.000000000000","0.999994555672","0.995815738964","0.998315014918","0.995798157357","0.993293958410","0.991631477927","0.995806948122","0.995804235618","0.000005444328","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ANY_CANDIDATE","1493","14150","4","14055","95","2","14245","1493","","","","","0","100039050","0.000000","0","65125","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","ALL_WORDS","ALL_CANDIDATES","1493","14150","4","14055","95","2","14245","1493","65125","0","0","100039050","0","100039050","0.000000","0","65125","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","1491","14141","4","14141","0","1","14141","1491","64564","0","545","99911761","0","99911761","0.000000","545","65109","0.837058","1.000000000000","0.991629421432","1.000000000000","0.999994548739","0.995814710716","0.998314598055","0.995797120449","0.993292307692","0.991629421432","0.995805915544","0.995803199587","0.000005451261","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","1491","14141","4","14046","95","2","14236","1491","","","","","0","99911761","0.000000","0","65109","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"UKRAINIAN_RADIXOR","UK_UA","uk-ua-default","1.0.0","cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","1491","14141","4","14046","95","2","14236","1491","65109","0","0","99911761","0","99911761","0.000000","0","65109","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","ALL_WORDS","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","802","6180","0","138","6229428","0","6229428","0.000000","138","6318","2.184236","1.000000000000","0.978157644824","1.000000000000","0.999977869528","0.989078822412","0.995553837232","0.988958233317","0.982449446776","0.978157644824","0.989018526027","0.989007571386","0.000022130472","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","ALL_WORDS","ANY_CANDIDATE","802","3532","0","3489","43","3","3578","802","","","","","0","6229428","0.000000","0","6318","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","ALL_WORDS","ALL_CANDIDATES","802","3532","0","3489","43","3","3578","802","6318","0","0","6229428","0","6229428","0.000000","0","6318","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","LOWERCASE_GROUPS_ONLY","PRIMARY_OUTPUT","802","3532","0","3532","0","1","3532","802","6180","0","138","6229428","0","6229428","0.000000","138","6318","2.184236","1.000000000000","0.978157644824","1.000000000000","0.999977869528","0.989078822412","0.995553837232","0.988958233317","0.982449446776","0.978157644824","0.989018526027","0.989007571386","0.000022130472","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","LOWERCASE_GROUPS_ONLY","ANY_CANDIDATE","802","3532","0","3489","43","3","3578","802","","","","","0","6229428","0.000000","0","6318","0.000000","","","","","","","","","","","","","","","","",""
|
||||
"YI_RADIXOR","YI","yi-default","1.0.0","f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3","LOWERCASE_GROUPS_ONLY","ALL_CANDIDATES","802","3532","0","3489","43","3","3578","802","6318","0","0","6229428","0","6229428","0.000000","0","6318","0.000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","1.000000000000","0.000000000000","","","","",""
|
||||
|
||||
|
@@ -1 +1 @@
|
||||
5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28 stemming-quality.csv
|
||||
edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8 stemming-quality.csv
|
||||
|
||||
@@ -10,7 +10,7 @@ two layers:
|
||||
candidate-policy, classification, and partition measurements from one checked result snapshot.
|
||||
|
||||
This structure keeps methodology separate from per-language result pages, while preserving all
|
||||
measured data and the command-class analysis for each Radixor language resource.
|
||||
measured data and the command-class analysis for each Radixor default model.
|
||||
|
||||
## Read This First
|
||||
|
||||
@@ -52,8 +52,8 @@ Open [Language Benchmark Pages](languages/index.md) for the complete language li
|
||||
|
||||
The English dictionary coverage benchmark shows the current contracted-trie operating curve. With
|
||||
the full English dictionary, Radixor reaches `97.478%` all-token exactness and `97.197%`
|
||||
changed-token exactness at `135.8 ns/token`. Even with a deterministic 10% dictionary slice, it
|
||||
keeps `92.868%` all-token exactness and `76.516%` changed-token exactness at `86.0 ns/token`.
|
||||
changed-token exactness at `98.0 ns/token`. Even with a deterministic 10% dictionary slice, it
|
||||
keeps `92.868%` all-token exactness and `76.516%` changed-token exactness at `80.6 ns/token`.
|
||||
|
||||
Those figures should not be reduced to a single speed badge. The professional interpretation is a
|
||||
quality/speed envelope: the amount and quality of dictionary knowledge affect stemming precision,
|
||||
@@ -61,83 +61,70 @@ while contracted tries reduce lookup cost in uniform regions of the compiled gra
|
||||
|
||||
## Quality versus performance
|
||||
|
||||
Each language page keeps exact-root accuracy, JMH latency, and pairwise linguistic-quality results in separate tables. No undocumented scalar combines them. The current repository checkout does not contain the dated machine-readable JMH CSV files named by the performance provenance page, so this revision preserves the existing performance tables but does not regenerate a cross-language Pareto frontier from rounded Markdown values. A defensible Pareto analysis requires the original unrounded JMH snapshot on the same hardware and JVM. Readers can still inspect the quality and speed dimensions side by side on every language page.
|
||||
Each language page keeps exact-root accuracy, JMH latency, and pairwise linguistic-quality results in separate tables. No undocumented scalar combines them. The 2026-07-23 language tables are generated from the unrounded JMH comparison report produced on the environment documented for this refresh. Readers should inspect the quality and speed dimensions side by side; no cross-language Pareto ranking is inferred from workloads with different dictionaries and token counts.
|
||||
|
||||
<!-- STEMMING-QUALITY-OVERVIEW:START -->
|
||||
|
||||
## Pairwise Quality Findings
|
||||
|
||||
The validated snapshot is a broad multilingual comparison covering the complete 20-language Radixor dictionary universe; 19 languages have existing benchmark pages. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim.
|
||||
The validated snapshot is a broad multilingual comparison covering the complete 20-language Radixor default-model universe, with one benchmark page per language. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim.
|
||||
|
||||
!!! success "Evidence-based primary-output result"
|
||||
Radixor achieved the highest balanced accuracy among the evaluated deterministic stemmers for every documented language in both `ALL_WORDS` and `LOWERCASE_GROUPS_ONLY`: **38 wins in 38 language-mode comparisons, with no exact first-place ties**. This statement is limited to the evaluated implementations, versions, dictionaries, adapters, and balanced-accuracy metric; it is not a universal claim about every stemming use case.
|
||||
Radixor achieved the highest balanced accuracy among the evaluated deterministic stemmers for every documented language in both `ALL_WORDS` and `LOWERCASE_GROUPS_ONLY`: **40 wins in 40 language-mode comparisons, with no exact first-place ties**. This statement is limited to the evaluated implementations, versions, dictionaries, adapters, and balanced-accuracy metric; it is not a universal claim about every stemming use case.
|
||||
|
||||
### Per-language winner matrix
|
||||
|
||||
| Language | Dictionary mode | Winner | Balanced accuracy | Runner-up | Difference | Exact tie | Deterministic stemmers |
|
||||
|---|---|---|---:|---|---:|---|---:|
|
||||
|Czech (`CS_CZ`)|ALL_WORDS|Radixor|0.996565|HUNSPELL CZECH LUCENE FILTER|0.142812638|no|3|
|
||||
|Czech (`CS_CZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997139|HUNSPELL CZECH LUCENE FILTER|0.144369049|no|3|
|
||||
|Danish (`DA_DK`)|ALL_WORDS|Radixor|0.996066|SNOWBALL DANISH LUCENE FILTER|0.058096771|no|3|
|
||||
|Danish (`DA_DK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996305|SNOWBALL DANISH DIRECT|0.058230346|no|3|
|
||||
|Dutch (`NL_NL`)|ALL_WORDS|Radixor|0.988661|SNOWBALL DUTCH DIRECT|0.261574077|no|4|
|
||||
|Dutch (`NL_NL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989040|SNOWBALL DUTCH DIRECT|0.258544404|no|4|
|
||||
|English (`US_UK`)|ALL_WORDS|Radixor|0.965159|ENGLISH LUCENE PORTER COPIED|0.010532535|no|11|
|
||||
|English (`US_UK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.965820|ENGLISH LUCENE PORTER COPIED|0.010920064|no|11|
|
||||
|Finnish (`FI_FI`)|ALL_WORDS|Radixor|0.984594|SNOWBALL FINNISH LUCENE FILTER|0.244241861|no|4|
|
||||
|Finnish (`FI_FI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.988068|SNOWBALL FINNISH DIRECT|0.249668284|no|4|
|
||||
|French (`FR_FR`)|ALL_WORDS|Radixor|0.956992|SNOWBALL FRENCH DIRECT|0.111730673|no|6|
|
||||
|French (`FR_FR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.957224|SNOWBALL FRENCH DIRECT|0.111809799|no|6|
|
||||
|German (`DE_DE`)|ALL_WORDS|Radixor|0.907901|GERMAN CISTEM|0.027131083|no|8|
|
||||
|German (`DE_DE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.966157|GERMAN CISTEM|0.050868631|no|8|
|
||||
|Hungarian (`HU_HU`)|ALL_WORDS|Radixor|0.995491|SNOWBALL HUNGARIAN LUCENE FILTER|0.172884951|no|4|
|
||||
|Hungarian (`HU_HU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996163|SNOWBALL HUNGARIAN DIRECT|0.174455479|no|4|
|
||||
|Italian (`IT_IT`)|ALL_WORDS|Radixor|0.996507|SNOWBALL ITALIAN DIRECT|0.130318040|no|4|
|
||||
|Italian (`IT_IT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996512|SNOWBALL ITALIAN DIRECT|0.130307087|no|4|
|
||||
|Norwegian Bokmal (`NB_NO`)|ALL_WORDS|Radixor|0.974783|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.099819340|no|5|
|
||||
|Norwegian Bokmal (`NB_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.975000|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.100008544|no|5|
|
||||
|Norwegian Nynorsk (`NN_NO`)|ALL_WORDS|Radixor|0.935777|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.076868986|no|3|
|
||||
|Norwegian Nynorsk (`NN_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.935853|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.076816096|no|3|
|
||||
|Persian (`FA_IR`)|ALL_WORDS|Radixor|0.974922|PERSIAN LUCENE PERSIAN STEM FILTER|0.472751327|no|2|
|
||||
|Persian (`FA_IR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.974922|PERSIAN LUCENE PERSIAN STEM FILTER|0.472751327|no|2|
|
||||
|Polish (`PL_PL`)|ALL_WORDS|Radixor|0.990388|POLISH LUCENE MORFOLOGIK FILTER|0.042233990|no|5|
|
||||
|Polish (`PL_PL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.990579|POLISH LUCENE MORFOLOGIK FILTER|0.042401633|no|5|
|
||||
|Portuguese (`PT_PT`)|ALL_WORDS|Radixor|0.998502|SNOWBALL PORTUGUESE DIRECT|0.059701750|no|6|
|
||||
|Portuguese (`PT_PT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998502|SNOWBALL PORTUGUESE DIRECT|0.059701750|no|6|
|
||||
|Russian (`RU_RU`)|ALL_WORDS|Radixor|0.989827|SNOWBALL RUSSIAN LUCENE FILTER|0.154951419|no|4|
|
||||
|Russian (`RU_RU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989852|SNOWBALL RUSSIAN DIRECT|0.154997931|no|4|
|
||||
|Spanish (`ES_ES`)|ALL_WORDS|Radixor|0.989295|SNOWBALL SPANISH LUCENE FILTER|0.336680479|no|7|
|
||||
|Spanish (`ES_ES`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989429|SNOWBALL SPANISH DIRECT|0.336708826|no|7|
|
||||
|Swedish (`SV_SE`)|ALL_WORDS|Radixor|0.974636|SNOWBALL SWEDISH DIRECT|0.167101450|no|5|
|
||||
|Swedish (`SV_SE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.974584|SNOWBALL SWEDISH DIRECT|0.166984893|no|5|
|
||||
|Ukrainian (`UK_UA`)|ALL_WORDS|Radixor|0.995343|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066574583|no|4|
|
||||
|Ukrainian (`UK_UA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995342|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066590889|no|4|
|
||||
|Yiddish (`YI`)|ALL_WORDS|Radixor|0.988241|SNOWBALL YIDDISH DIRECT|0.097253207|no|3|
|
||||
|Yiddish (`YI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.988241|SNOWBALL YIDDISH DIRECT|0.097253207|no|3|
|
||||
|Czech (`CS_CZ`)|ALL_WORDS|Radixor|0.996617|HUNSPELL CZECH LUCENE FILTER|0.142485045|no|3|
|
||||
|Czech (`CS_CZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997195|HUNSPELL CZECH LUCENE FILTER|0.144045088|no|3|
|
||||
|Danish (`DA_DK`)|ALL_WORDS|Radixor|0.996243|SNOWBALL DANISH LUCENE FILTER|0.058337376|no|3|
|
||||
|Danish (`DA_DK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996482|SNOWBALL DANISH DIRECT|0.058471663|no|3|
|
||||
|Dutch (`NL_NL`)|ALL_WORDS|Radixor|0.988733|SNOWBALL DUTCH DIRECT|0.261639748|no|4|
|
||||
|Dutch (`NL_NL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989114|SNOWBALL DUTCH DIRECT|0.258605347|no|4|
|
||||
|English (`US_UK`)|ALL_WORDS|Radixor|0.965537|ENGLISH LUCENE PORTER COPIED|0.010741250|no|11|
|
||||
|English (`US_UK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.966202|ENGLISH LUCENE PORTER COPIED|0.011138557|no|11|
|
||||
|Finnish (`FI_FI`)|ALL_WORDS|Radixor|0.984838|SNOWBALL FINNISH LUCENE FILTER|0.244558928|no|4|
|
||||
|Finnish (`FI_FI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.988242|SNOWBALL FINNISH DIRECT|0.249897933|no|4|
|
||||
|French (`FR_FR`)|ALL_WORDS|Radixor|0.958627|SNOWBALL FRENCH DIRECT|0.109964908|no|6|
|
||||
|French (`FR_FR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.958856|SNOWBALL FRENCH DIRECT|0.110030565|no|6|
|
||||
|German (`DE_DE`)|ALL_WORDS|Radixor|0.910445|GERMAN CISTEM|0.031918024|no|8|
|
||||
|German (`DE_DE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.966959|GERMAN CISTEM|0.052231588|no|8|
|
||||
|Hebrew (`HE_IL`)|ALL_WORDS|Radixor|0.986075|n/a|n/a|no|1|
|
||||
|Hebrew (`HE_IL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.986075|n/a|n/a|no|1|
|
||||
|Hungarian (`HU_HU`)|ALL_WORDS|Radixor|0.995555|SNOWBALL HUNGARIAN LUCENE FILTER|0.172591951|no|4|
|
||||
|Hungarian (`HU_HU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996227|SNOWBALL HUNGARIAN DIRECT|0.174150583|no|4|
|
||||
|Italian (`IT_IT`)|ALL_WORDS|Radixor|0.996651|SNOWBALL ITALIAN DIRECT|0.130360651|no|4|
|
||||
|Italian (`IT_IT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996656|SNOWBALL ITALIAN DIRECT|0.130349693|no|4|
|
||||
|Norwegian Bokmal (`NB_NO`)|ALL_WORDS|Radixor|0.976021|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101762107|no|5|
|
||||
|Norwegian Bokmal (`NB_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976240|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101954266|no|5|
|
||||
|Norwegian Nynorsk (`NN_NO`)|ALL_WORDS|Radixor|0.950991|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.082896791|no|3|
|
||||
|Norwegian Nynorsk (`NN_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.951104|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.082851757|no|3|
|
||||
|Persian (`FA_IR`)|ALL_WORDS|Radixor|0.976360|PERSIAN LUCENE PERSIAN STEM FILTER|0.474147508|no|2|
|
||||
|Persian (`FA_IR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976360|PERSIAN LUCENE PERSIAN STEM FILTER|0.474147508|no|2|
|
||||
|Polish (`PL_PL`)|ALL_WORDS|Radixor|0.991105|POLISH LUCENE MORFOLOGIK FILTER|0.042712804|no|5|
|
||||
|Polish (`PL_PL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991301|POLISH LUCENE MORFOLOGIK FILTER|0.042883749|no|5|
|
||||
|Portuguese (`PT_PT`)|ALL_WORDS|Radixor|0.998542|SNOWBALL PORTUGUESE DIRECT|0.059619854|no|6|
|
||||
|Portuguese (`PT_PT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998542|SNOWBALL PORTUGUESE DIRECT|0.059619854|no|6|
|
||||
|Russian (`RU_RU`)|ALL_WORDS|Radixor|0.990188|SNOWBALL RUSSIAN LUCENE FILTER|0.155623602|no|4|
|
||||
|Russian (`RU_RU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.990213|SNOWBALL RUSSIAN DIRECT|0.155670422|no|4|
|
||||
|Spanish (`ES_ES`)|ALL_WORDS|Radixor|0.989448|SNOWBALL SPANISH LUCENE FILTER|0.337009985|no|7|
|
||||
|Spanish (`ES_ES`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989580|SNOWBALL SPANISH DIRECT|0.337037572|no|7|
|
||||
|Swedish (`SV_SE`)|ALL_WORDS|Radixor|0.977619|SNOWBALL SWEDISH DIRECT|0.169075635|no|5|
|
||||
|Swedish (`SV_SE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.977573|SNOWBALL SWEDISH DIRECT|0.168961385|no|5|
|
||||
|Ukrainian (`UK_UA`)|ALL_WORDS|Radixor|0.995816|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066909929|no|4|
|
||||
|Ukrainian (`UK_UA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995815|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066926372|no|4|
|
||||
|Yiddish (`YI`)|ALL_WORDS|Radixor|0.989079|SNOWBALL YIDDISH DIRECT|0.097960961|no|3|
|
||||
|Yiddish (`YI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989079|SNOWBALL YIDDISH DIRECT|0.097960961|no|3|
|
||||
|
||||
### Secondary-metric trade-offs
|
||||
|
||||
Balanced-accuracy leadership does not imply leadership on every error trade-off. The table below lists all **15** deterministic primary-output language-mode-metric cases where a non-Radixor adapter has the best displayed value. Equal values are resolved by the authoritative row ordering and should be read as ties when the unrounded values are equal. Throughput leadership remains in the separate performance tables.
|
||||
Balanced-accuracy leadership does not imply leadership on every error trade-off. The table below lists all **0** deterministic primary-output language-mode-metric cases where a non-Radixor adapter has the best displayed value. Equal values are resolved by the authoritative row ordering and should be read as ties when the unrounded values are equal. Throughput leadership remains in the separate performance tables.
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Non-Radixor secondary-metric leaders</summary>
|
||||
|
||||
| Language | Dictionary mode | Metric | Leader | Value |
|
||||
|---|---|---|---|---:|
|
||||
|English|ALL_WORDS|Over-stemming percentage|ENGLISH LUCENE POSSESSIVE FILTER|0.000604|
|
||||
|English|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|ENGLISH LUCENE POSSESSIVE FILTER|0.000653|
|
||||
|French|ALL_WORDS|Over-stemming percentage|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|0.000177|
|
||||
|French|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|0.000166|
|
||||
|German|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|0.000188|
|
||||
|Italian|ALL_WORDS|Over-stemming percentage|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|0.000020|
|
||||
|Italian|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|0.000020|
|
||||
|Persian|ALL_WORDS|Over-stemming percentage|PERSIAN LUCENE PERSIAN STEM FILTER|0.002652|
|
||||
|Persian|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|PERSIAN LUCENE PERSIAN STEM FILTER|0.002652|
|
||||
|Portuguese|ALL_WORDS|Over-stemming percentage|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|0.000003|
|
||||
|Portuguese|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|0.000003|
|
||||
|Spanish|ALL_WORDS|Over-stemming percentage|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|0.000013|
|
||||
|Spanish|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|0.000012|
|
||||
|Ukrainian|ALL_WORDS|Over-stemming percentage|HUNSPELL UKRAINIAN LUCENE FILTER|0.000783|
|
||||
|Ukrainian|LOWERCASE_GROUPS_ONLY|Over-stemming percentage|HUNSPELL UKRAINIAN LUCENE FILTER|0.000784|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -149,7 +136,7 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa
|
||||
|
||||
| Stemmer | Evaluated languages | Wins | Exact first-place ties | Top-three placements | Average rank | Median rank |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19|19|0|19|1.000|1.000|
|
||||
|Radixor|20|20|0|20|1.000|1.000|
|
||||
|CZECH LUCENE CZECH STEM FILTER|1|0|0|1|3.000|3.000|
|
||||
|ENGLISH LUCENE KSTEM FILTER|1|0|0|0|8.000|8.000|
|
||||
|ENGLISH LUCENE MINIMAL FILTER|1|0|0|0|9.000|9.000|
|
||||
@@ -229,7 +216,7 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa
|
||||
|
||||
| Stemmer | Evaluated languages | Wins | Exact first-place ties | Top-three placements | Average rank | Median rank |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19|19|0|19|1.000|1.000|
|
||||
|Radixor|20|20|0|20|1.000|1.000|
|
||||
|CZECH LUCENE CZECH STEM FILTER|1|0|0|1|3.000|3.000|
|
||||
|ENGLISH LUCENE KSTEM FILTER|1|0|0|0|8.000|8.000|
|
||||
|ENGLISH LUCENE MINIMAL FILTER|1|0|0|0|9.000|9.000|
|
||||
@@ -308,17 +295,17 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa
|
||||
|
||||
### Radixor full-coverage aggregates
|
||||
|
||||
These aggregates cover all 19 documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking.
|
||||
These aggregates cover all 20 documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking.
|
||||
|
||||
| Dictionary mode | Languages | Macro balanced accuracy | Micro balanced accuracy | Micro precision | Micro recall | Micro F1 |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
|ALL_WORDS|19|0.978929|0.987664|0.975113|0.975328|0.975221|
|
||||
|LOWERCASE_GROUPS_ONLY|19|0.982354|0.989366|0.975322|0.978734|0.977025|
|
||||
|ALL_WORDS|20|0.980724|0.987976|0.999988|0.975952|0.987824|
|
||||
|LOWERCASE_GROUPS_ONLY|20|0.983891|0.989614|0.999992|0.979228|0.989501|
|
||||
|
||||
### Reproducible data
|
||||
|
||||
- [Machine-readable quality snapshot](data/stemming-quality.csv)
|
||||
- SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- [Linguistic quality methodology](reference/linguistic-quality.md)
|
||||
- [Tested stemmer inventory](reference/tested-stemmers.md)
|
||||
- [Reproducibility and raw data](reference/reproducibility.md)
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `CS_CZ` | 5,113 | 56,612 | 10,049 | 46,563 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `cs-cz-default` | `1.0.0` | `CS_CZ` | 5,113 | 56,612 | 10,049 | 46,563 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **56,612**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **56,612**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 675 | 1.192% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 22,681 | 40.064% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 14,980 | 26.461% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 10,109 | 17.857% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 8,167 | 14.426% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 711 | 1.256% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 22,643 | 39.997% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 15,007 | 26.509% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 10,046 | 17.745% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 8,205 | 14.493% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -34,15 +34,25 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene HunspellStemFilter | 84.850% | 82.269% | 96.806% | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene CzechStemFilter | 16.784% | 15.538% | 22.559% | Lucene Czech suffix stemmer implemented as a TokenFilter. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `czechRadixor` | 3.332 | 0.240 | 71.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 346.819 | 3.622 | 7448.4 | 104.091 | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 3.163 | 0.253 | 67.9 | 0.949 | Czech suffix stemmer implemented as a Lucene TokenFilter. |
|
||||
| Radixor | `czechRadixor` | 3.395 | 0.066 | 72.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 381.189 | 32.563 | 8186.5 | 112.265 | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 3.125 | 0.042 | 67.1 | 0.920 | Czech suffix stemmer implemented as a Lucene TokenFilter. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -56,29 +66,29 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `CS_CZ` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `CS_CZ` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/cs_cz/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `cs-cz-default`, loaded from classpath resource `org/egothor/stemmer/models/cs-cz-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996565** among 3 deterministic stemmers. The runner-up is `HUNSPELL CZECH LUCENE FILTER` at 0.853752, a difference of 0.142813. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.997139** among 3 deterministic stemmers. The runner-up is `HUNSPELL CZECH LUCENE FILTER` at 0.852770, a difference of 0.144369. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996617** among 3 deterministic stemmers. The runner-up is `HUNSPELL CZECH LUCENE FILTER` at 0.854132, a difference of 0.142485. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.997195** among 3 deterministic stemmers. The runner-up is `HUNSPELL CZECH LUCENE FILTER` at 0.853150, a difference of 0.144045. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996565|3867 / 1334876815 (0.000290%)|2073 / 301835 (0.686799%)|0.988432|0.990189|0.990191|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.853752|11408 / 1334876815 (0.000855%)|88283 / 301835 (29.248762%)|0.888560|0.810759|0.819499|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.793614|14480 / 1334876815 (0.001085%)|124586 / 301835 (41.276194%)|0.829234|0.718241|0.736765|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996617|0.000000%|0.676519%|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|0.854132|0.000691%|29.172837%|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|0.794343|0.000928%|41.130549%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -86,9 +96,9 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987264|0.993132|0.999997|0.996565|0.999996|0.000004|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.949289|0.707512|0.999991|0.853752|0.999925|0.000075|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.924477|0.587238|0.999989|0.793614|0.999896|0.000104|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.993235|1.000000|0.996617|0.999998|0.000002|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.958877|0.708272|0.999993|0.854132|0.999927|0.000073|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.935210|0.588695|0.999991|0.794343|0.999897|0.000103|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -96,19 +106,9 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988432|0.990189|0.991953|0.980569|0.990194|0.990191|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.888560|0.810759|0.745486|0.681745|0.819533|0.819499|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.829234|0.718241|0.633453|0.560356|0.736809|0.736765|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990187|0.998733|0.998686|0.998709|0.998709|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.810723|0.995777|0.952852|0.973842|0.973842|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.718192|0.993801|0.944977|0.968774|0.968774|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998640|0.996606|0.994581|0.993235|0.996612|0.996611|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.895506|0.814739|0.747335|0.687392|0.824103|0.824070|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.836710|0.722556|0.635811|0.565626|0.741992|0.741949|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -116,67 +116,42 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|299762|3867|2073|1334872948|3867 / 1334876815|2073 / 301835|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|213552|11408|88283|1334865407|11408 / 1334876815|88283 / 301835|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|177249|14480|124586|1334862335|14480 / 1334876815|124586 / 301835|
|
||||
|1|Radixor|PRIMARY_OUTPUT|298476|0|2033|1320705191|0 / 1320705191|2033 / 300509|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|212842|9128|87667|1320696063|9128 / 1320705191|87667 / 300509|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|176908|12256|123601|1320692935|12256 / 1320705191|123601 / 300509|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 1334876815 (0.000000%)|0 / 301835 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.871577|10102 / 1334876815 (0.000757%)|77523 / 301835 (25.683900%)|0.904855|0.836596|0.843258|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|HUNSPELL CZECH LUCENE FILTER|0.000650%|25.611213%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.956905|0.743161|0.999992|0.871577|0.999934|0.000066|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.904855|0.836596|0.777914|0.719094|0.843288|0.843258|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|301835|0|0|1334876815|0 / 1334876815|0 / 301835|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|224312|10102|77523|1334866713|10102 / 1334876815|77523 / 301835|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1320705191|0 / 300509|
|
||||
|HUNSPELL CZECH LUCENE FILTER|8582 / 1320705191|76964 / 300509|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|5850 / 1334876815 (0.000438%)|0 / 301835 (0.000000%)|0.984732|0.990402|0.990446|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.871575|13917 / 1334876815 (0.001043%)|77523 / 301835 (25.683900%)|0.893851|0.830687|0.836477|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|0.871940|0.000816%|25.611213%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -184,8 +159,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.980987|1.000000|0.999996|0.999998|0.999996|0.000004|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.941581|0.743161|0.999990|0.871575|0.999932|0.000068|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.954016|0.743888|0.999992|0.871940|0.999934|0.000066|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -193,17 +168,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.984732|0.990402|0.996139|0.980987|0.990448|0.990446|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.893851|0.830687|0.775861|0.710406|0.836509|0.836477|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.903001|0.835949|0.778167|0.718138|0.842426|0.842395|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -211,8 +177,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|301835|5850|0|1334870965|5850 / 1334876815|0 / 301835|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|224312|13917|77523|1334862898|13917 / 1334876815|77523 / 301835|
|
||||
|1|Radixor|ALL_CANDIDATES|300509|0|0|1320705191|0 / 1320705191|0 / 300509|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|223545|10775|76964|1320694416|10775 / 1320705191|76964 / 300509|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -222,22 +188,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|2073|3867|1983|596|1.153340%|4|52319|
|
||||
|HUNSPELL CZECH LUCENE FILTER|10760|1306|2509|3317|6.418840%|5|55596|
|
||||
|Radixor|2033|0|0|321|0.624501%|4|51739|
|
||||
|HUNSPELL CZECH LUCENE FILTER|10703|546|1647|3194|6.213887%|5|55179|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.997139|3863 / 1298544215 (0.000297%)|1709 / 298813 (0.571930%)|0.988580|0.990710|0.990714|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.852770|11239 / 1298544215 (0.000866%)|87986 / 298813 (29.445171%)|0.888009|0.809505|0.818403|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.791794|13950 / 1298544215 (0.001074%)|124426 / 298813 (41.640089%)|0.828709|0.715948|0.735055|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.997195|0.000000%|0.561033%|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|0.853150|0.000700%|29.369351%|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|0.792522|0.000918%|41.494586%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -245,9 +211,9 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987165|0.994281|0.999997|0.997139|0.999996|0.000004|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.949389|0.705548|0.999991|0.852770|0.999924|0.000076|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.925931|0.583599|0.999989|0.791794|0.999893|0.000107|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.994390|1.000000|0.997195|0.999999|0.000001|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.958957|0.706306|0.999993|0.853150|0.999925|0.000075|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.936557|0.585054|0.999991|0.792522|0.999895|0.000105|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -255,19 +221,9 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988580|0.990710|0.992849|0.981591|0.990716|0.990714|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.888009|0.809505|0.743753|0.679973|0.818437|0.818403|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.828709|0.715948|0.630198|0.557569|0.735100|0.735055|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990708|0.998726|0.999030|0.998878|0.998878|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.809467|0.995812|0.952394|0.973619|0.973619|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.715897|0.993897|0.944297|0.968463|0.968463|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998873|0.997187|0.995507|0.994390|0.997191|0.997190|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|0.894932|0.813466|0.745594|0.685581|0.822993|0.822960|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|0.836092|0.720206|0.632534|0.562751|0.740227|0.740184|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -275,67 +231,42 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|297104|3863|1709|1298540352|3863 / 1298544215|1709 / 298813|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|210827|11239|87986|1298532976|11239 / 1298544215|87986 / 298813|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|174387|13950|124426|1298530265|13950 / 1298544215|124426 / 298813|
|
||||
|1|Radixor|PRIMARY_OUTPUT|295818|0|1669|1284770069|0 / 1284770069|1669 / 297487|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|PRIMARY_OUTPUT|210117|8993|87370|1284761076|8993 / 1284770069|87370 / 297487|
|
||||
|3|CZECH LUCENE CZECH STEM FILTER|PRIMARY_OUTPUT|174046|11790|123441|1284758279|11790 / 1284770069|123441 / 297487|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 1298544215 (0.000000%)|0 / 298813 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.870432|10028 / 1298544215 (0.000772%)|77431 / 298813 (25.912862%)|0.904004|0.835052|0.841852|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|HUNSPELL CZECH LUCENE FILTER|0.000663%|25.840457%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.956666|0.740871|0.999992|0.870432|0.999933|0.000067|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|0.904004|0.835052|0.775874|0.716815|0.841883|0.841852|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|298813|0|0|1298544215|0 / 1298544215|0 / 298813|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ANY_CANDIDATE|221382|10028|77431|1298534187|10028 / 1298544215|77431 / 298813|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1284770069|0 / 297487|
|
||||
|HUNSPELL CZECH LUCENE FILTER|8518 / 1284770069|76872 / 297487|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|5782 / 1298544215 (0.000445%)|0 / 298813 (0.000000%)|0.984756|0.990418|0.990461|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.870430|13601 / 1298544215 (0.001047%)|77431 / 298813 (25.912862%)|0.893574|0.829463|0.835425|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|0.870794|0.000819%|25.840457%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -343,8 +274,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.981017|1.000000|0.999996|0.999998|0.999996|0.000004|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.942119|0.740871|0.999990|0.870430|0.999930|0.000070|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.954473|0.741595|0.999992|0.870794|0.999932|0.000068|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -352,17 +283,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.984756|0.990418|0.996145|0.981017|0.990463|0.990461|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.893574|0.829463|0.773936|0.708617|0.835457|0.835425|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|0.902651|0.834675|0.776220|0.716259|0.841328|0.841297|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -370,8 +292,8 @@ This mode contains **7 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|298813|5782|0|1298538433|5782 / 1298544215|0 / 298813|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|221382|13601|77431|1298530614|13601 / 1298544215|77431 / 298813|
|
||||
|1|Radixor|ALL_CANDIDATES|297487|0|0|1284770069|0 / 1284770069|0 / 297487|
|
||||
|2|HUNSPELL CZECH LUCENE FILTER|ALL_CANDIDATES|220615|10523|76872|1284759546|10523 / 1284770069|76872 / 297487|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -381,20 +303,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|1709|3863|1919|540|1.059488%|4|51543|
|
||||
|HUNSPELL CZECH LUCENE FILTER|10555|1211|2362|3237|6.351044%|5|54804|
|
||||
|Radixor|1669|0|0|269|0.530603%|4|50975|
|
||||
|HUNSPELL CZECH LUCENE FILTER|10498|475|1530|3117|6.148293%|5|54394|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -403,16 +325,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `CS_CZ`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,20 +8,20 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `DA_DK` | 4,179 | 32,256 | 8,356 | 23,900 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `da-dk-default` | `1.0.0` | `DA_DK` | 4,179 | 32,256 | 8,356 | 23,900 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **32,256**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **32,256**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 137 | 0.425% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 179 | 0.555% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 1,127 | 3.494% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 22,586 | 70.021% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 8,405 | 26.057% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 22,680 | 70.312% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 8,269 | 25.636% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 1 | 0.003% |
|
||||
|
||||
## Accuracy
|
||||
@@ -34,15 +34,25 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 55.509% | 54.159% | 59.371% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Official Snowball direct | 55.509% | 54.159% | 59.371% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `radixor[DANISH]` | 1.143 | 0.017 | 47.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[DANISH]` | 2.168 | 0.058 | 90.7 | 1.896 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 2.975 | 0.143 | 124.5 | 2.602 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `radixor[DANISH]` | 1.206 | 0.134 | 50.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[DANISH]` | 2.326 | 0.205 | 97.3 | 1.928 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 3.275 | 0.335 | 137.0 | 2.716 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -56,29 +66,29 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DA_DK` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DA_DK` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/da_dk/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `da-dk-default`, loaded from classpath resource `org/egothor/stemmer/models/da-dk-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996066** among 3 deterministic stemmers. The runner-up is `SNOWBALL DANISH LUCENE FILTER` at 0.937969, a difference of 0.058097. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996305** among 3 deterministic stemmers. The runner-up is `SNOWBALL DANISH DIRECT` at 0.938074, a difference of 0.058230. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996243** among 3 deterministic stemmers. The runner-up is `SNOWBALL DANISH LUCENE FILTER` at 0.937905, a difference of 0.058337. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996482** among 3 deterministic stemmers. The runner-up is `SNOWBALL DANISH DIRECT` at 0.938010, a difference of 0.058472. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996066|1165 / 394111186 (0.000296%)|707 / 89895 (0.786473%)|0.988108|0.989614|0.989615|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.937969|6507 / 394111186 (0.001651%)|11151 / 89895 (12.404472%)|0.913718|0.899181|0.899475|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.937903|6341 / 394111186 (0.001609%)|11163 / 89895 (12.417821%)|0.915090|0.899959|0.900279|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996243|0.000000%|0.751435%|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|0.937905|0.001273%|12.417638%|
|
||||
|3|SNOWBALL DANISH DIRECT|0.937839|0.001230%|12.431016%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -86,9 +96,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987106|0.992135|0.999997|0.996066|0.999995|0.000005|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.923672|0.875955|0.999983|0.937969|0.999955|0.000045|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.925464|0.875822|0.999984|0.937903|0.999956|0.000044|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.992486|1.000000|0.996243|0.999998|0.000002|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.940600|0.875824|0.999987|0.937905|0.999959|0.000041|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.942465|0.875690|0.999988|0.937839|0.999959|0.000041|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -96,19 +106,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988108|0.989614|0.991125|0.979442|0.989618|0.989615|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.913718|0.899181|0.885100|0.816830|0.899498|0.899475|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.915090|0.899959|0.885320|0.818114|0.900301|0.900279|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989612|0.998466|0.998719|0.998592|0.998592|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.899159|0.994053|0.978603|0.986268|0.986268|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.899937|0.994196|0.978579|0.986326|0.986326|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998488|0.996229|0.993979|0.992486|0.996236|0.996235|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.926889|0.907057|0.888055|0.829921|0.907634|0.907614|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.928307|0.907851|0.888277|0.831252|0.908464|0.908444|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -116,61 +116,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|89188|1165|707|394110021|1165 / 394111186|707 / 89895|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|78744|6507|11151|394104679|6507 / 394111186|11151 / 89895|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|78732|6341|11163|394104845|6341 / 394111186|11163 / 89895|
|
||||
|1|Radixor|PRIMARY_OUTPUT|89021|0|674|389687465|0 / 389687465|674 / 89695|
|
||||
|2|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|78557|4961|11138|389682504|4961 / 389687465|11138 / 89695|
|
||||
|3|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|78545|4795|11150|389682670|4795 / 389687465|11150 / 89695|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 394111186 (0.000000%)|0 / 89895 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|89895|0|0|394111186|0 / 394111186|0 / 89895|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 389687465|0 / 89695|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|1849 / 394111186 (0.000469%)|0 / 89895 (0.000000%)|0.983812|0.989820|0.989869|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -178,7 +156,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.979846|1.000000|0.999995|0.999998|0.999995|0.000005|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -186,15 +164,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.983812|0.989820|0.995903|0.979846|0.989872|0.989869|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -202,7 +172,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|89895|1849|0|394109337|1849 / 394111186|0 / 89895|
|
||||
|1|Radixor|ALL_CANDIDATES|89695|0|0|389687465|0 / 389687465|0 / 89695|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -212,21 +182,21 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|707|1165|684|323|1.150326%|3|28405|
|
||||
|Radixor|674|0|0|165|0.590953%|3|28087|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996305|1165 / 392820788 (0.000297%)|663 / 89740 (0.738801%)|0.988190|0.989843|0.989845|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.938074|6341 / 392820788 (0.001614%)|11113 / 89740 (12.383552%)|0.915093|0.900096|0.900410|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.938074|6341 / 392820788 (0.001614%)|11113 / 89740 (12.383552%)|0.915093|0.900096|0.900410|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996482|0.000000%|0.703596%|
|
||||
|2|SNOWBALL DANISH DIRECT|0.938010|0.001235%|12.396694%|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|0.938010|0.001235%|12.396694%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -234,9 +204,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987090|0.992612|0.999997|0.996305|0.999995|0.000005|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.925372|0.876164|0.999984|0.938074|0.999956|0.000044|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.925372|0.876164|0.999984|0.938074|0.999956|0.000044|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.992964|1.000000|0.996482|0.999998|0.000002|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.942392|0.876033|0.999988|0.938010|0.999959|0.000041|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.942392|0.876033|0.999988|0.938010|0.999959|0.000041|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -244,19 +214,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988190|0.989843|0.991503|0.979891|0.989847|0.989845|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.915093|0.900096|0.885583|0.818341|0.900432|0.900410|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.915093|0.900096|0.885583|0.818341|0.900432|0.900410|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989841|0.998463|0.998812|0.998637|0.998637|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.900074|0.994185|0.978644|0.986354|0.986354|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.900074|0.994185|0.978644|0.986354|0.986354|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998585|0.996470|0.994363|0.992964|0.996476|0.996475|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|0.928328|0.908002|0.888547|0.831505|0.908607|0.908587|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|0.928328|0.908002|0.888547|0.831505|0.908607|0.908587|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -264,61 +224,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|89077|1165|663|392819623|1165 / 392820788|663 / 89740|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|78627|6341|11113|392814447|6341 / 392820788|11113 / 89740|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|78627|6341|11113|392814447|6341 / 392820788|11113 / 89740|
|
||||
|1|Radixor|PRIMARY_OUTPUT|88910|0|630|388404335|0 / 388404335|630 / 89540|
|
||||
|2|SNOWBALL DANISH DIRECT|PRIMARY_OUTPUT|78440|4795|11100|388399540|4795 / 388404335|11100 / 89540|
|
||||
|3|SNOWBALL DANISH LUCENE FILTER|PRIMARY_OUTPUT|78440|4795|11100|388399540|4795 / 388404335|11100 / 89540|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 392820788 (0.000000%)|0 / 89740 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|89740|0|0|392820788|0 / 392820788|0 / 89740|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 388404335|0 / 89540|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|1849 / 392820788 (0.000471%)|0 / 89740 (0.000000%)|0.983784|0.989803|0.989852|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -326,7 +264,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.979812|1.000000|0.999995|0.999998|0.999995|0.000005|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -334,15 +272,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.983784|0.989803|0.995896|0.979812|0.989855|0.989852|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -350,7 +280,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|89740|1849|0|392818939|1849 / 392820788|0 / 89740|
|
||||
|1|Radixor|ALL_CANDIDATES|89540|0|0|388404335|0 / 388404335|0 / 89540|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -360,19 +290,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|663|1165|684|315|1.123676%|3|28351|
|
||||
|Radixor|630|0|0|157|0.563229%|3|28033|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -381,16 +311,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `DA_DK`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,13 +8,13 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `NL_NL` | 4,992 | 31,466 | 9,981 | 21,485 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `nl-nl-default` | `1.0.0` | `NL_NL` | 4,992 | 31,466 | 9,981 | 21,485 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **31,466**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **31,466**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
@@ -35,16 +35,26 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 15.954% | 8.992% | 30.939% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene SnowballFilter | 12.620% | 5.441% | 28.073% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `radixor[DUTCH]` | 1.331 | 0.114 | 61.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 22.760 | 1.387 | 1059.3 | 17.105 | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Official Snowball direct | `snowballDirect[DUTCH]` | 4.146 | 0.291 | 193.0 | 3.116 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 7.375 | 0.595 | 343.3 | 5.543 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `radixor[DUTCH]` | 1.410 | 0.139 | 65.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 24.183 | 2.889 | 1125.6 | 17.156 | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Official Snowball direct | `snowballDirect[DUTCH]` | 4.560 | 0.205 | 212.2 | 3.235 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 7.762 | 0.262 | 361.3 | 5.506 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -58,30 +68,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NL_NL` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NL_NL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/nl_nl/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `nl-nl-default`, loaded from classpath resource `org/egothor/stemmer/models/nl-nl-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.988661** among 4 deterministic stemmers. The runner-up is `SNOWBALL DUTCH DIRECT` at 0.727087, a difference of 0.261574. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989040** among 4 deterministic stemmers. The runner-up is `SNOWBALL DUTCH DIRECT` at 0.730495, a difference of 0.258544. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.988733** among 4 deterministic stemmers. The runner-up is `SNOWBALL DUTCH DIRECT` at 0.727093, a difference of 0.261640. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989114** among 4 deterministic stemmers. The runner-up is `SNOWBALL DUTCH DIRECT` at 0.730509, a difference of 0.258605. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988661|1214 / 350437960 (0.000346%)|1464 / 64566 (2.267447%)|0.980362|0.979221|0.979219|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.727087|4382 / 350437960 (0.001250%)|35241 / 64566 (54.581359%)|0.735353|0.596807|0.628557|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.643123|1333 / 350437960 (0.000380%)|46084 / 64566 (71.375027%)|0.642512|0.438061|0.516674|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.618497|1588 / 350437960 (0.000453%)|49264 / 64566 (76.300220%)|0.579068|0.375712|0.463333|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.988733|0.000000%|2.253364%|
|
||||
|2|SNOWBALL DUTCH DIRECT|0.727093|0.000870%|54.580443%|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|0.642844|0.000104%|71.431010%|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|0.617975|0.000221%|76.404861%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -89,10 +99,10 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.981124|0.977326|0.999997|0.988661|0.999992|0.000008|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.869997|0.454186|0.999987|0.727087|0.999887|0.000113|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.932728|0.286250|0.999996|0.643123|0.999865|0.000135|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.905980|0.236998|0.999995|0.618497|0.999855|0.000145|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.977466|1.000000|0.988733|0.999996|0.000004|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.907391|0.454196|0.999991|0.727093|0.999889|0.000111|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.981029|0.285690|0.999999|0.642844|0.999865|0.000135|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.952453|0.235951|0.999998|0.617975|0.999854|0.000146|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -100,21 +110,10 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.980362|0.979221|0.978083|0.959289|0.979223|0.979219|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.735353|0.596807|0.502190|0.425321|0.628602|0.628557|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.642512|0.438061|0.332316|0.280459|0.516714|0.516674|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.579068|0.375712|0.278062|0.231309|0.463374|0.463333|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.979217|0.997464|0.997003|0.997234|0.997234|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.596756|0.992815|0.917346|0.953590|0.953590|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.438012|0.996932|0.889026|0.939892|0.939892|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.375664|0.995828|0.888410|0.939057|0.939057|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995411|0.988605|0.981891|0.977466|0.988669|0.988667|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.756437|0.605372|0.504600|0.434074|0.641976|0.641934|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.659835|0.442513|0.332878|0.284120|0.529405|0.529368|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.592568|0.378209|0.277738|0.233204|0.474060|0.474022|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -122,68 +121,43 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|63102|1214|1464|350436746|1214 / 350437960|1464 / 64566|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|29325|4382|35241|350433578|4382 / 350437960|35241 / 64566|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|18482|1333|46084|350436627|1333 / 350437960|46084 / 64566|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|15302|1588|49264|350436372|1588 / 350437960|49264 / 64566|
|
||||
|1|Radixor|PRIMARY_OUTPUT|62985|0|1452|343168663|0 / 343168663|1452 / 64437|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|29267|2987|35170|343165676|2987 / 343168663|35170 / 64437|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|18409|356|46028|343168307|356 / 343168663|46028 / 64437|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|15204|759|49233|343167904|759 / 343168663|49233 / 64437|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 350437960 (0.000000%)|0 / 64566 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.665519|1164 / 350437960 (0.000332%)|43192 / 64566 (66.895889%)|0.690741|0.490770|0.560268|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|0.000096%|66.975495%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.948354|0.331041|0.999997|0.665519|0.999873|0.000127|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.690741|0.490770|0.380588|0.325179|0.560307|0.560268|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|64566|0|0|350437960|0 / 350437960|0 / 64566|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|21374|1164|43192|350436796|1164 / 350437960|43192 / 64566|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|330 / 343168663|43157 / 64437|
|
||||
|Radixor|0 / 343168663|0 / 64437|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999996|2651 / 350437960 (0.000756%)|0 / 64566 (0.000000%)|0.968198|0.979884|0.980078|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.665518|1738 / 350437960 (0.000496%)|43192 / 64566 (66.895889%)|0.680640|0.487557|0.553265|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|0.665122|0.000147%|66.975495%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -191,8 +165,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.960561|1.000000|0.999992|0.999996|0.999992|0.000008|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.924801|0.331041|0.999995|0.665518|0.999872|0.000128|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.976909|0.330245|0.999999|0.665122|0.999873|0.000127|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -200,17 +174,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.968198|0.979884|0.991855|0.960561|0.980082|0.980078|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.680640|0.487557|0.379812|0.322364|0.553306|0.553265|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.701991|0.493621|0.380638|0.327687|0.567996|0.567958|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -218,8 +183,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|64566|2651|0|350435309|2651 / 350437960|0 / 64566|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|21374|1738|43192|350436222|1738 / 350437960|43192 / 64566|
|
||||
|1|Radixor|ALL_CANDIDATES|64437|0|0|343168663|0 / 343168663|0 / 64437|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|21280|503|43157|343168160|503 / 343168663|43157 / 64437|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -229,23 +194,23 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|2892|169|405|1254|4.736186%|3|27763|
|
||||
|Radixor|1464|1214|1437|572|2.160366%|3|27061|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|2871|26|147|1199|4.576161%|3|27429|
|
||||
|Radixor|1452|0|0|296|1.129728%|3|26501|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989040|1214 / 329603856 (0.000368%)|1384 / 63147 (2.191711%)|0.980194|0.979401|0.979398|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.730495|4382 / 329603856 (0.001329%)|34036 / 63147 (53.899631%)|0.738412|0.602463|0.632953|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.645159|1310 / 329603856 (0.000397%)|44814 / 63147 (70.967742%)|0.646808|0.442880|0.520498|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.618546|1544 / 329603856 (0.000468%)|48175 / 63147 (76.290243%)|0.579362|0.375883|0.463566|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.989114|0.000000%|2.177156%|
|
||||
|2|SNOWBALL DUTCH DIRECT|0.730509|0.000926%|53.897299%|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|0.644879|0.000103%|71.024152%|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|0.618013|0.000222%|76.397220%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -253,10 +218,10 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.980723|0.978083|0.999996|0.989040|0.999992|0.000008|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.869167|0.461004|0.999987|0.730495|0.999883|0.000117|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.933310|0.290323|0.999996|0.645159|0.999860|0.000140|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.906515|0.237098|0.999995|0.618546|0.999849|0.000151|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.978228|1.000000|0.989114|0.999996|0.000004|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.906773|0.461027|0.999991|0.730509|0.999885|0.000115|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.982090|0.289758|0.999999|0.644879|0.999860|0.000140|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.954134|0.236028|0.999998|0.618013|0.999849|0.000151|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -264,21 +229,10 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.980194|0.979401|0.978610|0.959634|0.979402|0.979398|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.738412|0.602463|0.508789|0.431089|0.633000|0.632953|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.646808|0.442880|0.336718|0.284422|0.520539|0.520498|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.579362|0.375883|0.278182|0.231439|0.463608|0.463566|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.979397|0.997373|0.997139|0.997256|0.997256|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.602410|0.992557|0.918059|0.953856|0.953856|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.442829|0.996884|0.889061|0.939890|0.939890|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.375834|0.995817|0.887492|0.938539|0.938539|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995569|0.988994|0.982507|0.978228|0.989054|0.989052|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|0.759842|0.611269|0.511295|0.440164|0.646565|0.646521|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.664532|0.447489|0.337317|0.288235|0.533450|0.533411|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|0.593185|0.378440|0.277851|0.233380|0.474555|0.474515|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -286,68 +240,43 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|61763|1214|1384|329602642|1214 / 329603856|1384 / 63147|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|29111|4382|34036|329599474|4382 / 329603856|34036 / 63147|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|18333|1310|44814|329602546|1310 / 329603856|44814 / 63147|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|14972|1544|48175|329602312|1544 / 329603856|48175 / 63147|
|
||||
|1|Radixor|PRIMARY_OUTPUT|61646|0|1372|322555083|0 / 322555083|1372 / 63018|
|
||||
|2|SNOWBALL DUTCH DIRECT|PRIMARY_OUTPUT|29053|2987|33965|322552096|2987 / 322555083|33965 / 63018|
|
||||
|3|HUNSPELL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|18260|333|44758|322554750|333 / 322555083|44758 / 63018|
|
||||
|4|SNOWBALL DUTCH LUCENE FILTER|PRIMARY_OUTPUT|14874|715|48144|322554368|715 / 322555083|48144 / 63018|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 329603856 (0.000000%)|0 / 63147 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.667956|1141 / 329603856 (0.000346%)|41935 / 63147 (66.408539%)|0.695206|0.496187|0.564555|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|0.000095%|66.488940%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.948955|0.335915|0.999997|0.667956|0.999869|0.000131|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|0.695206|0.496187|0.385755|0.329953|0.564595|0.564555|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|63147|0|0|329603856|0 / 329603856|0 / 63147|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ANY_CANDIDATE|21212|1141|41935|329602715|1141 / 329603856|41935 / 63147|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|307 / 322555083|41900 / 63018|
|
||||
|Radixor|0 / 322555083|0 / 63018|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999996|2651 / 329603856 (0.000804%)|0 / 63147 (0.000000%)|0.967506|0.979441|0.979644|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.667955|1712 / 329603856 (0.000519%)|41935 / 63147 (66.408539%)|0.684952|0.492895|0.557477|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|0.667555|0.000148%|66.488940%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -355,8 +284,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.959710|1.000000|0.999992|0.999996|0.999992|0.000008|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.925318|0.335915|0.999995|0.667955|0.999868|0.000132|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.977912|0.335111|0.999999|0.667555|0.999869|0.000131|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -364,17 +293,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.967506|0.979441|0.991674|0.959710|0.979648|0.979644|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.684952|0.492895|0.384956|0.327048|0.557519|0.557477|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|0.706770|0.499167|0.385834|0.332593|0.572458|0.572419|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -382,8 +302,8 @@ This mode contains **8 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|63147|2651|0|329601205|2651 / 329603856|0 / 63147|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|21212|1712|41935|329602144|1712 / 329603856|41935 / 63147|
|
||||
|1|Radixor|ALL_CANDIDATES|63018|0|0|322555083|0 / 322555083|0 / 63018|
|
||||
|2|HUNSPELL DUTCH LUCENE FILTER|ALL_CANDIDATES|21118|477|41900|322554606|477 / 322555083|41900 / 63018|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -393,20 +313,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|2879|169|402|1186|4.618740%|3|26896|
|
||||
|Radixor|1384|1214|1437|549|2.138017%|3|26239|
|
||||
|HUNSPELL DUTCH LUCENE FILTER|2858|26|144|1131|4.452405%|3|26562|
|
||||
|Radixor|1372|0|0|273|1.074719%|3|25679|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -415,16 +335,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `NL_NL`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `US_UK` | 396,939 | 1,004,374 | 793,874 | 210,500 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `us-uk-default` | `1.0.0` | `US_UK` | 396,939 | 1,004,374 | 793,874 | 210,500 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **1,004,374**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **1,004,374**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 28 | 0.003% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 22,493 | 2.240% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 186,764 | 18.595% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 795,024 | 79.156% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 65 | 0.006% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 73 | 0.007% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 22,481 | 2.238% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 202,637 | 20.175% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 779,106 | 77.571% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 77 | 0.008% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -42,23 +42,31 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Snowball original Porter | 39.529% | 46.179% | 37.766% | Classic Porter rule-based suffix stemmer. |
|
||||
| Paice/Husk Lancaster | 28.055% | 37.039% | 25.673% | Aggressive Paice/Husk rule stemmer that often produces shorter stems. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `radixorUsUkProfiPreferredStem` | 21.987 | 8.707 | 104.5 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. |
|
||||
| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 24.539 | 1.515 | 116.6 | 1.116 | Possessive-ending remover only; not a full stemmer. |
|
||||
| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 22.702 | 1.195 | 107.8 | 1.032 | Narrow plural reduction filter; not a full stemmer. |
|
||||
| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 24.696 | 13.235 | 117.3 | 1.123 | Benchmark-only generated copy of Lucene package-private Porter implementation. |
|
||||
| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 23.121 | 12.528 | 109.8 | 1.052 | Apache OpenNLP Porter implementation. |
|
||||
| Snowball original Porter | `snowballOriginalPorter` | 38.904 | 10.353 | 184.8 | 1.769 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. |
|
||||
| Lucene PorterStemFilter | `lucenePorterStemFilter` | 37.021 | 1.196 | 175.9 | 1.684 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. |
|
||||
| Lucene KStemFilter | `luceneKStemFilter` | 51.640 | 2.591 | 245.3 | 2.349 | Krovetz-style English TokenFilter; broader than minimal suffix filters. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 79.785 | 1.347 | 379.0 | 3.629 | Benchmark-only English Hunspell comparison using the benchmark Hunspell corpus. |
|
||||
| Snowball English / Porter2 | `snowballEnglishPorter2` | 52.437 | 0.773 | 249.1 | 2.385 | Porter2 suffix-rule stemmer, distinct from original Porter. |
|
||||
| Paice/Husk Lancaster | `paiceHuskLancaster` | 141.556 | 12.324 | 672.5 | 6.438 | Aggressive rule-based English stemmer. |
|
||||
| Radixor | `radixorUsUkProfiPreferredStem` | 17.489 | 1.380 | 83.1 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. |
|
||||
| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 17.151 | 0.215 | 81.5 | 0.981 | Possessive-ending remover only; not a full stemmer. |
|
||||
| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 18.522 | 0.152 | 88.0 | 1.059 | Narrow plural reduction filter; not a full stemmer. |
|
||||
| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 17.651 | 0.129 | 83.9 | 1.009 | Benchmark-only generated copy of Lucene package-private Porter implementation. |
|
||||
| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 17.681 | 0.139 | 84.0 | 1.011 | Apache OpenNLP Porter implementation. |
|
||||
| Snowball original Porter | `snowballOriginalPorter` | 33.290 | 1.916 | 158.1 | 1.904 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. |
|
||||
| Lucene PorterStemFilter | `lucenePorterStemFilter` | 32.408 | 0.412 | 154.0 | 1.853 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. |
|
||||
| Lucene KStemFilter | `luceneKStemFilter` | 45.877 | 0.425 | 217.9 | 2.623 | Krovetz-style English TokenFilter; broader than minimal suffix filters. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 76.852 | 1.028 | 365.1 | 4.394 | Benchmark-only English Hunspell comparison using the benchmark Hunspell corpus. |
|
||||
| Snowball English / Porter2 | `snowballEnglishPorter2` | 46.568 | 2.414 | 221.2 | 2.663 | Porter2 suffix-rule stemmer, distinct from original Porter. |
|
||||
| Paice/Husk Lancaster | `paiceHuskLancaster` | 144.951 | 2.710 | 688.6 | 8.288 | Aggressive rule-based English stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -72,37 +80,37 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `US_UK` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `US_UK` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/us_uk/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `us-uk-default`, loaded from classpath resource `org/egothor/stemmer/models/us-uk-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.965159** among 11 deterministic stemmers. The runner-up is `ENGLISH LUCENE PORTER COPIED` at 0.954627, a difference of 0.010533. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.965820** among 11 deterministic stemmers. The runner-up is `ENGLISH LUCENE PORTER COPIED` at 0.954900, a difference of 0.010920. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.965537** among 11 deterministic stemmers. The runner-up is `ENGLISH LUCENE PORTER COPIED` at 0.954796, a difference of 0.010741. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.966202** among 11 deterministic stemmers. The runner-up is `ENGLISH LUCENE PORTER COPIED` at 0.955064, a difference of 0.011139. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.965159|1149886 / 184490451771 (0.000623%)|21869 / 313870 (6.967534%)|0.240076|0.332621|0.434052|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.954627|1557406 / 184490451771 (0.000844%)|28480 / 313870 (9.073820%)|0.185679|0.264659|0.375252|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.954627|1557406 / 184490451771 (0.000844%)|28480 / 313870 (9.073820%)|0.185679|0.264659|0.375252|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.954627|1557406 / 184490451771 (0.000844%)|28480 / 313870 (9.073820%)|0.185679|0.264659|0.375252|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.954537|1566711 / 184490451771 (0.000849%)|28536 / 313870 (9.091662%)|0.184753|0.263477|0.374240|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.954490|1555293 / 184490451771 (0.000843%)|28566 / 313870 (9.101220%)|0.185835|0.264849|0.375363|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.952394|3062661 / 184490451771 (0.001660%)|29879 / 313870 (9.519546%)|0.103643|0.155164|0.277089|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.878441|1368501 / 184490451771 (0.000742%)|76305 / 313870 (24.311020%)|0.176284|0.247472|0.334598|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.718599|1122264 / 184490451771 (0.000608%)|176645 / 313870 (56.279670%)|0.128204|0.174436|0.218251|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.573277|1981986 / 184490451771 (0.001074%)|267868 / 313870 (85.343614%)|0.027298|0.039287|0.057655|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.500008|1115154 / 184490451771 (0.000604%)|313863 / 313870 (99.997770%)|0.000007|0.000010|0.000009|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.965537|<0.000001%|6.892502%|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|0.954796|0.000207%|9.040545%|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|0.954796|0.000207%|9.040545%|
|
||||
|4|ENGLISH OPENNLP PORTER|0.954796|0.000207%|9.040545%|
|
||||
|5|ENGLISH SNOWBALL PORTER2|0.954708|0.000212%|9.058097%|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|0.954659|0.000206%|9.067990%|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|0.952535|0.000960%|9.492110%|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|0.878645|0.000110%|24.270875%|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|0.718958|0.000001%|56.208454%|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|0.573139|0.000012%|85.372182%|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|0.500011|<0.000001%|99.997766%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -110,17 +118,17 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.202513|0.930325|0.999994|0.965159|0.999994|0.000006|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.154868|0.909262|0.999992|0.954627|0.999991|0.000009|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.154868|0.909262|0.999992|0.954627|0.999991|0.000009|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.154868|0.909262|0.999992|0.954627|0.999991|0.000009|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.154064|0.909083|0.999992|0.954537|0.999991|0.000009|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.155006|0.908988|0.999992|0.954490|0.999991|0.000009|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.084858|0.904805|0.999983|0.952394|0.999983|0.000017|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.147917|0.756890|0.999993|0.878441|0.999992|0.000008|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.108953|0.437203|0.999994|0.718599|0.999993|0.000007|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.022684|0.146564|0.999989|0.573277|0.999988|0.000012|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000006|0.000022|0.999994|0.500008|0.999992|0.000008|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999990|0.931075|1.000000|0.965537|1.000000|0.000000|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.440121|0.909595|0.999998|0.954796|0.999998|0.000002|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.440121|0.909595|0.999998|0.954796|0.999998|0.000002|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.440121|0.909595|0.999998|0.954796|0.999998|0.000002|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.434174|0.909419|0.999998|0.954708|0.999998|0.000002|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.441440|0.909320|0.999998|0.954659|0.999998|0.000002|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.144284|0.905079|0.999990|0.952535|0.999990|0.000010|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.551014|0.757291|0.999999|0.878645|0.999998|0.000002|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.989894|0.437915|1.000000|0.718958|0.999999|0.000001|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.681277|0.146278|1.000000|0.573139|0.999998|0.000002|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.148936|0.000022|1.000000|0.500011|0.999998|0.000002|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -128,35 +136,17 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.240076|0.332621|0.541270|0.199487|0.434054|0.434052|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.185679|0.264659|0.460563|0.152511|0.375254|0.375252|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.185679|0.264659|0.460563|0.152511|0.375254|0.375252|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.185679|0.264659|0.460563|0.152511|0.375254|0.375252|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.184753|0.263477|0.459102|0.151727|0.374242|0.374240|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.185835|0.264849|0.460751|0.152637|0.375365|0.375363|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.103643|0.155164|0.308543|0.084107|0.277092|0.277089|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.176284|0.247472|0.415099|0.141208|0.334600|0.334598|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.128204|0.174436|0.272816|0.095552|0.218253|0.218251|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.027298|0.039287|0.070051|0.020037|0.057659|0.057655|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000007|0.000010|0.000015|0.000005|0.000012|0.000009|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.332619|0.994215|0.997770|0.995989|0.995989|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.264656|0.969648|0.997199|0.983231|0.983231|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.264656|0.969648|0.997199|0.983231|0.983231|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.264656|0.969648|0.997199|0.983231|0.983231|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.263474|0.969037|0.997182|0.982908|0.982908|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.264847|0.969891|0.997193|0.983353|0.983353|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.155162|0.937768|0.996600|0.966289|0.966289|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.247470|0.980687|0.992108|0.986364|0.986364|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.174433|0.995202|0.981174|0.988138|0.988138|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.039284|0.993096|0.963677|0.978166|0.978166|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000007|0.995789|0.958019|0.976539|0.976539|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.985403|0.964303|0.944087|0.931066|0.964917|0.964917|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.490783|0.593208|0.749662|0.421675|0.632717|0.632716|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.490783|0.593208|0.749662|0.421675|0.632717|0.632716|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.490783|0.593208|0.749662|0.421675|0.632717|0.632716|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.484849|0.587747|0.746086|0.416176|0.628368|0.628367|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.492079|0.594348|0.750277|0.422827|0.633570|0.633569|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.173443|0.248891|0.440518|0.142133|0.361370|0.361368|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.582762|0.637891|0.704541|0.468312|0.645971|0.645970|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.790591|0.607210|0.492883|0.435966|0.658399|0.658399|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.393465|0.240844|0.173533|0.136909|0.315683|0.315683|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000112|0.000045|0.000028|0.000022|0.001824|0.001824|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -164,75 +154,50 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|292001|1149886|21869|184489301885|1149886 / 184490451771|21869 / 313870|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|285390|1557406|28480|184488894365|1557406 / 184490451771|28480 / 313870|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|285390|1557406|28480|184488894365|1557406 / 184490451771|28480 / 313870|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|285390|1557406|28480|184488894365|1557406 / 184490451771|28480 / 313870|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|285334|1566711|28536|184488885060|1566711 / 184490451771|28536 / 313870|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|285304|1555293|28566|184488896478|1555293 / 184490451771|28566 / 313870|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|283991|3062661|29879|184487389110|3062661 / 184490451771|29879 / 313870|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|237565|1368501|76305|184489083270|1368501 / 184490451771|76305 / 313870|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|137225|1122264|176645|184489329507|1122264 / 184490451771|176645 / 313870|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|46002|1981986|267868|184488469785|1981986 / 184490451771|267868 / 313870|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|7|1115154|313863|184489336617|1115154 / 184490451771|313863 / 313870|
|
||||
|1|Radixor|PRIMARY_OUTPUT|291757|3|21598|175199424127|3 / 175199424130|21598 / 313355|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|285026|362583|28329|175199061547|362583 / 175199424130|28329 / 313355|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|285026|362583|28329|175199061547|362583 / 175199424130|28329 / 313355|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|285026|362583|28329|175199061547|362583 / 175199424130|28329 / 313355|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|284971|371381|28384|175199052749|371381 / 175199424130|28384 / 313355|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|284940|360538|28415|175199063592|360538 / 175199424130|28415 / 313355|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|283611|1682034|29744|175197742096|1682034 / 175199424130|29744 / 313355|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|237301|193361|76054|175199230769|193361 / 175199424130|76054 / 313355|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|137223|1401|176132|175199422729|1401 / 175199424130|176132 / 313355|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|45837|21444|267518|175199402686|21444 / 175199424130|267518 / 313355|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|7|40|313348|175199424090|40 / 175199424130|313348 / 313355|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999976|12 / 184490451771 (0.000000%)|15 / 313870 (0.004779%)|0.999960|0.999957|0.999957|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.581603|1978852 / 184490451771 (0.001073%)|262641 / 313870 (83.678274%)|0.030370|0.043712|0.064174|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.004787%|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|0.000012%|83.719424%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999962|0.999952|1.000000|0.999976|1.000000|0.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.025235|0.163217|0.999989|0.581603|0.999988|0.000012|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999960|0.999957|0.999954|0.999914|0.999957|0.999957|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.030370|0.043712|0.077961|0.022344|0.064178|0.064174|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|313855|12|15|184490451759|12 / 184490451771|15 / 313870|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|51229|1978852|262641|184488472919|1978852 / 184490451771|262641 / 313870|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 175199424130|15 / 313355|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|20367 / 175199424130|262339 / 313355|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999945|11482166 / 184490451771 (0.006224%)|15 / 313870 (0.004779%)|0.033039|0.051834|0.163107|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.581603|2008917 / 184490451771 (0.001089%)|262641 / 313870 (83.678274%)|0.029943|0.043158|0.063704|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.999976|<0.000001%|0.004787%|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|0.581403|0.000022%|83.719424%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -240,8 +205,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.026607|0.999952|0.999938|0.999945|0.999938|0.000062|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.024867|0.163217|0.999989|0.581603|0.999988|0.000012|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999825|0.999952|1.000000|0.999976|1.000000|0.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.568132|0.162806|1.000000|0.581403|0.999998|0.000002|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -249,17 +214,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.033039|0.051834|0.120237|0.026607|0.163112|0.163107|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.029943|0.043158|0.077254|0.022055|0.063708|0.063704|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999850|0.999888|0.999927|0.999777|0.999888|0.999888|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.379279|0.253086|0.189902|0.144876|0.304130|0.304130|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -267,8 +223,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|313855|11482166|15|184478969605|11482166 / 184490451771|15 / 313870|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|51229|2008917|262641|184488442854|2008917 / 184490451771|262641 / 313870|
|
||||
|1|Radixor|ALL_CANDIDATES|313340|55|15|175199424075|55 / 175199424130|15 / 313355|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|51016|38780|262339|175199385350|38780 / 175199424130|262339 / 313355|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -278,30 +234,30 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|21854|1149874|10332280|29208|4.808384%|1355|2838145|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|5227|3134|26931|6837|1.125545%|4|614296|
|
||||
|Radixor|21583|3|52|13718|2.317441%|1355|607918|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|5179|1077|17336|5736|0.969007%|4|597698|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.965820|1148489 / 170474840204 (0.000674%)|21319 / 311891 (6.835401%)|0.239424|0.331902|0.433722|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.954900|1552702 / 170474840204 (0.000911%)|28130 / 311891 (9.019177%)|0.185277|0.264166|0.374937|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.954900|1552702 / 170474840204 (0.000911%)|28130 / 311891 (9.019177%)|0.185277|0.264166|0.374937|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.954900|1552702 / 170474840204 (0.000911%)|28130 / 311891 (9.019177%)|0.185277|0.264166|0.374937|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.954850|1561891 / 170474840204 (0.000916%)|28161 / 311891 (9.029116%)|0.184375|0.263016|0.373964|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.954762|1550615 / 170474840204 (0.000910%)|28216 / 311891 (9.046750%)|0.185431|0.264353|0.375045|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.952710|3045870 / 170474840204 (0.001787%)|29493 / 311891 (9.456188%)|0.103633|0.155157|0.277170|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.880820|1367069 / 170474840204 (0.000802%)|74340 / 311891 (23.835250%)|0.176477|0.247899|0.335789|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.719516|1120871 / 170474840204 (0.000657%)|174959 / 311891 (56.096200%)|0.128139|0.174470|0.218621|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.573619|1978041 / 170474840204 (0.001160%)|265965 / 311891 (85.274984%)|0.027312|0.039323|0.057799|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.500005|1113773 / 170474840204 (0.000653%)|311886 / 311891 (99.998397%)|0.000005|0.000007|0.000005|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.966202|<0.000001%|6.759543%|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|0.955064|0.000222%|8.987032%|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|0.955064|0.000222%|8.987032%|
|
||||
|4|ENGLISH OPENNLP PORTER|0.955064|0.000222%|8.987032%|
|
||||
|5|ENGLISH SNOWBALL PORTER2|0.955016|0.000228%|8.996666%|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|0.954926|0.000221%|9.014651%|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|0.952850|0.001032%|9.428933%|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|0.881028|0.000120%|23.794246%|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|0.719875|0.000001%|56.025075%|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|0.573484|0.000012%|85.303261%|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|0.500008|<0.000001%|99.998394%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -309,17 +265,17 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.201918|0.931646|0.999993|0.965820|0.999993|0.000007|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.154515|0.909808|0.999991|0.954900|0.999991|0.000009|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.154515|0.909808|0.999991|0.954900|0.999991|0.000009|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.154515|0.909808|0.999991|0.954900|0.999991|0.000009|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.153731|0.909709|0.999991|0.954850|0.999991|0.000009|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.154651|0.909532|0.999991|0.954762|0.999991|0.000009|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.084848|0.905438|0.999982|0.952710|0.999982|0.000018|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.148042|0.761647|0.999992|0.880820|0.999992|0.000008|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.108866|0.439038|0.999993|0.719516|0.999992|0.000008|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.022691|0.147250|0.999988|0.573619|0.999987|0.000013|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000004|0.000016|0.999993|0.500005|0.999992|0.000008|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999990|0.932405|1.000000|0.966202|1.000000|0.000000|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.440920|0.910130|0.999998|0.955064|0.999998|0.000002|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.440920|0.910130|0.999998|0.955064|0.999998|0.000002|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.440920|0.910130|0.999998|0.955064|0.999998|0.000002|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.435017|0.910033|0.999998|0.955016|0.999998|0.000002|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.442235|0.909853|0.999998|0.954926|0.999998|0.000002|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.144700|0.905711|0.999990|0.952850|0.999990|0.000010|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.551013|0.762058|0.999999|0.881028|0.999998|0.000002|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.989965|0.439749|1.000000|0.719875|0.999999|0.000001|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.700136|0.146967|1.000000|0.573484|0.999998|0.000002|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.121951|0.000016|1.000000|0.500008|0.999998|0.000002|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -327,35 +283,17 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.239424|0.331902|0.540775|0.198970|0.433723|0.433722|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.185277|0.264166|0.460049|0.152184|0.374939|0.374937|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.185277|0.264166|0.460049|0.152184|0.374939|0.374937|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.185277|0.264166|0.460049|0.152184|0.374939|0.374937|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.184375|0.263016|0.458637|0.151421|0.373966|0.373964|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.185431|0.264353|0.460234|0.152308|0.375047|0.375045|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.103633|0.155157|0.308576|0.084103|0.277173|0.277170|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.176477|0.247899|0.416437|0.141487|0.335791|0.335789|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.128139|0.174470|0.273277|0.095572|0.218624|0.218621|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.027312|0.039323|0.070190|0.020056|0.057804|0.057799|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000005|0.000007|0.000011|0.000004|0.000008|0.000005|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.331900|0.993959|0.997731|0.995842|0.995842|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.264164|0.968645|0.997109|0.982671|0.982671|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.264164|0.968645|0.997109|0.982671|0.982671|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.264164|0.968645|0.997109|0.982671|0.982671|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.263014|0.968020|0.997096|0.982343|0.982343|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.264351|0.968894|0.997102|0.982795|0.982795|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.155154|0.936077|0.996487|0.965338|0.965338|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.247897|0.979822|0.991991|0.985869|0.985869|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.174467|0.994994|0.980520|0.987704|0.987704|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.039320|0.993066|0.962317|0.977450|0.977450|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000004|0.995605|0.956423|0.975621|0.975621|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.985700|0.965015|0.945181|0.932396|0.965606|0.965606|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|0.491609|0.594049|0.750417|0.422524|0.633478|0.633477|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|0.491609|0.594049|0.750417|0.422524|0.633478|0.633477|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|0.491609|0.594049|0.750417|0.422524|0.633478|0.633477|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|0.485725|0.588647|0.746915|0.417080|0.629190|0.629189|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|0.492900|0.595181|0.751027|0.423671|0.634326|0.634325|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|0.173928|0.249533|0.441413|0.142553|0.362017|0.362015|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|0.583322|0.639575|0.707836|0.470129|0.648000|0.647999|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|0.791820|0.608984|0.494744|0.437798|0.659800|0.659800|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|0.399444|0.242939|0.174549|0.138264|0.320776|0.320775|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|0.000080|0.000032|0.000020|0.000016|0.001399|0.001399|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -363,75 +301,50 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|290572|1148489|21319|170473691715|1148489 / 170474840204|21319 / 311891|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|283761|1552702|28130|170473287502|1552702 / 170474840204|28130 / 311891|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|283761|1552702|28130|170473287502|1552702 / 170474840204|28130 / 311891|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|283761|1552702|28130|170473287502|1552702 / 170474840204|28130 / 311891|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|283730|1561891|28161|170473278313|1561891 / 170474840204|28161 / 311891|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|283675|1550615|28216|170473289589|1550615 / 170474840204|28216 / 311891|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|282398|3045870|29493|170471794334|3045870 / 170474840204|29493 / 311891|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|237551|1367069|74340|170473473135|1367069 / 170474840204|74340 / 311891|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|136932|1120871|174959|170473719333|1120871 / 170474840204|174959 / 311891|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|45926|1978041|265965|170472862163|1978041 / 170474840204|265965 / 311891|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|5|1113773|311886|170473726431|1113773 / 170474840204|311886 / 311891|
|
||||
|1|Radixor|PRIMARY_OUTPUT|290334|3|21048|161561989635|3 / 161561989638|21048 / 311382|
|
||||
|2|ENGLISH LUCENE PORTER COPIED|PRIMARY_OUTPUT|283398|359344|27984|161561630294|359344 / 161561989638|27984 / 311382|
|
||||
|3|ENGLISH LUCENE PORTER FILTER|PRIMARY_OUTPUT|283398|359344|27984|161561630294|359344 / 161561989638|27984 / 311382|
|
||||
|4|ENGLISH OPENNLP PORTER|PRIMARY_OUTPUT|283398|359344|27984|161561630294|359344 / 161561989638|27984 / 311382|
|
||||
|5|ENGLISH SNOWBALL PORTER2|PRIMARY_OUTPUT|283368|368027|28014|161561621611|368027 / 161561989638|28014 / 311382|
|
||||
|6|ENGLISH SNOWBALL ORIGINAL PORTER|PRIMARY_OUTPUT|283312|357325|28070|161561632313|357325 / 161561989638|28070 / 311382|
|
||||
|7|ENGLISH PAICE HUSK LANCASTER|PRIMARY_OUTPUT|282022|1666990|29360|161560322648|1666990 / 161561989638|29360 / 311382|
|
||||
|8|ENGLISH LUCENE KSTEM FILTER|PRIMARY_OUTPUT|237291|193354|74091|161561796284|193354 / 161561989638|74091 / 311382|
|
||||
|9|ENGLISH LUCENE MINIMAL FILTER|PRIMARY_OUTPUT|136930|1388|174452|161561988250|1388 / 161561989638|174452 / 311382|
|
||||
|10|HUNSPELL ENGLISH LUCENE FILTER|PRIMARY_OUTPUT|45763|19600|265619|161561970038|19600 / 161561989638|265619 / 311382|
|
||||
|11|ENGLISH LUCENE POSSESSIVE FILTER|PRIMARY_OUTPUT|5|36|311377|161561989602|36 / 161561989638|311377 / 311382|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 170474840204 (0.000000%)|0 / 311891 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.581994|1974950 / 170474840204 (0.001158%)|260741 / 311891 (83.600040%)|0.030387|0.043756|0.064341|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|0.000011%|83.640994%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.025246|0.164000|0.999988|0.581994|0.999987|0.000013|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|0.030387|0.043756|0.078123|0.022367|0.064345|0.064341|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|311891|0|0|170474840204|0 / 170474840204|0 / 311891|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ANY_CANDIDATE|51150|1974950|260741|170472865254|1974950 / 170474840204|260741 / 311891|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 161561989638|0 / 311382|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|18564 / 161561989638|260443 / 311382|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999966|11470018 / 170474840204 (0.006728%)|0 / 311891 (0.000000%)|0.032872|0.051579|0.162697|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.581994|2004598 / 170474840204 (0.001176%)|260741 / 311891 (83.600040%)|0.029965|0.043208|0.063875|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|0.581795|0.000023%|83.640994%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -439,8 +352,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.026472|1.000000|0.999933|0.999966|0.999933|0.000067|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.024881|0.164000|0.999988|0.581994|0.999987|0.000013|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999952|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.581828|0.163590|1.000000|0.581795|0.999998|0.000002|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -448,17 +361,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.032872|0.051579|0.119687|0.026472|0.162702|0.162697|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.029965|0.043208|0.077422|0.022081|0.063879|0.063875|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999961|0.999976|0.999990|0.999952|0.999976|0.999976|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|0.384979|0.255377|0.191058|0.146379|0.308515|0.308514|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -466,8 +370,8 @@ This mode contains **15 result rows**, **11 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|311891|11470018|0|170463370186|11470018 / 170474840204|0 / 311891|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|51150|2004598|260741|170472835606|2004598 / 170474840204|260741 / 311891|
|
||||
|1|Radixor|ALL_CANDIDATES|311382|15|0|161561989623|15 / 161561989638|0 / 311382|
|
||||
|2|HUNSPELL ENGLISH LUCENE FILTER|ALL_CANDIDATES|50939|36611|260443|161561953027|36611 / 161561989638|260443 / 311382|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -477,20 +381,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|21319|1148489|10321529|28826|4.936720%|1355|2812871|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|5224|3091|26557|6786|1.162165%|4|590716|
|
||||
|Radixor|21048|3|12|13357|2.349760%|1355|584042|
|
||||
|HUNSPELL ENGLISH LUCENE FILTER|5176|1036|17011|5685|1.000104%|4|574142|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -499,16 +403,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `US_UK`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `FI_FI` | 57,027 | 1,865,215 | 110,525 | 1,754,690 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `fi-fi-default` | `1.0.0` | `FI_FI` | 57,027 | 1,865,215 | 110,525 | 1,754,690 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **1,865,215**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **1,865,215**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 745 | 0.040% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 1,176,003 | 63.049% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 565,585 | 30.323% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 116,946 | 6.270% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 5,936 | 0.318% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 1,117 | 0.060% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 1,175,880 | 63.043% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 570,130 | 30.566% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 112,029 | 6.006% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 6,059 | 0.325% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -35,16 +35,24 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 10.991% | 10.268% | 22.471% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene FinnishLightStemFilter | 4.351% | 4.294% | 5.264% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `finnishRadixor` | 308.076 | 15.529 | 175.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 175.250 | 46.995 | 99.9 | 0.569 | Light Finnish suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[FINNISH]` | 264.652 | 63.054 | 150.8 | 0.859 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 374.883 | 238.157 | 213.6 | 1.217 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `finnishRadixor` | 289.539 | 4.136 | 165.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 175.789 | 4.827 | 100.2 | 0.607 | Light Finnish suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[FINNISH]` | 259.889 | 8.924 | 148.1 | 0.898 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 332.524 | 9.490 | 189.5 | 1.148 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -58,30 +66,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FI_FI` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FI_FI` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/fi_fi/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `fi-fi-default`, loaded from classpath resource `org/egothor/stemmer/models/fi-fi-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.984594** among 4 deterministic stemmers. The runner-up is `SNOWBALL FINNISH LUCENE FILTER` at 0.740353, a difference of 0.244242. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.988068** among 4 deterministic stemmers. The runner-up is `SNOWBALL FINNISH DIRECT` at 0.738400, a difference of 0.249668. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.984838** among 4 deterministic stemmers. The runner-up is `SNOWBALL FINNISH LUCENE FILTER` at 0.740279, a difference of 0.244559. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.988242** among 4 deterministic stemmers. The runner-up is `SNOWBALL FINNISH DIRECT` at 0.738344, a difference of 0.249898. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.984594|731279 / 1641126814491 (0.000045%)|971268 / 31523695 (3.081073%)|0.975128|0.972893|0.972899|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.740353|1922153 / 1641126814491 (0.000117%)|16370057 / 31523695 (51.929372%)|0.758996|0.623613|0.653138|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.739729|1544812 / 1641126814491 (0.000094%)|16409363 / 31523695 (52.054060%)|0.769880|0.627374|0.659540|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.695969|2223150 / 1641126814491 (0.000135%)|19168306 / 31523695 (60.806025%)|0.687649|0.536000|0.576338|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.984838|<0.000001%|3.032474%|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|0.740279|0.000081%|51.944179%|
|
||||
|3|SNOWBALL FINNISH DIRECT|0.739671|0.000060%|52.065724%|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|0.695725|0.000094%|60.854936%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -89,10 +97,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.976624|0.969189|1.000000|0.984594|0.999999|0.000001|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.887434|0.480706|0.999999|0.740353|0.999989|0.000011|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.907269|0.479459|0.999999|0.739729|0.999989|0.000011|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.847505|0.391940|0.999999|0.695969|0.999987|0.000013|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999974|0.969675|1.000000|0.984838|0.999999|0.000001|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.921471|0.480558|0.999999|0.740279|0.999989|0.000011|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.940611|0.479343|0.999999|0.739671|0.999989|0.000011|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.890914|0.391451|0.999999|0.695725|0.999987|0.000013|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -100,21 +108,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.975128|0.972893|0.970667|0.947216|0.972900|0.972899|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.758996|0.623613|0.529216|0.453080|0.653142|0.653138|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.769880|0.627374|0.529384|0.457061|0.659544|0.659540|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.687649|0.536000|0.439152|0.366120|0.576343|0.576338|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.972892|0.996085|0.993746|0.994914|0.994914|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.623608|0.990718|0.904385|0.945585|0.945585|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.627369|0.991872|0.904139|0.945975|0.945975|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.535994|0.988126|0.886473|0.934544|0.934544|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.993763|0.984591|0.975587|0.969650|0.984708|0.984708|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.778598|0.631685|0.531413|0.461652|0.665448|0.665443|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.788800|0.635056|0.531468|0.465262|0.671472|0.671468|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.709787|0.543915|0.440884|0.373546|0.590550|0.590545|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -122,62 +119,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|30552427|731279|971268|1641126083212|731279 / 1641126814491|971268 / 31523695|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|15153638|1922153|16370057|1641124892338|1922153 / 1641126814491|16370057 / 31523695|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|15114332|1544812|16409363|1641125269679|1544812 / 1641126814491|16409363 / 31523695|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|12355389|2223150|19168306|1641124591341|2223150 / 1641126814491|19168306 / 31523695|
|
||||
|1|Radixor|PRIMARY_OUTPUT|30511413|804|954186|1599841738533|804 / 1599841739337|954186 / 31465599|
|
||||
|2|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|15121052|1288634|16344547|1599840450703|1288634 / 1599841739337|16344547 / 31465599|
|
||||
|3|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|15082807|952306|16382792|1599840787031|952306 / 1599841739337|16382792 / 31465599|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|12317229|1508153|19148370|1599840231184|1508153 / 1599841739337|19148370 / 31465599|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 1641126814491 (0.000000%)|0 / 31523695 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|31523695|0|0|1641126814491|0 / 1641126814491|0 / 31523695|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1599841739337|0 / 31465599|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|1683575 / 1641126814491 (0.000103%)|0 / 31523695 (0.000000%)|0.959025|0.973991|0.974320|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -185,7 +160,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.949301|1.000000|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999926|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -193,15 +168,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.959025|0.973991|0.989432|0.949301|0.974321|0.974320|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999941|0.999963|0.999985|0.999926|0.999963|0.999963|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -209,7 +176,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|31523695|1683575|0|1641125130916|1683575 / 1641126814491|0 / 31523695|
|
||||
|1|Radixor|ALL_CANDIDATES|31465599|2327|0|1599841737010|2327 / 1599841739337|0 / 31465599|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -219,22 +186,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|971268|731279|952296|57328|3.164291%|6|1876272|
|
||||
|Radixor|954186|804|1523|34395|1.922815%|6|1826768|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988068|730145 / 1543589444152 (0.000047%)|735305 / 30813833 (2.386282%)|0.976268|0.976219|0.976218|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.738400|1513705 / 1543589444152 (0.000098%)|16121763 / 30813833 (52.319888%)|0.768117|0.624934|0.657464|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.738400|1513705 / 1543589444152 (0.000098%)|16121763 / 30813833 (52.319888%)|0.768117|0.624934|0.657464|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.694529|1806392 / 1543589444152 (0.000117%)|18825444 / 30813833 (61.094133%)|0.697056|0.537492|0.581469|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.988242|<0.000001%|2.351587%|
|
||||
|2|SNOWBALL FINNISH DIRECT|0.738344|0.000062%|52.331112%|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|0.738344|0.000062%|52.331112%|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|0.694308|0.000077%|61.138333%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -242,10 +209,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.976301|0.976137|1.000000|0.988068|0.999999|0.000001|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.906595|0.476801|0.999999|0.738400|0.999989|0.000011|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.906595|0.476801|0.999999|0.738400|0.999989|0.000011|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.869053|0.389059|0.999999|0.694529|0.999987|0.000013|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999973|0.976484|1.000000|0.988242|1.000000|0.000000|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.939951|0.476689|0.999999|0.738344|0.999989|0.000011|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.939951|0.476689|0.999999|0.738344|0.999989|0.000011|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.911893|0.388617|0.999999|0.694308|0.999987|0.000013|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -253,21 +220,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.976268|0.976219|0.976170|0.953543|0.976219|0.976218|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.768117|0.624934|0.526744|0.454475|0.657469|0.657464|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.768117|0.624934|0.526744|0.454475|0.657469|0.657464|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.697056|0.537492|0.437372|0.367514|0.581474|0.581469|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.976218|0.996000|0.996069|0.996035|0.996035|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.624929|0.991732|0.902933|0.945252|0.945252|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.624929|0.991732|0.902933|0.945252|0.945252|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.537486|0.989268|0.885294|0.934397|0.934397|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995185|0.988089|0.981093|0.976459|0.988159|0.988159|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|0.786987|0.632573|0.528815|0.462601|0.669376|0.669372|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|0.786987|0.632573|0.528815|0.462601|0.669376|0.669372|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.718421|0.544981|0.438999|0.374553|0.595296|0.595291|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -275,62 +231,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|30078528|730145|735305|1543588714007|730145 / 1543589444152|735305 / 30813833|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|14692070|1513705|16121763|1543587930447|1513705 / 1543589444152|16121763 / 30813833|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|14692070|1513705|16121763|1543587930447|1513705 / 1543589444152|16121763 / 30813833|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|11988389|1806392|18825444|1543587637760|1806392 / 1543589444152|18825444 / 30813833|
|
||||
|1|Radixor|PRIMARY_OUTPUT|30037514|804|723369|1504706134249|804 / 1504706135053|723369 / 30760883|
|
||||
|2|SNOWBALL FINNISH DIRECT|PRIMARY_OUTPUT|14663371|936765|16097512|1504705198288|936765 / 1504706135053|16097512 / 30760883|
|
||||
|3|SNOWBALL FINNISH LUCENE FILTER|PRIMARY_OUTPUT|14663371|936765|16097512|1504705198288|936765 / 1504706135053|16097512 / 30760883|
|
||||
|4|FINNISH LUCENE FINNISH LIGHT STEM FILTER|PRIMARY_OUTPUT|11954192|1155011|18806691|1504704980042|1155011 / 1504706135053|18806691 / 30760883|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 1543589444152 (0.000000%)|0 / 30813833 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|30813833|0|0|1543589444152|0 / 1543589444152|0 / 30813833|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1504706135053|0 / 30760883|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|1653320 / 1543589444152 (0.000107%)|0 / 30813833 (0.000000%)|0.958843|0.973873|0.974205|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -338,7 +272,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.949077|1.000000|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999927|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -346,15 +280,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.958843|0.973873|0.989383|0.949077|0.974206|0.974205|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999942|0.999964|0.999985|0.999927|0.999964|0.999964|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -362,7 +288,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|30813833|1653320|0|1543587790832|1653320 / 1543589444152|0 / 30813833|
|
||||
|1|Radixor|ALL_CANDIDATES|30760883|2235|0|1504706132818|2235 / 1504706135053|0 / 30760883|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -372,19 +298,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|735305|730145|923175|44331|2.523029%|6|1805864|
|
||||
|Radixor|723369|804|1431|22060|1.271628%|6|1758300|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -393,16 +319,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `FI_FI`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `FR_FR` | 59,240 | 474,110 | 108,141 | 365,969 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `fr-fr-default` | `1.0.0` | `FR_FR` | 59,240 | 474,110 | 108,141 | 365,969 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **474,110**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **474,110**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 5,370 | 1.133% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 185,263 | 39.076% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 153,886 | 32.458% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 116,519 | 24.576% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 13,072 | 2.757% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 10,082 | 2.127% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 184,521 | 38.919% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 154,760 | 32.642% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 110,933 | 23.398% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 13,814 | 2.914% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -37,18 +37,26 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 8.462% | 5.067% | 19.952% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene FrenchLightStemFilter | 6.377% | 3.965% | 14.540% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `frenchRadixor` | 47.033 | 4.146 | 128.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1664.935 | 65.928 | 4549.4 | 35.399 | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 19.234 | 2.098 | 52.6 | 0.409 | Minimal French suffix reducer; narrow baseline. |
|
||||
| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 30.560 | 3.680 | 83.5 | 0.650 | Light French suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[FRENCH]` | 111.057 | 8.172 | 303.5 | 2.361 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 123.648 | 3.500 | 337.9 | 2.629 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `frenchRadixor` | 49.340 | 0.986 | 134.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1781.070 | 43.544 | 4866.7 | 36.098 | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 19.093 | 0.681 | 52.2 | 0.387 | Minimal French suffix reducer; narrow baseline. |
|
||||
| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 29.553 | 0.465 | 80.8 | 0.599 | Light French suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[FRENCH]` | 121.376 | 0.865 | 331.7 | 2.460 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 126.574 | 4.671 | 345.9 | 2.565 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -62,32 +70,32 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FR_FR` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FR_FR` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/fr_fr/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `fr-fr-default`, loaded from classpath resource `org/egothor/stemmer/models/fr-fr-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.956992** among 6 deterministic stemmers. The runner-up is `SNOWBALL FRENCH DIRECT` at 0.845262, a difference of 0.111731. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.957224** among 6 deterministic stemmers. The runner-up is `SNOWBALL FRENCH DIRECT` at 0.845414, a difference of 0.111810. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.958627** among 6 deterministic stemmers. The runner-up is `SNOWBALL FRENCH DIRECT` at 0.848662, a difference of 0.109965. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.958856** among 6 deterministic stemmers. The runner-up is `SNOWBALL FRENCH DIRECT` at 0.848826, a difference of 0.110031. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.956992|318767 / 90396104830 (0.000353%)|469160 / 5454615 (8.601157%)|0.934603|0.926765|0.926851|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.845262|1654723 / 90396104830 (0.001831%)|1687975 / 5454615 (30.945814%)|0.693926|0.692653|0.692638|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.844999|1661388 / 90396104830 (0.001838%)|1690838 / 5454615 (30.998301%)|0.693010|0.691885|0.691869|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.813742|776728 / 90396104830 (0.000859%)|2031881 / 5454615 (37.250677%)|0.769069|0.709075|0.715131|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.518587|276403 / 90396104830 (0.000306%)|5251833 / 5454615 (96.282377%)|0.137547|0.068348|0.125415|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.516830|160438 / 90396104830 (0.000177%)|5271003 / 5454615 (96.633823%)|0.134400|0.063329|0.134021|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.958627|<0.000001%|8.274665%|
|
||||
|2|SNOWBALL FRENCH DIRECT|0.848662|0.001338%|30.266309%|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|0.848404|0.001345%|30.317815%|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|0.816824|0.000540%|36.634583%|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|0.518478|0.000187%|96.304159%|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|0.516784|0.000083%|96.643216%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -95,12 +103,12 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.939903|0.913988|0.999996|0.956992|0.999991|0.000009|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.694777|0.690542|0.999982|0.845262|0.999963|0.000037|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.693763|0.690017|0.999982|0.844999|0.999963|0.000037|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.815041|0.627493|0.999991|0.813742|0.999969|0.000031|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.423181|0.037176|0.999997|0.518587|0.999939|0.000061|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.533678|0.033662|0.999998|0.516830|0.999940|0.000060|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999994|0.917253|1.000000|0.958627|0.999995|0.000005|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.774195|0.697337|0.999987|0.848662|0.999967|0.000033|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.773169|0.696822|0.999987|0.848404|0.999967|0.000033|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.885315|0.633654|0.999995|0.816824|0.999970|0.000030|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.565022|0.036958|0.999998|0.518478|0.999935|0.000065|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.726387|0.033568|0.999999|0.516784|0.999936|0.000064|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -108,25 +116,12 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.934603|0.926765|0.919056|0.863524|0.926855|0.926851|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.693926|0.692653|0.691385|0.529816|0.692656|0.692638|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.693010|0.691885|0.690763|0.528917|0.691887|0.691869|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.769069|0.709075|0.657765|0.549277|0.715145|0.715131|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.137547|0.068348|0.045472|0.035383|0.125428|0.125415|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.134400|0.063329|0.041424|0.032700|0.134032|0.134021|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.926760|0.988772|0.985214|0.986990|0.986990|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.692635|0.959459|0.944948|0.952148|0.952148|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.691866|0.958698|0.944715|0.951655|0.951655|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.709060|0.978337|0.913706|0.944918|0.944918|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.068339|0.974110|0.812376|0.885922|0.885922|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.063322|0.984019|0.810979|0.889158|0.889158|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.982273|0.956838|0.932688|0.917248|0.957731|0.957728|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.757497|0.733759|0.711463|0.579478|0.734761|0.734745|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.756590|0.733013|0.710861|0.578548|0.734003|0.733987|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.820168|0.738637|0.671850|0.585587|0.748988|0.748975|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.146469|0.069379|0.045455|0.035936|0.144507|0.144495|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.141655|0.064170|0.041481|0.033149|0.156151|0.156143|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -134,70 +129,45 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|4985455|318767|469160|90395786063|318767 / 90396104830|469160 / 5454615|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|3766640|1654723|1687975|90394450107|1654723 / 90396104830|1687975 / 5454615|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3763777|1661388|1690838|90394443442|1661388 / 90396104830|1690838 / 5454615|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3422734|776728|2031881|90395328102|776728 / 90396104830|2031881 / 5454615|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|202782|276403|5251833|90395828427|276403 / 90396104830|5251833 / 5454615|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|183612|160438|5271003|90395944392|160438 / 90396104830|5271003 / 5454615|
|
||||
|1|Radixor|PRIMARY_OUTPUT|4925833|29|444366|81606871827|29 / 81606871856|444366 / 5370199|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|3744838|1092238|1625361|81605779618|1092238 / 81606871856|1625361 / 5370199|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3742072|1097843|1628127|81605774013|1097843 / 81606871856|1628127 / 5370199|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3402849|440809|1967350|81606431047|440809 / 81606871856|1967350 / 5370199|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|198474|152794|5171725|81606719062|152794 / 81606871856|5171725 / 5370199|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|180266|67902|5189933|81606803954|67902 / 81606871856|5189933 / 5370199|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999979|12 / 90396104830 (0.000000%)|232 / 5454615 (0.004253%)|0.999990|0.999978|0.999978|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.830964|745831 / 90396104830 (0.000825%)|1844003 / 5454615 (33.806291%)|0.789019|0.736029|0.740670|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.004320%|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|0.000539%|33.189869%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999998|0.999957|1.000000|0.999979|1.000000|0.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.828798|0.661937|0.999992|0.830964|0.999971|0.000029|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999990|0.999978|0.999966|0.999955|0.999978|0.999978|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.789019|0.736029|0.689709|0.582315|0.740684|0.740670|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|5454383|12|232|90396104818|12 / 90396104830|232 / 5454615|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|3610612|745831|1844003|90395358999|745831 / 90396104830|1844003 / 5454615|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 81606871856|232 / 5370199|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|439665 / 81606871856|1782362 / 5370199|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999973|1056255 / 90396104830 (0.001168%)|232 / 5454615 (0.004253%)|0.865853|0.911704|0.915270|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.830963|1043199 / 90396104830 (0.001154%)|1844003 / 5454615 (33.806291%)|0.750028|0.714377|0.716613|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.999978|0.000003%|0.004320%|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|0.834048|0.000614%|33.189869%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -205,8 +175,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.837765|0.999957|0.999988|0.999973|0.999988|0.000012|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.775840|0.661937|0.999988|0.830963|0.999968|0.000032|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999571|0.999957|1.000000|0.999978|1.000000|0.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.877537|0.668101|0.999994|0.834048|0.999972|0.000028|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -214,17 +184,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.865853|0.911704|0.962682|0.837735|0.915275|0.915270|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.750028|0.714377|0.681961|0.555666|0.716629|0.716613|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999648|0.999764|0.999880|0.999528|0.999764|0.999764|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.825765|0.758630|0.701590|0.611123|0.765691|0.765678|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -232,8 +193,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|5454383|1056255|232|90395048575|1056255 / 90396104830|232 / 5454615|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|3610612|1043199|1844003|90395061631|1043199 / 90396104830|1844003 / 5454615|
|
||||
|1|Radixor|ALL_CANDIDATES|5369967|2303|232|81606869553|2303 / 81606871856|232 / 5370199|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|3587837|500695|1782362|81606371161|500695 / 81606871856|1782362 / 5370199|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -243,25 +204,25 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|468928|318755|737488|43040|10.122057%|56|477024|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|187878|30897|266471|13511|3.177489%|4|439015|
|
||||
|Radixor|444134|29|2274|21844|5.406783%|56|427440|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|184988|1144|59886|8230|2.037073%|4|412364|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.957224|315266 / 88712126506 (0.000355%)|465436 / 5440559 (8.554930%)|0.935099|0.927248|0.927334|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.845414|1646111 / 88712126506 (0.001856%)|1681970 / 5440559 (30.915389%)|0.694508|0.693130|0.693115|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.845163|1641925 / 88712126506 (0.001851%)|1684703 / 5440559 (30.965623%)|0.694714|0.693068|0.693055|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.813617|763305 / 88712126506 (0.000860%)|2028011 / 5440559 (37.275784%)|0.770537|0.709734|0.715938|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.518442|262689 / 88712126506 (0.000296%)|5239869 / 5440559 (96.311225%)|0.137571|0.067985|0.126383|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.516697|147476 / 88712126506 (0.000166%)|5258873 / 5440559 (96.660527%)|0.134439|0.062979|0.135757|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.958856|<0.000001%|8.228703%|
|
||||
|2|SNOWBALL FRENCH DIRECT|0.848826|0.001356%|30.233460%|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|0.848580|0.001353%|30.282729%|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|0.816702|0.000540%|36.658999%|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|0.518338|0.000181%|96.332173%|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|0.516654|0.000076%|96.669051%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -269,12 +230,12 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.940408|0.914451|0.999996|0.957224|0.999991|0.000009|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.695430|0.690846|0.999981|0.845414|0.999962|0.000038|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.695815|0.690344|0.999981|0.845163|0.999963|0.000037|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.817210|0.627242|0.999991|0.813617|0.999969|0.000031|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.433101|0.036888|0.999997|0.518442|0.999938|0.000062|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.551965|0.033395|0.999998|0.516697|0.999939|0.000061|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.917713|1.000000|0.958856|0.999995|0.000005|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.774357|0.697665|0.999986|0.848826|0.999966|0.000034|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.774620|0.697173|0.999986|0.848580|0.999966|0.000034|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.886736|0.633410|0.999995|0.816702|0.999970|0.000030|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.575115|0.036678|0.999998|0.518338|0.999934|0.000066|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.746071|0.033309|0.999999|0.516654|0.999935|0.000065|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -282,25 +243,12 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.935099|0.927248|0.919527|0.864363|0.927338|0.927334|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.694508|0.693130|0.691758|0.530374|0.693134|0.693115|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.694714|0.693068|0.691431|0.530302|0.693074|0.693055|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.770537|0.709734|0.657826|0.550068|0.715953|0.715938|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.137571|0.067985|0.045148|0.035189|0.126397|0.126383|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.134439|0.062979|0.041121|0.032513|0.135767|0.135757|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.927243|0.988916|0.985550|0.987230|0.987230|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.693112|0.959521|0.944537|0.951970|0.951970|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.693050|0.959566|0.944385|0.951915|0.951915|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.709719|0.979328|0.913162|0.945088|0.945088|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.067976|0.975086|0.811144|0.885591|0.885591|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.062973|0.985086|0.809774|0.888868|0.888868|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.982383|0.957091|0.933069|0.917713|0.957973|0.957971|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|0.757699|0.734013|0.711764|0.579795|0.735012|0.734995|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.757784|0.733859|0.711398|0.579603|0.734877|0.734860|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|0.821061|0.738965|0.671794|0.585999|0.749445|0.749431|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.146117|0.068959|0.045128|0.035711|0.145238|0.145227|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.141311|0.063772|0.041177|0.032936|0.157643|0.157634|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -308,70 +256,45 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|4975123|315266|465436|88711811240|315266 / 88712126506|465436 / 5440559|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|3758589|1646111|1681970|88710480395|1646111 / 88712126506|1681970 / 5440559|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3755856|1641925|1684703|88710484581|1641925 / 88712126506|1684703 / 5440559|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3412548|763305|2028011|88711363201|763305 / 88712126506|2028011 / 5440559|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|200690|262689|5239869|88711863817|262689 / 88712126506|5239869 / 5440559|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|181686|147476|5258873|88711979030|147476 / 88712126506|5258873 / 5440559|
|
||||
|1|Radixor|PRIMARY_OUTPUT|4915501|1|440750|80279496864|1 / 80279496865|440750 / 5356251|
|
||||
|2|SNOWBALL FRENCH DIRECT|PRIMARY_OUTPUT|3736871|1088903|1619380|80278407962|1088903 / 80279496865|1619380 / 5356251|
|
||||
|3|SNOWBALL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3734232|1086494|1622019|80278410371|1086494 / 80279496865|1622019 / 5356251|
|
||||
|4|HUNSPELL FRENCH LUCENE FILTER|PRIMARY_OUTPUT|3392703|433354|1963548|80279063511|433354 / 80279496865|1963548 / 5356251|
|
||||
|5|FRENCH LUCENE FRENCH LIGHT STEM FILTER|PRIMARY_OUTPUT|196458|145140|5159793|80279351725|145140 / 80279496865|5159793 / 5356251|
|
||||
|6|FRENCH LUCENE FRENCH MINIMAL STEM FILTER|PRIMARY_OUTPUT|178414|60724|5177837|80279436141|60724 / 80279496865|5177837 / 5356251|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 88712126506 (0.000000%)|0 / 5440559 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.830852|733584 / 88712126506 (0.000827%)|1840476 / 5440559 (33.828803%)|0.790351|0.736648|0.741404|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|0.000539%|33.211718%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.830724|0.661712|0.999992|0.830852|0.999971|0.000029|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|0.790351|0.736648|0.689779|0.583090|0.741418|0.741404|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|5440559|0|0|88712126506|0 / 88712126506|0 / 5440559|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ANY_CANDIDATE|3600083|733584|1840476|88711392922|733584 / 88712126506|1840476 / 5440559|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 80279496865|0 / 5356251|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|432307 / 80279496865|1778903 / 5356251|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999995|938985 / 88712126506 (0.001058%)|0 / 5440559 (0.000000%)|0.878679|0.920560|0.923474|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.830850|1027635 / 88712126506 (0.001158%)|1840476 / 5440559 (33.828803%)|0.751538|0.715134|0.717460|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|0.833938|0.000614%|33.211718%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -379,8 +302,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.852813|1.000000|0.999989|0.999995|0.999989|0.000011|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.777939|0.661712|0.999988|0.830850|0.999968|0.000032|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999986|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.878983|0.667883|0.999994|0.833938|0.999972|0.000028|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -388,17 +311,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.878679|0.920560|0.966634|0.852813|0.923479|0.923474|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.751538|0.715134|0.682093|0.556582|0.717476|0.717460|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999989|0.999993|0.999997|0.999986|0.999993|0.999993|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|0.826722|0.759029|0.701582|0.611641|0.766197|0.766184|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -406,8 +320,8 @@ This mode contains **10 result rows**, **6 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|5440559|938985|0|88711187521|938985 / 88712126506|0 / 5440559|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|3600083|1027635|1840476|88711098871|1027635 / 88712126506|1840476 / 5440559|
|
||||
|1|Radixor|ALL_CANDIDATES|5356251|75|0|80279496790|75 / 80279496865|0 / 5356251|
|
||||
|2|HUNSPELL FRENCH LUCENE FILTER|ALL_CANDIDATES|3577348|492522|1778903|80279004343|492522 / 80279496865|1778903 / 5356251|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -417,20 +331,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|465436|315266|623719|41130|9.764239%|56|468574|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|187535|29721|264330|13437|3.189936%|4|434961|
|
||||
|Radixor|440750|1|74|20611|5.143594%|56|422336|
|
||||
|HUNSPELL FRENCH LUCENE FILTER|184645|1047|59168|8194|2.044860%|4|409028|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -439,16 +353,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `FR_FR`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `DE_DE` | 39,315 | 213,440 | 73,799 | 139,641 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `de-de-default` | `1.0.0` | `DE_DE` | 54,092 | 333,036 | 90,535 | 242,501 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **213,440**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **333,036**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 3,627 | 1.699% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 48,605 | 22.772% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 80,443 | 37.689% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 75,717 | 35.475% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 5,048 | 2.365% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 12,107 | 3.635% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 81,805 | 24.563% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 142,376 | 42.751% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 88,820 | 26.670% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 7,928 | 2.381% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -39,20 +39,28 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 30.481% | 29.027% | 34.376% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene GermanStemFilter | 21.559% | 19.312% | 27.576% | German Lucene stemming TokenFilter; broader than minimal/light variants. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `germanRadixor` | 41.166 | 2.396 | 294.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| CISTEM | `germanCistem` | 248.392 | 12.294 | 1778.8 | 6.034 | Benchmark-only CISTEM implementation. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 281.322 | 3.411 | 2014.6 | 6.834 | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 23.562 | 0.969 | 168.7 | 0.572 | Minimal German suffix reduction; narrow baseline. |
|
||||
| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 24.410 | 1.034 | 174.8 | 0.593 | Light German suffix stemmer; narrower than a dictionary stemmer. |
|
||||
| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 71.039 | 4.443 | 508.7 | 1.726 | Older German stemming TokenFilter with normalization requirements. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 105.771 | 9.617 | 757.4 | 2.569 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Official Snowball direct | `snowballDirect[GERMAN]` | 100.688 | 9.018 | 721.0 | 2.446 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Radixor | `germanRadixor` | 40.571 | 1.647 | 167.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| CISTEM | `germanCistem` | 305.166 | 4.590 | 1258.4 | 7.522 | Benchmark-only CISTEM implementation. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 291.791 | 21.769 | 1203.3 | 7.192 | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 23.903 | 0.208 | 98.6 | 0.589 | Minimal German suffix reduction; narrow baseline. |
|
||||
| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 24.695 | 0.322 | 101.8 | 0.609 | Light German suffix stemmer; narrower than a dictionary stemmer. |
|
||||
| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 72.140 | 1.544 | 297.5 | 1.778 | Older German stemming TokenFilter with normalization requirements. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 110.086 | 2.315 | 454.0 | 2.713 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Official Snowball direct | `snowballDirect[GERMAN]` | 100.122 | 2.623 | 412.9 | 2.468 | Official Snowball generated Java stemmer; direct API. |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -66,34 +74,34 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DE_DE` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DE_DE` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/de_de/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `de-de-default`, loaded from classpath resource `org/egothor/stemmer/models/de-de-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.907901** among 8 deterministic stemmers. The runner-up is `GERMAN CISTEM` at 0.880770, a difference of 0.027131. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.966157** among 8 deterministic stemmers. The runner-up is `GERMAN CISTEM` at 0.915288, a difference of 0.050869. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.910445** among 8 deterministic stemmers. The runner-up is `GERMAN CISTEM` at 0.878527, a difference of 0.031918. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.966959** among 8 deterministic stemmers. The runner-up is `GERMAN CISTEM` at 0.914727, a difference of 0.052232. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.907901|98192 / 44095245979 (0.000223%)|254903 / 1383872 (18.419550%)|0.897073|0.864768|0.866326|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.880770|477122 / 44095245979 (0.001082%)|329983 / 1383872 (23.844908%)|0.701852|0.723109|0.724023|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.778614|190680 / 44095245979 (0.000432%)|612734 / 1383872 (44.276783%)|0.737064|0.657494|0.668394|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.771357|295701 / 44095245979 (0.000671%)|632816 / 1383872 (45.727929%)|0.674089|0.617993|0.624014|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.756258|205740 / 44095245979 (0.000467%)|674609 / 1383872 (48.747933%)|0.703092|0.617052|0.630292|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.723772|331871 / 44095245979 (0.000753%)|764518 / 1383872 (55.244849%)|0.596821|0.530474|0.539809|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.641579|203883 / 44095245979 (0.000462%)|992010 / 1383872 (71.683653%)|0.520145|0.395897|0.431563|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.598139|110840 / 44095245979 (0.000251%)|1112246 / 1383872 (80.372029%)|0.466113|0.307558|0.373350|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.910445|0.000002%|17.910967%|
|
||||
|2|GERMAN CISTEM|0.878527|0.000674%|24.293900%|
|
||||
|3|SNOWBALL GERMAN DIRECT|0.776006|0.000171%|44.798684%|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|0.769071|0.000371%|46.185528%|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|0.753833|0.000191%|49.233299%|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|0.720992|0.000443%|55.801084%|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|0.640308|0.000290%|71.938102%|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|0.595748|0.000088%|80.850384%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -101,14 +109,14 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.919984|0.815804|0.999998|0.907901|0.999992|0.000008|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.688361|0.761551|0.999989|0.880770|0.999982|0.000018|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.801750|0.557232|0.999996|0.778614|0.999982|0.000018|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.717508|0.542721|0.999993|0.771357|0.999979|0.000021|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.775148|0.512521|0.999995|0.756258|0.999980|0.000020|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.651112|0.447552|0.999992|0.723772|0.999975|0.000025|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.657768|0.283163|0.999995|0.641579|0.999973|0.000027|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.710196|0.196280|0.999997|0.598139|0.999972|0.000028|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999400|0.820890|1.000000|0.910445|0.999994|0.000006|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.797231|0.757061|0.999993|0.878527|0.999985|0.000015|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.918570|0.552013|0.999998|0.776006|0.999983|0.000017|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.835220|0.538145|0.999996|0.769071|0.999980|0.000020|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.902792|0.507667|0.999998|0.753833|0.999981|0.000019|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.777304|0.441989|0.999996|0.720992|0.999976|0.000024|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.771720|0.280619|0.999997|0.640308|0.999972|0.000028|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.883845|0.191496|0.999999|0.595748|0.999971|0.000029|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -116,29 +124,14 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.897073|0.864768|0.834709|0.761755|0.866330|0.866326|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.701852|0.723109|0.745694|0.566304|0.724032|0.724023|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.737064|0.657494|0.593429|0.489751|0.668402|0.668394|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.674089|0.617993|0.570517|0.447171|0.624024|0.624014|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.703092|0.617052|0.549774|0.446186|0.630301|0.630292|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.596821|0.530474|0.477402|0.360983|0.539820|0.539809|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.520145|0.395897|0.319562|0.246803|0.431574|0.431563|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.466113|0.307558|0.229493|0.181725|0.373359|0.373350|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.864764|0.989946|0.975085|0.982460|0.982460|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.723100|0.974048|0.975147|0.974597|0.974597|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.657485|0.983725|0.949324|0.966218|0.966218|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.617983|0.975845|0.942925|0.959102|0.959102|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.617043|0.980753|0.936533|0.958133|0.958133|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.530462|0.975550|0.942890|0.958942|0.958942|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.395885|0.980463|0.886873|0.931322|0.931322|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.307549|0.983615|0.896264|0.937910|0.937910|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.957746|0.901392|0.851302|0.820486|0.905758|0.905755|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.788860|0.776627|0.764768|0.634824|0.776886|0.776879|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.810879|0.689608|0.599891|0.526260|0.712083|0.712076|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.752175|0.654552|0.579359|0.486494|0.670425|0.670416|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.781189|0.649884|0.556368|0.481355|0.676991|0.676984|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.674901|0.563540|0.483723|0.392311|0.586140|0.586130|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.571639|0.411577|0.321543|0.259110|0.465359|0.465349|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.512941|0.314789|0.227071|0.186795|0.411404|0.411396|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -146,72 +139,47 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1128969|98192|254903|44095147787|98192 / 44095245979|254903 / 1383872|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|1053889|477122|329983|44094768857|477122 / 44095245979|329983 / 1383872|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|771138|190680|612734|44095055299|190680 / 44095245979|612734 / 1383872|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|751056|295701|632816|44094950278|295701 / 44095245979|632816 / 1383872|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|709263|205740|674609|44095040239|205740 / 44095245979|674609 / 1383872|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|619354|331871|764518|44094914108|331871 / 44095245979|764518 / 1383872|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|391862|203883|992010|44095042096|203883 / 44095245979|992010 / 1383872|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|271626|110840|1112246|44095135139|110840 / 44095245979|1112246 / 1383872|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1103976|663|240876|38436733230|663 / 38436733893|240876 / 1344852|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|1018135|258954|326717|38436474939|258954 / 38436733893|326717 / 1344852|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|742376|65811|602476|38436668082|65811 / 38436733893|602476 / 1344852|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|723725|142783|621127|38436591110|142783 / 38436733893|621127 / 1344852|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|682737|73514|662115|38436660379|73514 / 38436733893|662115 / 1344852|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|594410|170297|750442|38436563596|170297 / 38436733893|750442 / 1344852|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|377391|111635|967461|38436622258|111635 / 38436733893|967461 / 1344852|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|257534|33845|1087318|38436700048|33845 / 38436733893|1087318 / 1344852|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.959835|1375 / 44095245979 (0.000003%)|111167 / 1383872 (8.033041%)|0.981996|0.957658|0.958475|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.647474|158403 / 44095245979 (0.000359%)|975697 / 1383872 (70.504859%)|0.559116|0.418544|0.460956|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000001%|8.261653%|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|0.000216%|70.811435%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.998921|0.919670|1.000000|0.959835|0.999997|0.000003|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.720422|0.294951|0.999996|0.647474|0.999974|0.000026|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.981996|0.957658|0.934498|0.918757|0.958476|0.958475|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.559116|0.418544|0.334456|0.264658|0.460966|0.460956|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1272705|1375|111167|44095244604|1375 / 44095245979|111167 / 1383872|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|408175|158403|975697|44095087576|158403 / 44095245979|975697 / 1383872|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|502 / 38436733893|111107 / 1344852|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|83073 / 38436733893|952309 / 1344852|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.959832|244817 / 44095245979 (0.000555%)|111167 / 1383872 (8.033041%)|0.853711|0.877306|0.878234|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.647473|242551 / 44095245979 (0.000550%)|975697 / 1383872 (70.504859%)|0.511911|0.401234|0.430118|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.958692|0.000018%|8.261653%|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|0.645941|0.000354%|70.811435%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -219,8 +187,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.838673|0.919670|0.999994|0.959832|0.999992|0.000008|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.627261|0.294951|0.999994|0.647473|0.999972|0.000028|
|
||||
|1|Radixor|ALL_CANDIDATES|0.994469|0.917383|1.000000|0.958692|0.999997|0.000003|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.742744|0.291886|0.999996|0.645941|0.999972|0.000028|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -228,17 +196,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.853711|0.877306|0.902242|0.781429|0.878238|0.878234|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.511911|0.401234|0.329907|0.250965|0.430130|0.430118|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.978033|0.954372|0.931829|0.912726|0.955149|0.955147|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.567444|0.419080|0.332218|0.265086|0.465614|0.465603|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -246,8 +205,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1272705|244817|111167|44095001162|244817 / 44095245979|111167 / 1383872|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|408175|242551|975697|44095003428|242551 / 44095245979|975697 / 1383872|
|
||||
|1|Radixor|ALL_CANDIDATES|1233745|6862|111107|38436727031|6862 / 38436733893|111107 / 1344852|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|392543|135961|952309|38436597932|135961 / 38436733893|952309 / 1344852|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -257,27 +216,27 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|143736|96817|146625|48574|16.356314%|8|361016|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|16313|45480|38668|7891|2.657135%|3|305052|
|
||||
|Radixor|129769|161|6199|29035|10.471893%|8|313927|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|15152|28562|24326|6482|2.337827%|3|283881|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.966157|47898 / 11263756342 (0.000425%)|59114 / 873411 (6.768177%)|0.941996|0.938343|0.938358|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.915288|156784 / 11263756342 (0.001392%)|147964 / 873411 (16.940936%)|0.823934|0.826418|0.826415|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.795926|87697 / 11263756342 (0.000779%)|356475 / 873411 (40.814118%)|0.785153|0.699487|0.711329|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.775641|77653 / 11263756342 (0.000689%)|391910 / 873411 (44.871200%)|0.774111|0.672222|0.688986|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.769953|55477 / 11263756342 (0.000493%)|401846 / 873411 (46.008809%)|0.790797|0.673446|0.695023|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.716810|78723 / 11263756342 (0.000699%)|494677 / 873411 (56.637368%)|0.700519|0.569153|0.599149|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.659196|84679 / 11263756342 (0.000752%)|595318 / 873411 (68.160122%)|0.598178|0.449922|0.494019|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.575691|21214 / 11263756342 (0.000188%)|741190 / 873411 (84.861537%)|0.444545|0.257528|0.361168|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.966959|0.000001%|6.608210%|
|
||||
|2|GERMAN CISTEM|0.914727|0.000812%|17.053716%|
|
||||
|3|SNOWBALL GERMAN DIRECT|0.794994|0.000391%|41.000819%|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|0.774716|0.000325%|45.056540%|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|0.768968|0.000130%|46.206331%|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|0.716147|0.000358%|56.770194%|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|0.659574|0.000556%|68.084626%|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|0.574999|0.000045%|85.000064%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -285,14 +244,14 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.944446|0.932318|0.999996|0.966157|0.999991|0.000009|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.822287|0.830591|0.999986|0.915288|0.999973|0.000027|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.854958|0.591859|0.999992|0.795926|0.999961|0.000039|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.861124|0.551288|0.999993|0.775641|0.999958|0.000042|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.894739|0.539912|0.999995|0.769953|0.999959|0.000041|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.827912|0.433626|0.999993|0.716810|0.999949|0.000051|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.766578|0.318399|0.999992|0.659196|0.999940|0.000060|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.861739|0.151385|0.999998|0.575691|0.999932|0.000068|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999900|0.933918|1.000000|0.966959|0.999995|0.000005|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.892172|0.829463|0.999992|0.914727|0.999978|0.000022|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.924303|0.589992|0.999996|0.794994|0.999963|0.000037|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.931871|0.549435|0.999997|0.774716|0.999960|0.000040|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.971001|0.537937|0.999999|0.768968|0.999961|0.000039|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.907196|0.432298|0.999996|0.716147|0.999950|0.000050|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.823043|0.319154|0.999994|0.659574|0.999939|0.000061|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.964480|0.149999|1.000000|0.574999|0.999931|0.000069|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -300,29 +259,14 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.941996|0.938343|0.934719|0.883848|0.938363|0.938358|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.823934|0.826418|0.828917|0.704184|0.826428|0.826415|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.785153|0.699487|0.630675|0.537854|0.711347|0.711329|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.774111|0.672222|0.594035|0.506276|0.689005|0.688986|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.790797|0.673446|0.586424|0.507666|0.695040|0.695023|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.700519|0.569153|0.479277|0.397774|0.599170|0.599149|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.598178|0.449922|0.360559|0.290258|0.494042|0.494019|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.444545|0.257528|0.181270|0.147795|0.361184|0.361168|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.938338|0.994062|0.990664|0.992360|0.992360|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.826404|0.985936|0.973570|0.979714|0.979714|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.699468|0.988418|0.932452|0.959619|0.959619|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.672202|0.989021|0.919542|0.953017|0.953017|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.673427|0.991320|0.915070|0.951670|0.951670|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.569130|0.988584|0.918718|0.952371|0.952371|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.449897|0.988041|0.865581|0.922766|0.922766|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.257511|0.992643|0.854403|0.918349|0.918349|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.985968|0.965783|0.946408|0.933831|0.966346|0.966343|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|0.878883|0.859676|0.841289|0.753887|0.860246|0.860236|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|0.830217|0.720244|0.635999|0.562799|0.738466|0.738450|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.817997|0.691285|0.598564|0.528217|0.715543|0.715527|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.836342|0.692324|0.590620|0.529431|0.722729|0.722714|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|0.743781|0.585563|0.482850|0.413990|0.626242|0.626223|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|0.625524|0.459951|0.363685|0.298660|0.512520|0.512499|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.462363|0.259621|0.180482|0.149175|0.380357|0.380343|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -330,72 +274,47 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|814297|47898|59114|11263708444|47898 / 11263756342|59114 / 873411|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|725447|156784|147964|11263599558|156784 / 11263756342|147964 / 873411|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|516936|87697|356475|11263668645|87697 / 11263756342|356475 / 873411|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|481501|77653|391910|11263678689|77653 / 11263756342|391910 / 873411|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|471565|55477|401846|11263700865|55477 / 11263756342|401846 / 873411|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|378734|78723|494677|11263677619|78723 / 11263756342|494677 / 873411|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|278093|84679|595318|11263671663|84679 / 11263756342|595318 / 873411|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|132221|21214|741190|11263735128|21214 / 11263756342|741190 / 873411|
|
||||
|1|Radixor|PRIMARY_OUTPUT|801691|80|56726|10594963454|80 / 10594963534|56726 / 858417|
|
||||
|2|GERMAN CISTEM|PRIMARY_OUTPUT|712025|86055|146392|10594877479|86055 / 10594963534|146392 / 858417|
|
||||
|3|SNOWBALL GERMAN DIRECT|PRIMARY_OUTPUT|506459|41477|351958|10594922057|41477 / 10594963534|351958 / 858417|
|
||||
|4|SNOWBALL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|471644|34482|386773|10594929052|34482 / 10594963534|386773 / 858417|
|
||||
|5|GERMAN LUCENE GERMAN LIGHT STEM FILTER|PRIMARY_OUTPUT|461774|13791|396643|10594949743|13791 / 10594963534|396643 / 858417|
|
||||
|6|GERMAN LUCENE GERMAN STEM FILTER|PRIMARY_OUTPUT|371092|37962|487325|10594925572|37962 / 10594963534|487325 / 858417|
|
||||
|7|HUNSPELL GERMAN LUCENE FILTER|PRIMARY_OUTPUT|273967|58904|584450|10594904630|58904 / 10594963534|584450 / 858417|
|
||||
|8|GERMAN LUCENE GERMAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|128762|4742|729655|10594958792|4742 / 10594963534|729655 / 858417|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 11263756342 (0.000000%)|0 / 873411 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.665363|60996 / 11263756342 (0.000542%)|584547 / 873411 (66.926911%)|0.635466|0.472281|0.522540|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|0.000383%|66.866802%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.825656|0.330731|0.999995|0.665363|0.999943|0.000057|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|0.635466|0.472281|0.375782|0.309142|0.522561|0.522540|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|873411|0|0|11263756342|0 / 11263756342|0 / 873411|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ANY_CANDIDATE|288864|60996|584547|11263695346|60996 / 11263756342|584547 / 873411|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 10594963534|0 / 858417|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|40608 / 10594963534|573996 / 858417|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999996|97544 / 11263756342 (0.000866%)|0 / 873411 (0.000000%)|0.917983|0.947112|0.948436|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.665361|96545 / 11263756342 (0.000857%)|584547 / 873411 (66.926911%)|0.598050|0.458944|0.497855|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000014%|0.000000%|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|0.665663|0.000629%|66.866802%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -403,8 +322,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.899538|1.000000|0.999991|0.999996|0.999991|0.000009|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.749500|0.330731|0.999991|0.665361|0.999940|0.000060|
|
||||
|1|Radixor|ALL_CANDIDATES|0.998267|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.810178|0.331332|0.999994|0.665663|0.999940|0.000060|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -412,17 +331,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.917983|0.947112|0.978152|0.899538|0.948440|0.948436|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.598050|0.458944|0.372338|0.297811|0.497878|0.497855|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.998613|0.999133|0.999653|0.998267|0.999133|0.999133|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|0.628511|0.470321|0.375748|0.307464|0.518110|0.518088|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -430,8 +340,8 @@ This mode contains **12 result rows**, **8 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|873411|97544|0|11263658798|97544 / 11263756342|0 / 873411|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|288864|96545|584547|11263659797|96545 / 11263756342|584547 / 873411|
|
||||
|1|Radixor|ALL_CANDIDATES|858417|1490|0|10594962044|1490 / 10594963534|0 / 858417|
|
||||
|2|HUNSPELL GERMAN LUCENE FILTER|ALL_CANDIDATES|284421|66639|573996|10594896895|66639 / 10594963534|573996 / 858417|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -441,20 +351,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|59114|47898|49646|14978|9.978814%|8|167157|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|10771|23683|11866|4989|3.323828%|3|155207|
|
||||
|Radixor|56726|80|1410|10454|7.181227%|8|157137|
|
||||
|HUNSPELL GERMAN LUCENE FILTER|10454|18296|7735|4538|3.117315%|3|150205|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -463,16 +373,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `DE_DE`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
301
docs/benchmarks/languages/hebrew.md
Normal file
301
docs/benchmarks/languages/hebrew.md
Normal file
@@ -0,0 +1,301 @@
|
||||
# Hebrew Stemmer Benchmarks
|
||||
|
||||
This page reports same-language stemming benchmarks for Hebrew. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality.
|
||||
|
||||
All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language.
|
||||
|
||||
The default Hebrew model currently has no same-language third-party adapter in the benchmark matrix. Its Radixor measurements are still published so the complete default-model language universe has identical corpus, command-distribution, exact-root, runtime, and pairwise-quality coverage.
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `he-il-default` | `1.0.0` | `HE_IL` | 2,358 | 61,071 | 4,715 | 56,356 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **61,071**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `DeletePrefixCommand` | Deletes one or more leading characters from the word form in forward traversal. | 2,188 | 3.583% |
|
||||
| `ForwardCompoundCommand` | Applies a multi-step forward patch made from skip, delete, insert, and replace operations. | 51,692 | 84.642% |
|
||||
| `PrependCharacterCommand` | Prepends one character to the beginning of the word form. | 11 | 0.018% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 4,828 | 7.906% |
|
||||
| `ReplaceFirstCharacterCommand` | Replaces the first character of the word form in forward traversal. | 2,352 | 3.851% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||
|
||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||
| --- | ---: | ---: | ---: | --- |
|
||||
| Radixor | 98.228% | 98.172% | 98.897% | Full default-model Radixor dictionary patch-command stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `hebrewRadixor` | 3.921 | 0.140 | 69.6 | 1.000 | Full default-model Radixor dictionary patch-command stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
- Radixor is a dictionary-derived patch-command stemmer. Its quality depends on the default language model used to train the compiled trie.
|
||||
- Hebrew patch commands use forward traversal as declared by the model metadata.
|
||||
- Results are environment-specific and should be compared only with rows from the same benchmark run.
|
||||
|
||||
<!-- STEMMING-QUALITY:START -->
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `HE_IL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The default model is `he-il-default`, loaded from classpath resource `org/egothor/stemmer/models/he-il-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.986075** among 1 deterministic stemmers; no same-language competitor was available. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.986075** among 1 deterministic stemmers; no same-language competitor was available. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **3 result rows**, **1 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.986075|0.000000%|2.784905%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.972151|1.000000|0.986075|0.999988|0.000012|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.994303|0.985879|0.977596|0.972151|0.985977|0.985971|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|685765|0|19645|1661488243|0 / 1661488243|19645 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1661488243|0 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|705410|0|0|1661488243|0 / 1661488243|0 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### Multi-output analysis
|
||||
|
||||
Alternative candidates are capability analyses, not replacements for the deterministic comparison.
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19645|0|0|984|1.706615%|40|58714|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **3 result rows**, **1 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.986075|0.000000%|2.784905%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.972151|1.000000|0.986075|0.999988|0.000012|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.994303|0.985879|0.977596|0.972151|0.985977|0.985971|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|685765|0|19645|1661488243|0 / 1661488243|19645 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 1661488243|0 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|705410|0|0|1661488243|0 / 1661488243|0 / 705410|
|
||||
|
||||
</details>
|
||||
|
||||
#### Multi-output analysis
|
||||
|
||||
Alternative candidates are capability analyses, not replacements for the deterministic comparison.
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19645|0|0|984|1.706615%|40|58714|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
- Jaccard index: `TP / (TP + FP + FN)`.
|
||||
- Fowlkes–Mallows index: `sqrt(precision * recall)`.
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `HE_IL`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
@@ -8,20 +8,20 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `HU_HU` | 19,406 | 935,713 | 38,775 | 896,938 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `hu-hu-default` | `1.0.0` | `HU_HU` | 19,406 | 935,713 | 38,775 | 896,938 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **935,713**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **935,713**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 15 | 0.002% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 149,173 | 15.942% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 746,296 | 79.757% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 40,125 | 4.288% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 750,282 | 80.183% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 36,139 | 3.862% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 104 | 0.011% |
|
||||
|
||||
## Accuracy
|
||||
@@ -35,16 +35,22 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 66.445% | 66.938% | 55.043% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene HungarianLightStemFilter | 14.748% | 14.777% | 14.086% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `hungarianRadixor` | 62.232 | 6.412 | 69.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 92.813 | 6.929 | 103.5 | 1.491 | Light Hungarian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 157.765 | 13.202 | 175.9 | 2.535 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 188.863 | 15.880 | 210.6 | 3.035 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `hungarianRadixor` | 61.205 | 0.944 | 68.2 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 92.090 | 3.410 | 102.7 | 1.505 | Light Hungarian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 152.969 | 4.468 | 170.5 | 2.499 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 188.807 | 5.290 | 210.5 | 3.085 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -58,30 +64,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `HU_HU` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `HU_HU` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/hu_hu/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `hu-hu-default`, loaded from classpath resource `org/egothor/stemmer/models/hu-hu-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.995491** among 4 deterministic stemmers. The runner-up is `SNOWBALL HUNGARIAN LUCENE FILTER` at 0.822606, a difference of 0.172885. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996163** among 4 deterministic stemmers. The runner-up is `SNOWBALL HUNGARIAN DIRECT` at 0.821708, a difference of 0.174455. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.995555** among 4 deterministic stemmers. The runner-up is `SNOWBALL HUNGARIAN LUCENE FILTER` at 0.822963, a difference of 0.172592. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996227** among 4 deterministic stemmers. The runner-up is `SNOWBALL HUNGARIAN DIRECT` at 0.822077, a difference of 0.174151. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995491|272900 / 419820542893 (0.000065%)|199837 / 22162103 (0.901706%)|0.988376|0.989352|0.989353|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.822606|1792049 / 419820542893 (0.000427%)|7862745 / 22162103 (35.478334%)|0.826288|0.747610|0.757196|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.822348|1506056 / 419820542893 (0.000359%)|7874191 / 22162103 (35.529981%)|0.837137|0.752866|0.763681|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.816668|4132555 / 419820542893 (0.000984%)|8125833 / 22162103 (36.665442%)|0.740018|0.696055|0.699478|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.995555|<0.000001%|0.889037%|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|0.822963|0.000378%|35.407050%|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|0.822704|0.000309%|35.458800%|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|0.816967|0.000915%|36.605593%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -89,10 +95,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987727|0.990983|0.999999|0.995491|0.999999|0.000001|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.888633|0.645217|0.999996|0.822606|0.999977|0.000023|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.904644|0.644700|0.999996|0.822348|0.999978|0.000022|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.772547|0.633346|0.999990|0.816668|0.999971|0.000029|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999998|0.991110|1.000000|0.995555|1.000000|0.000000|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.901236|0.645929|0.999996|0.822963|0.999977|0.000023|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.917622|0.645412|0.999997|0.822704|0.999978|0.000022|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.786953|0.633944|0.999991|0.816967|0.999971|0.000029|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -100,21 +106,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988376|0.989352|0.990330|0.978929|0.989353|0.989353|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.826288|0.747610|0.682613|0.596947|0.757206|0.757196|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.837137|0.752866|0.684009|0.603677|0.763691|0.763681|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.740018|0.696055|0.657023|0.533807|0.699492|0.699478|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989352|0.998036|0.997809|0.997922|0.997922|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.747599|0.990687|0.924490|0.956445|0.956445|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.752855|0.991948|0.924304|0.956932|0.956932|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.696040|0.982615|0.926772|0.953877|0.953877|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998208|0.995534|0.992875|0.991108|0.995544|0.995544|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.835212|0.752518|0.684724|0.603229|0.762978|0.762967|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.846240|0.757814|0.686119|0.610064|0.769574|0.769564|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.750715|0.702210|0.659593|0.541082|0.706318|0.706304|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -122,62 +117,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|21962266|272900|199837|419820269993|272900 / 419820542893|199837 / 22162103|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|14299358|1792049|7862745|419818750844|1792049 / 419820542893|7862745 / 22162103|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|14287912|1506056|7874191|419819036837|1506056 / 419820542893|7874191 / 22162103|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|14036270|4132555|8125833|419816410338|4132555 / 419820542893|8125833 / 22162103|
|
||||
|1|Radixor|PRIMARY_OUTPUT|21921219|39|196636|414653743434|39 / 414653743473|196636 / 22117855|
|
||||
|2|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|14286575|1565633|7831280|414652177840|1565633 / 414653743473|7831280 / 22117855|
|
||||
|3|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|14275129|1281527|7842726|414652461946|1281527 / 414653743473|7842726 / 22117855|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|14021483|3795942|8096372|414649947531|3795942 / 414653743473|8096372 / 22117855|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 419820542893 (0.000000%)|0 / 22162103 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|22162103|0|0|419820542893|0 / 419820542893|0 / 22162103|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 414653743473|0 / 22117855|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|460158 / 419820542893 (0.000110%)|0 / 22162103 (0.000000%)|0.983661|0.989725|0.989777|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -185,7 +158,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.979659|1.000000|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999991|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -193,15 +166,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.983661|0.989725|0.995865|0.979659|0.989777|0.989777|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999993|0.999996|0.999998|0.999991|0.999996|0.999996|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -209,7 +174,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|22162103|460158|0|419820082735|460158 / 419820542893|0 / 22162103|
|
||||
|1|Radixor|ALL_CANDIDATES|22117855|192|0|414653743281|192 / 414653743473|0 / 22117855|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -219,22 +184,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|199837|272900|187258|12320|1.344473%|5|929326|
|
||||
|Radixor|196636|39|153|6664|0.731754%|5|917595|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996163|272775 / 385870694917 (0.000071%)|164277 / 21411411 (0.767240%)|0.988321|0.989820|0.989822|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.821708|1496670 / 385870694917 (0.000388%)|7634885 / 21411411 (35.658019%)|0.834899|0.751079|0.761809|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.821708|1496670 / 385870694917 (0.000388%)|7634885 / 21411411 (35.658019%)|0.834899|0.751079|0.761809|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.815077|3639046 / 385870694917 (0.000943%)|7918708 / 21411411 (36.983588%)|0.750108|0.700135|0.704477|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996227|<0.000001%|0.754564%|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|0.822077|0.000334%|35.584346%|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|0.822077|0.000334%|35.584346%|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|0.815385|0.000869%|36.922109%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -242,10 +207,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987325|0.992328|0.999999|0.996163|0.999999|0.000001|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.902007|0.643420|0.999996|0.821708|0.999976|0.000024|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.902007|0.643420|0.999996|0.821708|0.999976|0.000024|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.787585|0.630164|0.999991|0.815077|0.999970|0.000030|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999998|0.992454|1.000000|0.996227|1.000000|0.000000|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.915319|0.644157|0.999997|0.822077|0.999977|0.000023|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.915319|0.644157|0.999997|0.822077|0.999977|0.000023|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.802756|0.630779|0.999991|0.815385|0.999971|0.000029|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -253,21 +218,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988321|0.989820|0.991323|0.979845|0.989823|0.989822|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.834899|0.751079|0.682555|0.601383|0.761820|0.761809|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.834899|0.751079|0.682555|0.601383|0.761820|0.761809|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.750108|0.700135|0.656404|0.538621|0.704491|0.704477|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989819|0.997945|0.998273|0.998109|0.998109|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.751068|0.991610|0.923288|0.956230|0.956230|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.751068|0.991610|0.923288|0.956230|0.956230|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.700120|0.983687|0.925487|0.953700|0.953700|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998480|0.996212|0.993954|0.992453|0.996219|0.996219|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|0.844241|0.756163|0.684726|0.607928|0.767860|0.767849|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|0.844241|0.756163|0.684726|0.607928|0.767860|0.767849|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.761246|0.706452|0.659016|0.546135|0.711591|0.711577|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -275,62 +229,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|21247134|272775|164277|385870422142|272775 / 385870694917|164277 / 21411411|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|13776526|1496670|7634885|385869198247|1496670 / 385870694917|7634885 / 21411411|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|13776526|1496670|7634885|385869198247|1496670 / 385870694917|7634885 / 21411411|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|13492703|3639046|7918708|385867055871|3639046 / 385870694917|7918708 / 21411411|
|
||||
|1|Radixor|PRIMARY_OUTPUT|21206087|39|161230|380936197647|39 / 380936197686|161230 / 21367317|
|
||||
|2|SNOWBALL HUNGARIAN DIRECT|PRIMARY_OUTPUT|13763897|1273370|7603420|380934924316|1273370 / 380936197686|7603420 / 21367317|
|
||||
|3|SNOWBALL HUNGARIAN LUCENE FILTER|PRIMARY_OUTPUT|13763897|1273370|7603420|380934924316|1273370 / 380936197686|7603420 / 21367317|
|
||||
|4|HUNGARIAN LUCENE HUNGARIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|13478053|3311675|7889264|380932886011|3311675 / 380936197686|7889264 / 21367317|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 385870694917 (0.000000%)|0 / 21411411 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|21411411|0|0|385870694917|0 / 385870694917|0 / 21411411|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 380936197686|0 / 21367317|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|458462 / 385870694917 (0.000119%)|0 / 21411411 (0.000000%)|0.983159|0.989407|0.989462|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -338,7 +270,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.979037|1.000000|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999991|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -346,15 +278,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.983159|0.989407|0.995736|0.979037|0.989463|0.989462|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999993|0.999996|0.999998|0.999991|0.999996|0.999996|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -362,7 +286,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|21411411|458462|0|385870236455|458462 / 385870694917|0 / 21411411|
|
||||
|1|Radixor|ALL_CANDIDATES|21367317|192|0|380936197494|192 / 380936197686|0 / 21367317|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -372,19 +296,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|164277|272775|185687|11153|1.269532%|5|890245|
|
||||
|Radixor|161230|39|153|5518|0.632162%|5|878574|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -393,16 +317,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `HU_HU`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Language Benchmark Pages
|
||||
|
||||
This section splits Radixor stemmer benchmark results by language. Each language page preserves the existing exact-root accuracy and runtime-performance results and adds pairwise stemming-quality tables for both dictionary-processing modes.
|
||||
This section splits Radixor stemmer benchmark results by language. Each of the 20 registered default models has one language page containing the refreshed corpus, patch-command distribution, exact-root accuracy, runtime performance, and pairwise stemming-quality tables for both dictionary-processing modes.
|
||||
|
||||
## Reference Pages
|
||||
|
||||
@@ -23,6 +23,7 @@ This section splits Radixor stemmer benchmark results by language. Each language
|
||||
| Finnish | `FI_FI` | [Finnish](finnish.md) |
|
||||
| French | `FR_FR` | [French](french.md) |
|
||||
| German | `DE_DE` | [German](german.md) |
|
||||
| Hebrew | `HE_IL` | [Hebrew](hebrew.md) |
|
||||
| Hungarian | `HU_HU` | [Hungarian](hungarian.md) |
|
||||
| Italian | `IT_IT` | [Italian](italian.md) |
|
||||
| Norwegian Bokmal | `NB_NO` | [Norwegian Bokmal](norwegian-bokmal.md) |
|
||||
|
||||
@@ -8,20 +8,20 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `IT_IT` | 10,009 | 337,546 | 20,004 | 317,542 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `it-it-default` | `1.0.0` | `IT_IT` | 10,009 | 337,546 | 20,004 | 317,542 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **337,546**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **337,546**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 302,171 | 89.520% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 302,089 | 89.496% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 12,348 | 3.658% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 20,013 | 5.929% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 3,014 | 0.893% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 3,096 | 0.917% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -34,16 +34,22 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 0.041% | 0.043% | 0.010% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Official Snowball direct | 0.041% | 0.043% | 0.010% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `italianRadixor` | 24.491 | 3.128 | 77.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 15.977 | 1.041 | 50.3 | 0.652 | Light Italian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[ITALIAN]` | 109.526 | 12.572 | 344.9 | 4.472 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 116.260 | 7.459 | 366.1 | 4.747 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `italianRadixor` | 25.073 | 0.534 | 79.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 15.956 | 0.184 | 50.2 | 0.636 | Light Italian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[ITALIAN]` | 115.818 | 3.174 | 364.7 | 4.619 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 123.974 | 4.405 | 390.4 | 4.944 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -57,30 +63,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `IT_IT` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `IT_IT` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/it_it/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `it-it-default`, loaded from classpath resource `org/egothor/stemmer/models/it-it-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996507** among 4 deterministic stemmers. The runner-up is `SNOWBALL ITALIAN DIRECT` at 0.866189, a difference of 0.130318. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996512** among 4 deterministic stemmers. The runner-up is `SNOWBALL ITALIAN DIRECT` at 0.866205, a difference of 0.130307. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.996651** among 4 deterministic stemmers. The runner-up is `SNOWBALL ITALIAN DIRECT` at 0.866290, a difference of 0.130361. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.996656** among 4 deterministic stemmers. The runner-up is `SNOWBALL ITALIAN DIRECT` at 0.866307, a difference of 0.130350. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996507|124172 / 53638521211 (0.000231%)|42908 / 6143814 (0.698394%)|0.982618|0.986492|0.986512|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.866189|504775 / 53638521211 (0.000941%)|1644164 / 6143814 (26.761292%)|0.859975|0.807240|0.811470|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.866189|504775 / 53638521211 (0.000941%)|1644164 / 6143814 (26.761292%)|0.859975|0.807240|0.811470|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.508926|10589 / 53638521211 (0.000020%)|6034130 / 6143814 (98.214725%)|0.082782|0.035020|0.127588|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996651|0.000000%|0.669827%|
|
||||
|2|SNOWBALL ITALIAN DIRECT|0.866290|0.000738%|26.741219%|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|0.866290|0.000738%|26.741219%|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|0.508920|0.000005%|98.216094%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -88,10 +94,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.980053|0.993016|0.999998|0.996507|0.999997|0.000003|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.899134|0.732387|0.999991|0.866189|0.999960|0.000040|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.899134|0.732387|0.999991|0.866189|0.999960|0.000040|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.911959|0.017853|1.000000|0.508926|0.999887|0.000113|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.993302|1.000000|0.996651|0.999999|0.000001|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.920474|0.732588|0.999993|0.866290|0.999961|0.000039|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.920474|0.732588|0.999993|0.866290|0.999961|0.000039|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.975468|0.017839|1.000000|0.508920|0.999885|0.000115|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -99,21 +105,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.982618|0.986492|0.990396|0.973344|0.986513|0.986512|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.859975|0.807240|0.760598|0.676783|0.811489|0.811470|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.859975|0.807240|0.760598|0.676783|0.811489|0.811470|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.082782|0.035020|0.022207|0.017822|0.127597|0.127588|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986490|0.995780|0.997113|0.996446|0.996446|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.807220|0.987994|0.933408|0.959925|0.959925|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.807220|0.987994|0.933408|0.959925|0.959925|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.035016|0.997481|0.737537|0.848037|0.848037|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998653|0.996640|0.994634|0.993302|0.996645|0.996645|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.875563|0.815854|0.763768|0.688980|0.821175|0.821157|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.875563|0.815854|0.763768|0.688980|0.821175|0.821157|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.083115|0.035037|0.022197|0.017831|0.131914|0.131907|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -121,62 +116,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6100906|124172|42908|53638397039|124172 / 53638521211|42908 / 6143814|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|4499650|504775|1644164|53638016436|504775 / 53638521211|1644164 / 6143814|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|4499650|504775|1644164|53638016436|504775 / 53638521211|1644164 / 6143814|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|109684|10589|6034130|53638510622|10589 / 53638521211|6034130 / 6143814|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6093034|0|41088|52600354673|0 / 52600354673|41088 / 6134122|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|4493783|388246|1640339|52599966427|388246 / 52600354673|1640339 / 6134122|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|4493783|388246|1640339|52599966427|388246 / 52600354673|1640339 / 6134122|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|109427|2752|6024695|52600351921|2752 / 52600354673|6024695 / 6134122|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999993|0 / 53638521211 (0.000000%)|80 / 6143814 (0.001302%)|0.999997|0.999993|0.999993|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.001304%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0.999987|1.000000|0.999993|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999997|0.999993|0.999990|0.999987|0.999993|0.999993|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|6143734|0|80|53638521211|0 / 53638521211|80 / 6143814|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 52600354673|80 / 6134122|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999992|170950 / 53638521211 (0.000319%)|80 / 6143814 (0.001302%)|0.978222|0.986272|0.986363|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.999993|0.000000%|0.001304%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -184,7 +157,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.972928|0.999987|0.999997|0.999992|0.999997|0.000003|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|0.999987|1.000000|0.999993|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -192,15 +165,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.978222|0.986272|0.994455|0.972916|0.986365|0.986363|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999997|0.999993|0.999990|0.999987|0.999993|0.999993|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -208,7 +173,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|6143734|170950|80|53638350261|170950 / 53638521211|80 / 6143814|
|
||||
|1|Radixor|ALL_CANDIDATES|6134042|0|80|52600354673|0 / 52600354673|80 / 6134122|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -218,22 +183,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|42828|124172|46778|6254|1.909321%|4|334175|
|
||||
|Radixor|41008|0|0|3069|0.946153%|4|327552|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996512|124171 / 53611667072 (0.000232%)|42828 / 6142174 (0.697278%)|0.982617|0.986495|0.986515|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.866205|504774 / 53611667072 (0.000942%)|1643522 / 6142174 (26.757985%)|0.859970|0.807252|0.811479|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.866205|504774 / 53611667072 (0.000942%)|1643522 / 6142174 (26.757985%)|0.859970|0.807252|0.811479|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.508927|10588 / 53611667072 (0.000020%)|6032516 / 6142174 (98.214671%)|0.082784|0.035021|0.127589|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.996656|0.000000%|0.668702%|
|
||||
|2|SNOWBALL ITALIAN DIRECT|0.866307|0.000738%|26.737902%|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|0.866307|0.000738%|26.737902%|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|0.508920|0.000005%|98.216040%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -241,10 +206,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.980048|0.993027|0.999998|0.996512|0.999997|0.000003|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.899114|0.732420|0.999991|0.866205|0.999960|0.000040|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.899114|0.732420|0.999991|0.866205|0.999960|0.000040|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.911947|0.017853|1.000000|0.508927|0.999887|0.000113|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.993313|1.000000|0.996656|0.999999|0.000001|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.920458|0.732621|0.999993|0.866307|0.999961|0.000039|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.920458|0.732621|0.999993|0.866307|0.999961|0.000039|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.975462|0.017840|1.000000|0.508920|0.999885|0.000115|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -252,21 +217,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.982617|0.986495|0.990404|0.973350|0.986516|0.986515|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.859970|0.807252|0.760624|0.676800|0.811498|0.811479|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.859970|0.807252|0.760624|0.676800|0.811498|0.811479|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.082784|0.035021|0.022208|0.017823|0.127598|0.127589|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986493|0.995780|0.997115|0.996447|0.996447|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.807232|0.987991|0.933413|0.959927|0.959927|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.807232|0.987991|0.933413|0.959927|0.959927|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.035017|0.997481|0.737534|0.848035|0.848035|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998655|0.996645|0.994643|0.993313|0.996651|0.996650|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|0.875561|0.815868|0.763794|0.689001|0.821186|0.821168|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|0.875561|0.815868|0.763794|0.689001|0.821186|0.821168|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.083118|0.035038|0.022198|0.017832|0.131916|0.131908|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -274,62 +228,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6099346|124171|42828|53611542901|124171 / 53611667072|42828 / 6142174|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|4498652|504774|1643522|53611162298|504774 / 53611667072|1643522 / 6142174|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|4498652|504774|1643522|53611162298|504774 / 53611667072|1643522 / 6142174|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|109658|10588|6032516|53611656484|10588 / 53611667072|6032516 / 6142174|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6091474|0|41008|52574085988|0 / 52574085988|41008 / 6132482|
|
||||
|2|SNOWBALL ITALIAN DIRECT|PRIMARY_OUTPUT|4492785|388246|1639697|52573697742|388246 / 52574085988|1639697 / 6132482|
|
||||
|3|SNOWBALL ITALIAN LUCENE FILTER|PRIMARY_OUTPUT|4492785|388246|1639697|52573697742|388246 / 52574085988|1639697 / 6132482|
|
||||
|4|ITALIAN LUCENE ITALIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|109401|2752|6023081|52574083236|2752 / 52574085988|6023081 / 6132482|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 53611667072 (0.000000%)|0 / 6142174 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|6142174|0|0|53611667072|0 / 53611667072|0 / 6142174|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 52574085988|0 / 6132482|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|170949 / 53611667072 (0.000319%)|0 / 6142174 (0.000000%)|0.978219|0.986275|0.986366|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -337,7 +269,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.972922|1.000000|0.999997|0.999998|0.999997|0.000003|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -345,15 +277,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.978219|0.986275|0.994464|0.972922|0.986368|0.986366|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -361,7 +285,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|6142174|170949|0|53611496123|170949 / 53611667072|0 / 6142174|
|
||||
|1|Radixor|ALL_CANDIDATES|6132482|0|0|52574085988|0 / 52574085988|0 / 6132482|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -371,19 +295,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|42828|124171|46778|6252|1.909188%|4|334089|
|
||||
|Radixor|41008|0|0|3068|0.946081%|4|327469|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -392,16 +316,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `IT_IT`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `NB_NO` | 17,929 | 90,757 | 33,376 | 57,381 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `nb-no-default` | `1.0.0` | `NB_NO` | 17,929 | 90,757 | 33,376 | 57,381 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **90,757**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **90,757**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 1,500 | 1.653% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 4,296 | 4.734% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 47,619 | 52.469% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 34,420 | 37.925% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 2,922 | 3.220% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 2,528 | 2.785% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 4,258 | 4.692% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 48,925 | 53.908% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 32,086 | 35.354% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 2,960 | 3.261% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -36,17 +36,23 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 54.803% | 51.780% | 60.001% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Lucene NorwegianLightStemFilter | 52.136% | 50.616% | 54.749% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `norwegianBokmalRadixor` | 3.631 | 1.377 | 63.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.910 | 0.177 | 50.7 | 0.801 | Minimal Norwegian suffix reducer. |
|
||||
| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.335 | 0.116 | 58.1 | 0.919 | Light Norwegian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 4.277 | 0.082 | 74.5 | 1.178 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 6.077 | 0.208 | 105.9 | 1.674 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `norwegianBokmalRadixor` | 3.401 | 0.055 | 59.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.943 | 0.023 | 51.3 | 0.865 | Minimal Norwegian suffix reducer. |
|
||||
| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.358 | 0.036 | 58.5 | 0.987 | Light Norwegian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 4.378 | 0.295 | 76.3 | 1.287 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 6.114 | 0.436 | 106.5 | 1.797 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -60,31 +66,31 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NB_NO` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NB_NO` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/nb_no/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `nb-no-default`, loaded from classpath resource `org/egothor/stemmer/models/nb-no-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.974783** among 5 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN BOKMAL DIRECT` at 0.874964, a difference of 0.099819. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.975000** among 5 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN BOKMAL DIRECT` at 0.874991, a difference of 0.100009. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.976021** among 5 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN BOKMAL DIRECT` at 0.874259, a difference of 0.101762. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.976240** among 5 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN BOKMAL DIRECT` at 0.874286, a difference of 0.101954. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.974783|11482 / 2835618215 (0.000405%)|7170 / 142180 (5.042903%)|0.927078|0.935387|0.935488|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.874964|23997 / 2835618215 (0.000846%)|35554 / 142180 (25.006330%)|0.802095|0.781707|0.782399|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.874834|24046 / 2835618215 (0.000848%)|35591 / 142180 (25.032353%)|0.801759|0.781401|0.782091|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.850006|25171 / 2835618215 (0.000888%)|42651 / 142180 (29.997890%)|0.776381|0.745871|0.747464|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.832414|14772 / 2835618215 (0.000521%)|47654 / 142180 (33.516669%)|0.815763|0.751764|0.758263|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.976021|0.000000%|4.795770%|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.874259|0.000386%|25.147805%|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|0.874138|0.000389%|25.171937%|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|0.849389|0.000416%|30.121722%|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|0.831282|0.000110%|33.743568%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -92,11 +98,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.921620|0.949571|0.999996|0.974783|0.999993|0.000007|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.816288|0.749937|0.999992|0.874964|0.999979|0.000021|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.815930|0.749676|0.999992|0.874834|0.999979|0.000021|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.798148|0.700021|0.999991|0.850006|0.999976|0.000024|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.864847|0.664833|0.999995|0.832414|0.999978|0.000022|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.952042|1.000000|0.976021|0.999997|0.000003|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.910734|0.748522|0.999996|0.874259|0.999983|0.000017|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.910189|0.748281|0.999996|0.874138|0.999983|0.000017|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.898501|0.698783|0.999996|0.849389|0.999980|0.000020|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.969387|0.662564|0.999999|0.831282|0.999981|0.000019|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -104,23 +110,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.927078|0.935387|0.943846|0.878617|0.935491|0.935488|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.802095|0.781707|0.762330|0.641641|0.782409|0.782399|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.801759|0.781401|0.762052|0.641229|0.782102|0.782091|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.776381|0.745871|0.717668|0.594732|0.747476|0.747464|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.815763|0.751764|0.697076|0.602261|0.758274|0.758263|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.935384|0.993354|0.994615|0.993984|0.993984|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.781696|0.988328|0.971120|0.979648|0.979648|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.781391|0.988295|0.971086|0.979615|0.979615|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.745859|0.987774|0.965622|0.976573|0.976573|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.751753|0.992089|0.962516|0.977079|0.977079|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990026|0.975432|0.961262|0.952042|0.975727|0.975725|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.872901|0.821699|0.776171|0.697359|0.825654|0.825646|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.872435|0.821332|0.775884|0.696830|0.825274|0.825266|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.849918|0.786156|0.731293|0.647658|0.792374|0.792365|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.887216|0.787133|0.707341|0.648985|0.801425|0.801417|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -128,63 +122,41 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|135010|11482|7170|2835606733|11482 / 2835618215|7170 / 142180|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|106626|23997|35554|2835594218|23997 / 2835618215|35554 / 142180|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|106589|24046|35591|2835594169|24046 / 2835618215|35591 / 142180|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|99529|25171|42651|2835593044|25171 / 2835618215|42651 / 142180|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|94526|14772|47654|2835603443|14772 / 2835618215|47654 / 142180|
|
||||
|1|Radixor|PRIMARY_OUTPUT|134138|0|6757|2676746970|0 / 2676746970|6757 / 140895|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|105463|10337|35432|2676736633|10337 / 2676746970|35432 / 140895|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|105429|10403|35466|2676736567|10403 / 2676746970|35466 / 140895|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|98455|11122|42440|2676735848|11122 / 2676746970|42440 / 140895|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|93352|2948|47543|2676744022|2948 / 2676746970|47543 / 140895|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 2835618215 (0.000000%)|0 / 142180 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|142180|0|0|2835618215|0 / 2835618215|0 / 142180|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 2676746970|0 / 140895|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999996|20161 / 2835618215 (0.000711%)|0 / 142180 (0.000000%)|0.898118|0.933794|0.935844|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -192,7 +164,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.875811|1.000000|0.999993|0.999996|0.999993|0.000007|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -200,15 +172,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.898118|0.933794|0.972422|0.875811|0.935848|0.935844|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -216,7 +180,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|142180|20161|0|2835598054|20161 / 2835618215|0 / 142180|
|
||||
|1|Radixor|ALL_CANDIDATES|140895|0|0|2676746970|0 / 2676746970|0 / 140895|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -226,23 +190,23 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|7170|11482|8679|4237|5.626079%|9|79825|
|
||||
|Radixor|6757|0|0|2097|2.865929%|9|75343|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.975000|11482 / 2831176784 (0.000406%)|7104 / 142091 (4.999613%)|0.927151|0.935591|0.935695|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.874991|23997 / 2831176784 (0.000848%)|35524 / 142091 (25.000880%)|0.802043|0.781698|0.782388|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.874798|23993 / 2831176784 (0.000847%)|35579 / 142091 (25.039587%)|0.801914|0.781464|0.782161|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.849947|25118 / 2831176784 (0.000887%)|42641 / 142091 (30.009642%)|0.776513|0.745896|0.747500|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.832344|14719 / 2831176784 (0.000520%)|47644 / 142091 (33.530625%)|0.815950|0.751796|0.758325|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.976240|0.000000%|4.751928%|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.874286|0.000387%|25.142395%|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|0.874101|0.000387%|25.179325%|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|0.849330|0.000414%|30.133659%|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|0.831210|0.000108%|33.757794%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -250,11 +214,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.921608|0.950004|0.999996|0.975000|0.999993|0.000007|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.816205|0.749991|0.999992|0.874991|0.999979|0.000021|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.816153|0.749604|0.999992|0.874798|0.999979|0.000021|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.798359|0.699904|0.999991|0.849947|0.999976|0.000024|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.865169|0.664694|0.999995|0.832344|0.999978|0.000022|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.952481|1.000000|0.976240|0.999997|0.000003|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.910689|0.748576|0.999996|0.874286|0.999983|0.000017|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.910546|0.748207|0.999996|0.874101|0.999983|0.000017|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.898862|0.698663|0.999996|0.849330|0.999980|0.000020|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.969896|0.662422|0.999999|0.831210|0.999981|0.000019|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -262,23 +226,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.927151|0.935591|0.944186|0.878976|0.935698|0.935695|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.802043|0.781698|0.762360|0.641630|0.782398|0.782388|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.801914|0.781464|0.762031|0.641314|0.782171|0.782161|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.776513|0.745896|0.717603|0.594765|0.747512|0.747500|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.815950|0.751796|0.696995|0.602302|0.758335|0.758325|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.935587|0.993348|0.994694|0.994020|0.994020|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.781688|0.988318|0.971127|0.979647|0.979647|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.781454|0.988310|0.971074|0.979616|0.979616|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.745885|0.987789|0.965603|0.976570|0.976570|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.751785|0.992107|0.962494|0.977076|0.977076|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990121|0.975662|0.961620|0.952481|0.975951|0.975950|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|0.872882|0.821713|0.776211|0.697379|0.825663|0.825655|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|0.872677|0.821432|0.775872|0.696975|0.825395|0.825387|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.850142|0.786219|0.731236|0.647743|0.792466|0.792457|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.887506|0.787200|0.707265|0.649077|0.801549|0.801541|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -286,63 +238,41 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|134987|11482|7104|2831165302|11482 / 2831176784|7104 / 142091|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|106567|23997|35524|2831152787|23997 / 2831176784|35524 / 142091|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|106512|23993|35579|2831152791|23993 / 2831176784|35579 / 142091|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|99450|25118|42641|2831151666|25118 / 2831176784|42641 / 142091|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|94447|14719|47644|2831162065|14719 / 2831176784|47644 / 142091|
|
||||
|1|Radixor|PRIMARY_OUTPUT|134115|0|6691|2672431799|0 / 2672431799|6691 / 140806|
|
||||
|2|SNOWBALL NORWEGIAN BOKMAL DIRECT|PRIMARY_OUTPUT|105404|10337|35402|2672421462|10337 / 2672431799|35402 / 140806|
|
||||
|3|SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|PRIMARY_OUTPUT|105352|10350|35454|2672421449|10350 / 2672431799|35454 / 140806|
|
||||
|4|NORWEGIAN BOKMAL LUCENE NORWEGIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|98376|11069|42430|2672420730|11069 / 2672431799|42430 / 140806|
|
||||
|5|NORWEGIAN BOKMAL LUCENE NORWEGIAN MINIMAL STEM FILTER|PRIMARY_OUTPUT|93273|2895|47533|2672428904|2895 / 2672431799|47533 / 140806|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 2831176784 (0.000000%)|0 / 142091 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|142091|0|0|2831176784|0 / 2831176784|0 / 142091|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 2672431799|0 / 140806|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999996|20161 / 2831176784 (0.000712%)|0 / 142091 (0.000000%)|0.898061|0.933756|0.935808|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -350,7 +280,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.875743|1.000000|0.999993|0.999996|0.999993|0.000007|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -358,15 +288,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.898061|0.933756|0.972405|0.875743|0.935811|0.935808|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -374,7 +296,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|142091|20161|0|2831156623|20161 / 2831176784|0 / 142091|
|
||||
|1|Radixor|ALL_CANDIDATES|140806|0|0|2672431799|0 / 2672431799|0 / 140806|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -384,19 +306,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|7104|11482|8679|4204|5.586637%|9|79733|
|
||||
|Radixor|6691|0|0|2064|2.823105%|9|75251|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -405,16 +327,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `NB_NO`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `NN_NO` | 4,688 | 19,651 | 6,089 | 13,562 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `nn-no-default` | `1.0.0` | `NN_NO` | 4,688 | 19,651 | 6,089 | 13,562 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **19,651**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **19,651**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 224 | 1.140% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 1,505 | 7.659% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 11,017 | 56.063% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 6,427 | 32.706% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 478 | 2.432% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 312 | 1.588% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 1,456 | 7.409% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 11,325 | 57.631% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 6,031 | 30.691% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 527 | 2.682% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -34,15 +34,21 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 60.974% | 60.212% | 62.670% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene SnowballFilter | 60.918% | 60.146% | 62.638% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.571 | 0.012 | 42.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 0.919 | 0.038 | 67.7 | 1.609 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.309 | 0.015 | 96.5 | 2.292 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.617 | 0.062 | 45.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 0.955 | 0.076 | 70.4 | 1.548 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.352 | 0.106 | 99.7 | 2.191 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -56,29 +62,29 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NN_NO` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NN_NO` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/nn_no/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `nn-no-default`, loaded from classpath resource `org/egothor/stemmer/models/nn-no-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.935777** among 3 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN NYNORSK DIRECT` at 0.858908, a difference of 0.076869. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.935853** among 3 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN NYNORSK DIRECT` at 0.859037, a difference of 0.076816. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.950991** among 3 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN NYNORSK DIRECT` at 0.868094, a difference of 0.082897. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.951104** among 3 deterministic stemmers. The runner-up is `SNOWBALL NORWEGIAN NYNORSK DIRECT` at 0.868252, a difference of 0.082852. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.935777|6230 / 166491473 (0.003742%)|3936 / 30652 (12.840924%)|0.822355|0.840152|0.840669|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.858908|8274 / 166491473 (0.004970%)|8648 / 30652 (28.213493%)|0.724941|0.722271|0.722234|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.858484|8295 / 166491473 (0.004982%)|8674 / 30652 (28.298317%)|0.724180|0.721477|0.721440|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.950991|0.000000%|9.801848%|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.868094|0.000838%|26.380368%|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|0.867636|0.000852%|26.472040%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -86,9 +92,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.810903|0.871591|0.999963|0.935777|0.999939|0.000061|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.726732|0.717865|0.999950|0.858908|0.999898|0.000102|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.725993|0.717017|0.999950|0.858484|0.999898|0.000102|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.901982|1.000000|0.950991|0.999981|0.000019|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.945609|0.736196|0.999992|0.868094|0.999939|0.000061|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.944646|0.735280|0.999991|0.867636|0.999939|0.000061|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -96,19 +102,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.822355|0.840152|0.858737|0.724364|0.840699|0.840669|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.724941|0.722271|0.719621|0.565278|0.722285|0.722234|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.724180|0.721477|0.718794|0.564305|0.721491|0.721440|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.840122|0.983845|0.986802|0.985321|0.985321|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.722221|0.980542|0.964998|0.972708|0.972708|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.721426|0.980461|0.964862|0.972599|0.972599|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.978728|0.948465|0.920017|0.901982|0.949727|0.949718|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.894709|0.827865|0.770315|0.706288|0.834359|0.834331|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.893748|0.826916|0.769384|0.704908|0.833414|0.833386|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -116,61 +112,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|26716|6230|3936|166485243|6230 / 166491473|3936 / 30652|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|22004|8274|8648|166483199|8274 / 166491473|8648 / 30652|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|21978|8295|8674|166483178|8295 / 166491473|8674 / 30652|
|
||||
|1|Radixor|PRIMARY_OUTPUT|25582|0|2780|143394154|0 / 143394154|2780 / 28362|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|20880|1201|7482|143392953|1201 / 143394154|7482 / 28362|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|20854|1222|7508|143392932|1222 / 143394154|7508 / 28362|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 166491473 (0.000000%)|0 / 30652 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|30652|0|0|166491473|0 / 166491473|0 / 30652|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 143394154|0 / 28362|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999960|13214 / 166491473 (0.007937%)|0 / 30652 (0.000000%)|0.743562|0.822674|0.835888|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -178,7 +152,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.698764|1.000000|0.999921|0.999960|0.999921|0.000079|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -186,15 +160,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.743562|0.822674|0.920624|0.698764|0.835921|0.835888|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -202,7 +168,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|30652|13214|0|166478259|13214 / 166491473|0 / 30652|
|
||||
|1|Radixor|ALL_CANDIDATES|28362|0|0|143394154|0 / 143394154|0 / 28362|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -212,21 +178,21 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|3936|6230|6984|2404|13.172603%|5|21513|
|
||||
|Radixor|2780|0|0|1091|6.441519%|5|18255|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.935853|6230 / 165926276 (0.003755%)|3924 / 30595 (12.825625%)|0.822169|0.840084|0.840609|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.859037|8274 / 165926276 (0.004987%)|8624 / 30595 (28.187612%)|0.724757|0.722255|0.722216|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.858661|8274 / 165926276 (0.004987%)|8647 / 30595 (28.262788%)|0.724438|0.721772|0.721734|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.951104|0.000000%|9.779191%|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.868252|0.000841%|26.348702%|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|0.867846|0.000841%|26.429959%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -234,9 +200,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.810644|0.871744|0.999962|0.935853|0.999939|0.000061|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.726434|0.718124|0.999950|0.859037|0.999898|0.000102|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.726226|0.717372|0.999950|0.858661|0.999898|0.000102|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.902208|1.000000|0.951104|0.999981|0.000019|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.945528|0.736513|0.999992|0.868252|0.999939|0.000061|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.945471|0.735700|0.999992|0.867846|0.999939|0.000061|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -244,19 +210,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.822169|0.840084|0.858798|0.724263|0.840639|0.840609|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.724757|0.722255|0.719771|0.565258|0.722267|0.722216|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.724438|0.721772|0.719126|0.564666|0.721785|0.721734|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.840054|0.983815|0.986842|0.985326|0.985326|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.722204|0.980506|0.965065|0.972724|0.972724|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.721721|0.980506|0.964945|0.972663|0.972663|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.978782|0.948590|0.920206|0.902208|0.949846|0.949837|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|0.894744|0.828034|0.770581|0.706534|0.834502|0.834474|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|0.894463|0.827499|0.769862|0.705755|0.834016|0.833989|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -264,61 +220,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|26671|6230|3924|165920046|6230 / 165926276|3924 / 30595|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|21971|8274|8624|165918002|8274 / 165926276|8624 / 30595|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|21948|8274|8647|165918002|8274 / 165926276|8647 / 30595|
|
||||
|1|Radixor|PRIMARY_OUTPUT|25537|0|2768|142869660|0 / 142869660|2768 / 28305|
|
||||
|2|SNOWBALL NORWEGIAN NYNORSK DIRECT|PRIMARY_OUTPUT|20847|1201|7458|142868459|1201 / 142869660|7458 / 28305|
|
||||
|3|SNOWBALL NORWEGIAN NYNORSK LUCENE FILTER|PRIMARY_OUTPUT|20824|1201|7481|142868459|1201 / 142869660|7481 / 28305|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 165926276 (0.000000%)|0 / 30595 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|30595|0|0|165926276|0 / 165926276|0 / 30595|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 142869660|0 / 28305|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999960|13214 / 165926276 (0.007964%)|0 / 30595 (0.000000%)|0.743207|0.822402|0.835654|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -326,7 +260,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.698372|1.000000|0.999920|0.999960|0.999920|0.000080|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -334,15 +268,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.743207|0.822402|0.920488|0.698372|0.835687|0.835654|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -350,7 +276,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|30595|13214|0|165913062|13214 / 165926276|0 / 30595|
|
||||
|1|Radixor|ALL_CANDIDATES|28305|0|0|142869660|0 / 142869660|0 / 28305|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -360,19 +286,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|3924|6230|6984|2399|13.167572%|5|21477|
|
||||
|Radixor|2768|0|0|1086|6.423755%|5|18219|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -381,16 +307,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `NN_NO`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,13 +8,13 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `FA_IR` | 69 | 3,770 | 138 | 3,632 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `fa-ir-default` | `1.0.0` | `FA_IR` | 69 | 3,770 | 138 | 3,632 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **3,770**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **3,770**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
@@ -31,14 +31,20 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Radixor | 95.836% | 95.677% | 100.000% | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene PersianStemFilter | 1.485% | 0.000% | 40.580% | Lucene Persian suffix stemmer with required normalization in the measured path. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `persianRadixor` | 0.245 | 0.025 | 49.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.466 | 0.015 | 93.3 | 1.902 | Persian suffix stemmer with Lucene normalization in the measured path. |
|
||||
| Radixor | `persianRadixor` | 0.243 | 0.004 | 66.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.469 | 0.007 | 129.1 | 1.930 | Persian suffix stemmer with Lucene normalization in the measured path. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -52,28 +58,28 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FA_IR` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FA_IR` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/fa_ir/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `fa-ir-default`, loaded from classpath resource `org/egothor/stemmer/models/fa-ir-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.974922** among 2 deterministic stemmers. The runner-up is `PERSIAN LUCENE PERSIAN STEM FILTER` at 0.502171, a difference of 0.472751. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.974922** among 2 deterministic stemmers. The runner-up is `PERSIAN LUCENE PERSIAN STEM FILTER` at 0.502171, a difference of 0.472751. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.976360** among 2 deterministic stemmers. The runner-up is `PERSIAN LUCENE PERSIAN STEM FILTER` at 0.502212, a difference of 0.474148. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.976360** among 2 deterministic stemmers. The runner-up is `PERSIAN LUCENE PERSIAN STEM FILTER` at 0.502212, a difference of 0.474148. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.974922|8621 / 6748402 (0.127749%)|4812 / 98448 (4.887860%)|0.922566|0.933071|0.932249|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.502171|179 / 6748402 (0.002652%)|98018 / 98448 (99.563221%)|0.021312|0.008682|0.054801|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.976360|0.000000%|4.728041%|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|0.502212|0.000049%|99.557494%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -81,8 +87,8 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.915693|0.951121|0.998723|0.974922|0.998038|0.001962|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.706076|0.004368|0.999973|0.502171|0.985658|0.014342|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.952720|1.000000|0.976360|0.999277|0.000723|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.992991|0.004425|1.000000|0.502212|0.984769|0.015231|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -90,17 +96,8 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.922566|0.933071|0.943818|0.874539|0.933239|0.932249|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.021312|0.008682|0.005451|0.004360|0.055534|0.054801|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.932076|0.980343|0.984586|0.982460|0.982460|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.008507|0.985686|0.520347|0.681125|0.681125|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990172|0.975787|0.961815|0.952720|0.976074|0.975715|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.021738|0.008811|0.005525|0.004425|0.066288|0.065774|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -108,60 +105,38 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|93636|8621|4812|6739781|8621 / 6748402|4812 / 98448|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|430|179|98018|6748223|179 / 6748402|98018 / 98448|
|
||||
|1|Radixor|PRIMARY_OUTPUT|91503|0|4541|6182152|0 / 6182152|4541 / 96044|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|425|3|95619|6182149|3 / 6182152|95619 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 6748402 (0.000000%)|0 / 98448 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|98448|0|0|6748402|0 / 6748402|0 / 98448|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 6182152|0 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999005|13433 / 6748402 (0.199055%)|0 / 98448 (0.000000%)|0.901585|0.936133|0.937114|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -169,7 +144,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.879935|1.000000|0.998009|0.999005|0.998038|0.001962|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -177,15 +152,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.901585|0.936133|0.973435|0.879935|0.938048|0.937114|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -193,7 +160,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|98448|13433|0|6734969|13433 / 6748402|0 / 98448|
|
||||
|1|Radixor|ALL_CANDIDATES|96044|0|0|6182152|0 / 6182152|0 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -203,20 +170,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|4812|8621|4812|314|8.484193%|2|4015|
|
||||
|Radixor|4541|0|0|157|4.430023%|2|3701|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.974922|8621 / 6748402 (0.127749%)|4812 / 98448 (4.887860%)|0.922566|0.933071|0.932249|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.502171|179 / 6748402 (0.002652%)|98018 / 98448 (99.563221%)|0.021312|0.008682|0.054801|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.976360|0.000000%|4.728041%|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|0.502212|0.000049%|99.557494%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -224,8 +191,8 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.915693|0.951121|0.998723|0.974922|0.998038|0.001962|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.706076|0.004368|0.999973|0.502171|0.985658|0.014342|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.952720|1.000000|0.976360|0.999277|0.000723|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.992991|0.004425|1.000000|0.502212|0.984769|0.015231|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -233,17 +200,8 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.922566|0.933071|0.943818|0.874539|0.933239|0.932249|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.021312|0.008682|0.005451|0.004360|0.055534|0.054801|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.932076|0.980343|0.984586|0.982460|0.982460|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.008507|0.985686|0.520347|0.681125|0.681125|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990172|0.975787|0.961815|0.952720|0.976074|0.975715|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|0.021738|0.008811|0.005525|0.004425|0.066288|0.065774|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -251,60 +209,38 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|93636|8621|4812|6739781|8621 / 6748402|4812 / 98448|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|430|179|98018|6748223|179 / 6748402|98018 / 98448|
|
||||
|1|Radixor|PRIMARY_OUTPUT|91503|0|4541|6182152|0 / 6182152|4541 / 96044|
|
||||
|2|PERSIAN LUCENE PERSIAN STEM FILTER|PRIMARY_OUTPUT|425|3|95619|6182149|3 / 6182152|95619 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 6748402 (0.000000%)|0 / 98448 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|98448|0|0|6748402|0 / 6748402|0 / 98448|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 6182152|0 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999005|13433 / 6748402 (0.199055%)|0 / 98448 (0.000000%)|0.901585|0.936133|0.937114|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -312,7 +248,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.879935|1.000000|0.998009|0.999005|0.998038|0.001962|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -320,15 +256,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.901585|0.936133|0.973435|0.879935|0.938048|0.937114|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -336,7 +264,7 @@ This mode contains **4 result rows**, **2 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|98448|13433|0|6734969|13433 / 6748402|0 / 98448|
|
||||
|1|Radixor|ALL_CANDIDATES|96044|0|0|6182152|0 / 6182152|0 / 96044|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -346,19 +274,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|4812|8621|4812|314|8.484193%|2|4015|
|
||||
|Radixor|4541|0|0|157|4.430023%|2|3701|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -367,16 +295,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `FA_IR`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `PL_PL` | 9,990 | 132,308 | 19,957 | 112,351 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `pl-pl-unimorph` | `1.0.0` | `PL_PL` | 9,990 | 132,308 | 19,957 | 112,351 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **132,308**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **132,308**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 1,719 | 1.299% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 53,303 | 40.287% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 37,051 | 28.004% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 20,415 | 15.430% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 19,820 | 14.980% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 1,836 | 1.388% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 52,996 | 40.055% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 37,137 | 28.069% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 20,219 | 15.282% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 20,120 | 15.207% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -36,17 +36,23 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene StempelFilter | 70.009% | 69.262% | 74.220% | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
||||
| Lucene StempelStemmer direct | 70.009% | 69.262% | 74.220% | Direct table-driven Polish Stempel stemmer API. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `polishRadixor` | 9.049 | 0.485 | 80.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 483.316 | 11.455 | 4301.8 | 53.408 | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 41.932 | 1.916 | 373.2 | 4.634 | Direct table-driven Polish Stempel stemmer API. |
|
||||
| Lucene StempelFilter | `polishLuceneStempelFilter` | 45.277 | 13.693 | 403.0 | 5.003 | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
||||
| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 135.763 | 31.634 | 1208.4 | 15.002 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||
| Radixor | `polishRadixor` | 8.972 | 0.203 | 79.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 524.081 | 35.121 | 4664.7 | 58.412 | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 37.947 | 0.335 | 337.8 | 4.229 | Direct table-driven Polish Stempel stemmer API. |
|
||||
| Lucene StempelFilter | `polishLuceneStempelFilter` | 43.090 | 0.411 | 383.5 | 4.803 | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
||||
| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 143.527 | 1.176 | 1277.5 | 15.997 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -60,31 +66,31 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PL_PL` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PL_PL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/pl_pl/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `pl-pl-unimorph`, loaded from classpath resource `org/egothor/stemmer/models/pl-pl-unimorph/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.990388** among 5 deterministic stemmers. The runner-up is `POLISH LUCENE MORFOLOGIK FILTER` at 0.948154, a difference of 0.042234. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.990579** among 5 deterministic stemmers. The runner-up is `POLISH LUCENE MORFOLOGIK FILTER` at 0.948177, a difference of 0.042402. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.991105** among 5 deterministic stemmers. The runner-up is `POLISH LUCENE MORFOLOGIK FILTER` at 0.948392, a difference of 0.042713. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.991301** among 5 deterministic stemmers. The runner-up is `POLISH LUCENE MORFOLOGIK FILTER` at 0.948417, a difference of 0.042884. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990388|13669 / 7482478003 (0.000183%)|21547 / 1120967 (1.922180%)|0.986324|0.984237|0.984241|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.948154|99228 / 7482478003 (0.001326%)|116220 / 1120967 (10.367834%)|0.907324|0.903167|0.903179|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.933222|52652 / 7482478003 (0.000704%)|149705 / 1120967 (13.354987%)|0.930930|0.905656|0.906571|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.855748|66669 / 7482478003 (0.000891%)|323394 / 1120967 (28.849556%)|0.871106|0.803515|0.810296|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.855748|66669 / 7482478003 (0.000891%)|323394 / 1120967 (28.849556%)|0.871106|0.803515|0.810296|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.991105|0.000000%|1.779024%|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|0.948392|0.001042%|10.320543%|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|0.933457|0.000383%|13.308172%|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|0.855699|0.000602%|28.859618%|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|0.855699|0.000602%|28.859618%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -92,11 +98,11 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987720|0.980778|0.999998|0.990388|0.999995|0.000005|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.910118|0.896322|0.999987|0.948154|0.999971|0.000029|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.948578|0.866450|0.999993|0.933222|0.999973|0.000027|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.922858|0.711504|0.999991|0.855748|0.999948|0.000052|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.922858|0.711504|0.999991|0.855748|0.999948|0.000052|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.982210|1.000000|0.991105|0.999997|0.000003|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.929398|0.896795|0.999990|0.948392|0.999974|0.000026|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.971931|0.866918|0.999996|0.933457|0.999976|0.000024|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.947549|0.711404|0.999994|0.855699|0.999950|0.000050|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.947549|0.711404|0.999994|0.855699|0.999950|0.000050|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -104,23 +110,11 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986324|0.984237|0.982159|0.968963|0.984243|0.984241|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.907324|0.903167|0.899047|0.823432|0.903193|0.903179|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.930930|0.905656|0.881718|0.827579|0.906584|0.906571|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.871106|0.803515|0.745659|0.671564|0.810320|0.810296|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.871106|0.803515|0.745659|0.671564|0.810320|0.810296|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.984234|0.996967|0.996469|0.996718|0.996718|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.903153|0.990022|0.977054|0.983495|0.983495|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.905642|0.994546|0.970520|0.982386|0.982386|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.803490|0.991767|0.931069|0.960460|0.960460|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.803490|0.991767|0.931069|0.960460|0.960460|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996391|0.991025|0.985717|0.982210|0.991065|0.991064|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.922689|0.912805|0.903131|0.839597|0.912951|0.912938|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.948942|0.916426|0.886065|0.845744|0.917924|0.917913|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.888559|0.812669|0.748723|0.684450|0.821030|0.821007|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.888559|0.812669|0.748723|0.684450|0.821030|0.821007|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -128,75 +122,47 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1099420|13669|21547|7482464334|13669 / 7482478003|21547 / 1120967|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|1004747|99228|116220|7482378775|99228 / 7482478003|116220 / 1120967|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|971262|52652|149705|7482425351|52652 / 7482478003|149705 / 1120967|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|797573|66669|323394|7482411334|66669 / 7482478003|323394 / 1120967|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|797573|66669|323394|7482411334|66669 / 7482478003|323394 / 1120967|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1097200|0|19873|7303238338|0 / 7303238338|19873 / 1117073|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|1001785|76101|115288|7303162237|76101 / 7303238338|115288 / 1117073|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|968411|27967|148662|7303210371|27967 / 7303238338|148662 / 1117073|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|794690|43990|322383|7303194348|43990 / 7303238338|322383 / 1117073|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|794690|43990|322383|7303194348|43990 / 7303238338|322383 / 1117073|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 7482478003 (0.000000%)|0 / 1120967 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.987570|85532 / 7482478003 (0.001143%)|27855 / 1120967 (2.484908%)|0.936598|0.950693|0.950985|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.963982|42213 / 7482478003 (0.000564%)|80743 / 1120967 (7.202977%)|0.954209|0.944197|0.944333|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|0.000356%|7.227639%|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|0.001000%|2.493123%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.927432|0.975151|0.999989|0.987570|0.999985|0.000015|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.961002|0.927970|0.999994|0.963982|0.999984|0.000016|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.936598|0.950693|0.965218|0.906020|0.950992|0.950985|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.954209|0.944197|0.934394|0.894293|0.944342|0.944333|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1120967|0|0|7482478003|0 / 7482478003|0 / 1120967|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|1093112|85532|27855|7482392471|85532 / 7482478003|27855 / 1120967|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|1040224|42213|80743|7482435790|42213 / 7482478003|80743 / 1120967|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|25967 / 7303238338|80738 / 1117073|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|73019 / 7303238338|27850 / 1117073|
|
||||
|Radixor|0 / 7303238338|0 / 1117073|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999997|38073 / 7482478003 (0.000509%)|0 / 1120967 (0.000000%)|0.973547|0.983301|0.983436|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.987566|143096 / 7482478003 (0.001912%)|27855 / 1120967 (2.484908%)|0.901045|0.927476|0.928576|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.963980|82745 / 7482478003 (0.001106%)|80743 / 1120967 (7.202977%)|0.926646|0.927142|0.927132|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|0.987528|0.001376%|2.493123%|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|0.963859|0.000609%|7.227639%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -204,9 +170,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.967151|1.000000|0.999995|0.999997|0.999995|0.000005|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.884246|0.975151|0.999981|0.987566|0.999977|0.000023|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.926316|0.927970|0.999989|0.963980|0.999978|0.000022|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.915516|0.975069|0.999986|0.987528|0.999982|0.000018|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.958830|0.927724|0.999994|0.963859|0.999983|0.000017|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -214,19 +180,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.973547|0.983301|0.993253|0.967151|0.983438|0.983436|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.901045|0.927476|0.955505|0.864761|0.928587|0.928576|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.926646|0.927142|0.927639|0.864180|0.927143|0.927132|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.926837|0.944354|0.962546|0.894575|0.944823|0.944815|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.952443|0.943020|0.933782|0.892184|0.943149|0.943140|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -234,9 +190,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1120967|38073|0|7482439930|38073 / 7482478003|0 / 1120967|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|1093112|143096|27855|7482334907|143096 / 7482478003|27855 / 1120967|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|1040224|82745|80743|7482395258|82745 / 7482478003|80743 / 1120967|
|
||||
|1|Radixor|ALL_CANDIDATES|1117073|0|0|7303238338|0 / 7303238338|0 / 1117073|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|1089223|100514|27850|7303137824|100514 / 7303238338|27850 / 1117073|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|1036335|44498|80738|7303193840|44498 / 7303238338|80738 / 1117073|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -246,25 +202,25 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|68962|10439|30093|11447|9.356634%|6|135231|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|88365|13696|43868|12873|10.522229%|5|136636|
|
||||
|Radixor|21547|13669|24404|2866|2.342632%|4|125778|
|
||||
|HUNSPELL POLISH LUCENE FILTER|67924|2000|16531|10485|8.674824%|6|132492|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|87438|3082|24413|11776|9.742941%|5|133810|
|
||||
|Radixor|19873|0|0|1392|1.151679%|4|122430|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990579|13669 / 7310252699 (0.000187%)|21000 / 1114651 (1.883998%)|0.986350|0.984397|0.984400|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.948177|99224 / 7310252699 (0.001357%)|115513 / 1114651 (10.363154%)|0.906972|0.902966|0.902976|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.933309|51950 / 7310252699 (0.000711%)|148667 / 1114651 (13.337538%)|0.931269|0.905928|0.906847|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.856382|66274 / 7310252699 (0.000907%)|320158 / 1114651 (28.722712%)|0.871591|0.804380|0.811082|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.856382|66274 / 7310252699 (0.000907%)|320158 / 1114651 (28.722712%)|0.871591|0.804380|0.811082|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.991301|0.000000%|1.739895%|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|0.948417|0.001067%|10.315578%|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|0.933546|0.000382%|13.290396%|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|0.856335|0.000611%|28.732387%|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|0.856335|0.000611%|28.732387%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -272,11 +228,11 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987656|0.981160|0.999998|0.990579|0.999995|0.000005|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.909662|0.896368|0.999986|0.948177|0.999971|0.000029|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.948965|0.866625|0.999993|0.933309|0.999973|0.000027|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.923006|0.712773|0.999991|0.856382|0.999947|0.000053|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.923006|0.712773|0.999991|0.856382|0.999947|0.000053|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.982601|1.000000|0.991301|0.999997|0.000003|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.929032|0.896844|0.999989|0.948417|0.999973|0.000027|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.972469|0.867096|0.999996|0.933546|0.999975|0.000025|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.947796|0.712676|0.999994|0.856335|0.999949|0.000051|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.947796|0.712676|0.999994|0.856335|0.999949|0.000051|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -284,23 +240,11 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986350|0.984397|0.982452|0.969274|0.984403|0.984400|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.906972|0.902966|0.898996|0.823098|0.902991|0.902976|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.931269|0.905928|0.881929|0.828033|0.906861|0.906847|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.871591|0.804380|0.746792|0.672772|0.811106|0.811082|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.871591|0.804380|0.746792|0.672772|0.811106|0.811082|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.984395|0.996926|0.996647|0.996786|0.996786|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.902952|0.989889|0.977012|0.983408|0.983408|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.905914|0.994584|0.970514|0.982402|0.982402|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.804354|0.991711|0.931318|0.960566|0.960566|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.804354|0.991711|0.931318|0.960566|0.960566|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996471|0.991224|0.986032|0.982601|0.991262|0.991261|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.922411|0.912654|0.903102|0.839342|0.912796|0.912783|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|0.949394|0.916764|0.886303|0.846320|0.918272|0.918260|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|0.889130|0.813590|0.749881|0.685758|0.821871|0.821848|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|0.889130|0.813590|0.749881|0.685758|0.821871|0.821848|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -308,75 +252,47 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1093651|13669|21000|7310239030|13669 / 7310252699|21000 / 1114651|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|999138|99224|115513|7310153475|99224 / 7310252699|115513 / 1114651|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|965984|51950|148667|7310200749|51950 / 7310252699|148667 / 1114651|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|794493|66274|320158|7310186425|66274 / 7310252699|320158 / 1114651|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|794493|66274|320158|7310186425|66274 / 7310252699|320158 / 1114651|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1091431|0|19326|7133100218|0 / 7133100218|19326 / 1110757|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|996176|76097|114581|7133024121|76097 / 7133100218|114581 / 1110757|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|PRIMARY_OUTPUT|963133|27267|147624|7133072951|27267 / 7133100218|147624 / 1110757|
|
||||
|4|POLISH LUCENE STEMPEL DIRECT|PRIMARY_OUTPUT|791610|43601|319147|7133056617|43601 / 7133100218|319147 / 1110757|
|
||||
|5|POLISH LUCENE STEMPEL FILTER|PRIMARY_OUTPUT|791610|43601|319147|7133056617|43601 / 7133100218|319147 / 1110757|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 7310252699 (0.000000%)|0 / 1114651 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.987661|85532 / 7310252699 (0.001170%)|27494 / 1114651 (2.466602%)|0.936331|0.950586|0.950885|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.963946|41671 / 7310252699 (0.000570%)|80368 / 1114651 (7.210149%)|0.954406|0.944290|0.944429|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|0.000356%|7.234976%|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|0.001024%|2.474799%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.927063|0.975334|0.999988|0.987661|0.999985|0.000015|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.961271|0.927899|0.999994|0.963946|0.999983|0.000017|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.936331|0.950586|0.965282|0.905826|0.950892|0.950885|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|0.954406|0.944290|0.934386|0.894459|0.944437|0.944429|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1114651|0|0|7310252699|0 / 7310252699|0 / 1114651|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|1087157|85532|27494|7310167167|85532 / 7310252699|27494 / 1114651|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ANY_CANDIDATE|1034283|41671|80368|7310211028|41671 / 7310252699|80368 / 1114651|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|25425 / 7133100218|80363 / 1110757|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|73019 / 7133100218|27489 / 1110757|
|
||||
|Radixor|0 / 7133100218|0 / 1110757|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999997|38073 / 7310252699 (0.000521%)|0 / 1114651 (0.000000%)|0.973401|0.983208|0.983344|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.987657|143085 / 7310252699 (0.001957%)|27494 / 1114651 (2.466602%)|0.900618|0.927255|0.928372|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.963944|81865 / 7310252699 (0.001120%)|80368 / 1114651 (7.210149%)|0.926903|0.927276|0.927265|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|0.987619|0.001409%|2.474799%|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|0.963822|0.000612%|7.234976%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -384,9 +300,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.966971|1.000000|0.999995|0.999997|0.999995|0.000005|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.883694|0.975334|0.999980|0.987657|0.999977|0.000023|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.926654|0.927899|0.999989|0.963944|0.999978|0.000022|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.915099|0.975252|0.999986|0.987619|0.999982|0.000018|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.959377|0.927650|0.999994|0.963822|0.999983|0.000017|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -394,19 +310,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.973401|0.983208|0.993215|0.966971|0.983347|0.983344|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.900618|0.927255|0.955516|0.864376|0.928384|0.928372|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.926903|0.927276|0.927649|0.864412|0.927276|0.927265|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.926529|0.944219|0.962597|0.894332|0.944697|0.944688|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|0.952859|0.943247|0.933827|0.892590|0.943380|0.943372|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -414,9 +320,9 @@ This mode contains **11 result rows**, **5 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|1114651|38073|0|7310214626|38073 / 7310252699|0 / 1114651|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|1087157|143085|27494|7310109614|143085 / 7310252699|27494 / 1114651|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|1034283|81865|80368|7310170834|81865 / 7310252699|80368 / 1114651|
|
||||
|1|Radixor|ALL_CANDIDATES|1110757|0|0|7133100218|0 / 7133100218|0 / 1110757|
|
||||
|2|POLISH LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|1083268|100503|27489|7132999715|100503 / 7133100218|27489 / 1110757|
|
||||
|3|HUNSPELL POLISH LUCENE FILTER|ALL_CANDIDATES|1030394|43630|80363|7133056588|43630 / 7133100218|80363 / 1110757|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -426,21 +332,21 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL POLISH LUCENE FILTER|68299|10279|29915|11265|9.315692%|6|133595|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|88019|13692|43861|12763|10.554476%|5|135105|
|
||||
|Radixor|21000|13669|24404|2780|2.298946%|4|124274|
|
||||
|HUNSPELL POLISH LUCENE FILTER|67261|1842|16363|10303|8.625294%|6|130856|
|
||||
|POLISH LUCENE MORFOLOGIK FILTER|87092|3078|24406|11666|9.766348%|5|132279|
|
||||
|Radixor|19326|0|0|1306|1.093335%|4|120926|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -449,16 +355,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `PL_PL`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `PT_PT` | 4,001 | 215,490 | 8,002 | 207,488 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `pt-pt-default` | `1.0.0` | `PT_PT` | 4,001 | 215,490 | 8,002 | 207,488 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **215,490**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **215,490**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 3,806 | 1.766% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 120,691 | 56.008% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 120,535 | 55.935% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 71,284 | 33.080% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 8,003 | 3.714% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 11,706 | 5.432% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 11,862 | 5.505% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -37,18 +37,24 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 0.625% | 0.558% | 2.374% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene PortugueseStemFilter | 0.312% | 0.308% | 0.425% | Portuguese RSLP-style Lucene TokenFilter stemmer. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `portugueseRadixor` | 12.109 | 0.698 | 58.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 11.172 | 1.870 | 53.8 | 0.923 | Light Portuguese suffix stemmer. |
|
||||
| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 16.038 | 1.752 | 77.3 | 1.325 | Minimal Portuguese suffix reducer. |
|
||||
| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 53.725 | 5.356 | 258.9 | 4.437 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 57.457 | 1.182 | 276.9 | 4.745 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 165.447 | 40.334 | 797.4 | 13.663 | Portuguese RSLP-style Lucene TokenFilter. |
|
||||
| Radixor | `portugueseRadixor` | 12.301 | 0.252 | 59.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 11.409 | 0.151 | 55.0 | 0.927 | Light Portuguese suffix stemmer. |
|
||||
| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 15.619 | 0.084 | 75.3 | 1.270 | Minimal Portuguese suffix reducer. |
|
||||
| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 57.577 | 1.591 | 277.5 | 4.681 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 63.403 | 2.720 | 305.6 | 5.154 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 158.014 | 5.150 | 761.6 | 12.845 | Portuguese RSLP-style Lucene TokenFilter. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -62,32 +68,32 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PT_PT` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PT_PT` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/pt_pt/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `pt-pt-default`, loaded from classpath resource `org/egothor/stemmer/models/pt-pt-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.998502** among 6 deterministic stemmers. The runner-up is `SNOWBALL PORTUGUESE DIRECT` at 0.938800, a difference of 0.059702. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.998502** among 6 deterministic stemmers. The runner-up is `SNOWBALL PORTUGUESE DIRECT` at 0.938800, a difference of 0.059702. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.998542** among 6 deterministic stemmers. The runner-up is `SNOWBALL PORTUGUESE DIRECT` at 0.938922, a difference of 0.059620. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.998542** among 6 deterministic stemmers. The runner-up is `SNOWBALL PORTUGUESE DIRECT` at 0.938922, a difference of 0.059620. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998502|20678 / 22358203756 (0.000092%)|16444 / 5489060 (0.299578%)|0.996389|0.996620|0.996619|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.938800|167230 / 22358203756 (0.000748%)|671821 / 5489060 (12.239272%)|0.947271|0.919888|0.920940|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.938800|167230 / 22358203756 (0.000748%)|671821 / 5489060 (12.239272%)|0.947271|0.919888|0.920940|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.846459|99075 / 22358203756 (0.000443%)|1685572 / 5489060 (30.707844%)|0.901330|0.809975|0.821750|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.513648|2511 / 22358203756 (0.000011%)|5339230 / 5489060 (97.270389%)|0.122843|0.053118|0.163828|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.503954|598 / 22358203756 (0.000003%)|5445654 / 5489060 (99.209227%)|0.038310|0.015690|0.088308|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.998542|0.000000%|0.291615%|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|0.938922|0.000656%|12.214929%|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|0.938922|0.000656%|12.214929%|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|0.846554|0.000364%|30.688771%|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|0.513632|0.000006%|97.273598%|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|0.503949|<0.000001%|99.210240%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -95,12 +101,12 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996236|0.997004|0.999999|0.998502|0.999998|0.000002|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.966450|0.877607|0.999993|0.938800|0.999962|0.000038|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.966450|0.877607|0.999993|0.938800|0.999962|0.000038|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.974613|0.692922|0.999996|0.846459|0.999920|0.000080|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.983517|0.027296|1.000000|0.513648|0.999761|0.000239|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.986410|0.007908|1.000000|0.503954|0.999756|0.000244|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.997084|1.000000|0.998542|0.999999|0.000001|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.970538|0.877851|0.999993|0.938922|0.999963|0.000037|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.970538|0.877851|0.999993|0.938922|0.999963|0.000037|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.979145|0.693112|0.999996|0.846554|0.999921|0.000079|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.991719|0.027264|1.000000|0.513632|0.999760|0.000240|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.999608|0.007898|1.000000|0.503949|0.999756|0.000244|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -108,25 +114,12 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996389|0.996620|0.996850|0.993262|0.996620|0.996619|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.947271|0.919888|0.894045|0.851661|0.920958|0.920940|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.947271|0.919888|0.894045|0.851661|0.920958|0.920940|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.901330|0.809975|0.735434|0.680636|0.821785|0.821750|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.122843|0.053118|0.033885|0.027284|0.163848|0.163828|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.038310|0.015690|0.009865|0.007907|0.088319|0.088308|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996619|0.999299|0.999347|0.999323|0.999323|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.919870|0.996663|0.967924|0.982083|0.982083|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.919870|0.996663|0.967924|0.982083|0.982083|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.809936|0.996729|0.918475|0.956003|0.956003|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.053105|0.999226|0.720580|0.837330|0.837330|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.015686|0.999664|0.692383|0.818122|0.818122|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999415|0.998540|0.997666|0.997084|0.998541|0.998541|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.950467|0.921871|0.894944|0.855065|0.923032|0.923014|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.950467|0.921871|0.894944|0.855065|0.923032|0.923014|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.904492|0.811666|0.736120|0.683029|0.823807|0.823773|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.122815|0.053069|0.033847|0.027258|0.164433|0.164413|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.038278|0.015671|0.009853|0.007898|0.088851|0.088840|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -134,64 +127,42 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|5472616|20678|16444|22358183078|20678 / 22358203756|16444 / 5489060|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|4817239|167230|671821|22358036526|167230 / 22358203756|671821 / 5489060|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|4817239|167230|671821|22358036526|167230 / 22358203756|671821 / 5489060|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|3803488|99075|1685572|22358104681|99075 / 22358203756|1685572 / 5489060|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|149830|2511|5339230|22358201245|2511 / 22358203756|5339230 / 5489060|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|43406|598|5445654|22358203158|598 / 22358203756|5445654 / 5489060|
|
||||
|1|Radixor|PRIMARY_OUTPUT|5470353|0|15999|22274113243|0 / 22274113243|15999 / 5486352|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|4816198|146201|670154|22273967042|146201 / 22274113243|670154 / 5486352|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|4816198|146201|670154|22273967042|146201 / 22274113243|670154 / 5486352|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|3802658|80995|1683694|22274032248|80995 / 22274113243|1683694 / 5486352|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|149580|1249|5336772|22274111994|1249 / 22274113243|5336772 / 5486352|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|43329|17|5443023|22274113226|17 / 22274113243|5443023 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 22358203756 (0.000000%)|0 / 5489060 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|5489060|0|0|22358203756|0 / 22358203756|0 / 5489060|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 22274113243|0 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|38310 / 22358203756 (0.000171%)|0 / 5489060 (0.000000%)|0.994448|0.996522|0.996528|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -199,7 +170,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.993069|1.000000|0.999998|0.999999|0.999998|0.000002|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -207,15 +178,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.994448|0.996522|0.998606|0.993069|0.996528|0.996528|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -223,7 +186,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|5489060|38310|0|22358165446|38310 / 22358203756|0 / 5489060|
|
||||
|1|Radixor|ALL_CANDIDATES|5486352|0|0|22274113243|0 / 22274113243|0 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -233,24 +196,24 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|16444|20678|17632|790|0.373542%|3|212297|
|
||||
|Radixor|15999|0|0|392|0.185702%|3|211489|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998502|20678 / 22358203756 (0.000092%)|16444 / 5489060 (0.299578%)|0.996389|0.996620|0.996619|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.938800|167230 / 22358203756 (0.000748%)|671821 / 5489060 (12.239272%)|0.947271|0.919888|0.920940|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.938800|167230 / 22358203756 (0.000748%)|671821 / 5489060 (12.239272%)|0.947271|0.919888|0.920940|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.846459|99075 / 22358203756 (0.000443%)|1685572 / 5489060 (30.707844%)|0.901330|0.809975|0.821750|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.513648|2511 / 22358203756 (0.000011%)|5339230 / 5489060 (97.270389%)|0.122843|0.053118|0.163828|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.503954|598 / 22358203756 (0.000003%)|5445654 / 5489060 (99.209227%)|0.038310|0.015690|0.088308|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.998542|0.000000%|0.291615%|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|0.938922|0.000656%|12.214929%|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|0.938922|0.000656%|12.214929%|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|0.846554|0.000364%|30.688771%|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|0.513632|0.000006%|97.273598%|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|0.503949|<0.000001%|99.210240%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -258,12 +221,12 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996236|0.997004|0.999999|0.998502|0.999998|0.000002|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.966450|0.877607|0.999993|0.938800|0.999962|0.000038|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.966450|0.877607|0.999993|0.938800|0.999962|0.000038|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.974613|0.692922|0.999996|0.846459|0.999920|0.000080|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.983517|0.027296|1.000000|0.513648|0.999761|0.000239|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.986410|0.007908|1.000000|0.503954|0.999756|0.000244|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.997084|1.000000|0.998542|0.999999|0.000001|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.970538|0.877851|0.999993|0.938922|0.999963|0.000037|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.970538|0.877851|0.999993|0.938922|0.999963|0.000037|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.979145|0.693112|0.999996|0.846554|0.999921|0.000079|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.991719|0.027264|1.000000|0.513632|0.999760|0.000240|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.999608|0.007898|1.000000|0.503949|0.999756|0.000244|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -271,25 +234,12 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996389|0.996620|0.996850|0.993262|0.996620|0.996619|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.947271|0.919888|0.894045|0.851661|0.920958|0.920940|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.947271|0.919888|0.894045|0.851661|0.920958|0.920940|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.901330|0.809975|0.735434|0.680636|0.821785|0.821750|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.122843|0.053118|0.033885|0.027284|0.163848|0.163828|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.038310|0.015690|0.009865|0.007907|0.088319|0.088308|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996619|0.999299|0.999347|0.999323|0.999323|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.919870|0.996663|0.967924|0.982083|0.982083|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.919870|0.996663|0.967924|0.982083|0.982083|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.809936|0.996729|0.918475|0.956003|0.956003|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.053105|0.999226|0.720580|0.837330|0.837330|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.015686|0.999664|0.692383|0.818122|0.818122|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999415|0.998540|0.997666|0.997084|0.998541|0.998541|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|0.950467|0.921871|0.894944|0.855065|0.923032|0.923014|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|0.950467|0.921871|0.894944|0.855065|0.923032|0.923014|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|0.904492|0.811666|0.736120|0.683029|0.823807|0.823773|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|0.122815|0.053069|0.033847|0.027258|0.164433|0.164413|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.038278|0.015671|0.009853|0.007898|0.088851|0.088840|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -297,64 +247,42 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|5472616|20678|16444|22358183078|20678 / 22358203756|16444 / 5489060|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|4817239|167230|671821|22358036526|167230 / 22358203756|671821 / 5489060|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|4817239|167230|671821|22358036526|167230 / 22358203756|671821 / 5489060|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|3803488|99075|1685572|22358104681|99075 / 22358203756|1685572 / 5489060|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|149830|2511|5339230|22358201245|2511 / 22358203756|5339230 / 5489060|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|43406|598|5445654|22358203158|598 / 22358203756|5445654 / 5489060|
|
||||
|1|Radixor|PRIMARY_OUTPUT|5470353|0|15999|22274113243|0 / 22274113243|15999 / 5486352|
|
||||
|2|SNOWBALL PORTUGUESE DIRECT|PRIMARY_OUTPUT|4816198|146201|670154|22273967042|146201 / 22274113243|670154 / 5486352|
|
||||
|3|SNOWBALL PORTUGUESE LUCENE FILTER|PRIMARY_OUTPUT|4816198|146201|670154|22273967042|146201 / 22274113243|670154 / 5486352|
|
||||
|4|PORTUGUESE LUCENE PORTUGUESE STEM FILTER|PRIMARY_OUTPUT|3802658|80995|1683694|22274032248|80995 / 22274113243|1683694 / 5486352|
|
||||
|5|PORTUGUESE LUCENE PORTUGUESE LIGHT STEM FILTER|PRIMARY_OUTPUT|149580|1249|5336772|22274111994|1249 / 22274113243|5336772 / 5486352|
|
||||
|6|PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|PRIMARY_OUTPUT|43329|17|5443023|22274113226|17 / 22274113243|5443023 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 22358203756 (0.000000%)|0 / 5489060 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|5489060|0|0|22358203756|0 / 22358203756|0 / 5489060|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 22274113243|0 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|38310 / 22358203756 (0.000171%)|0 / 5489060 (0.000000%)|0.994448|0.996522|0.996528|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -362,7 +290,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.993069|1.000000|0.999998|0.999999|0.999998|0.000002|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -370,15 +298,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.994448|0.996522|0.998606|0.993069|0.996528|0.996528|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -386,7 +306,7 @@ This mode contains **8 result rows**, **6 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|5489060|38310|0|22358165446|38310 / 22358203756|0 / 5489060|
|
||||
|1|Radixor|ALL_CANDIDATES|5486352|0|0|22274113243|0 / 22274113243|0 / 5486352|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -396,19 +316,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|16444|20678|17632|790|0.373542%|3|212297|
|
||||
|Radixor|15999|0|0|392|0.185702%|3|211489|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -417,16 +337,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `PT_PT`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `RU_RU` | 37,410 | 806,279 | 74,808 | 731,471 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `ru-ru-default` | `1.0.0` | `RU_RU` | 37,410 | 806,279 | 74,808 | 731,471 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **806,279**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **806,279**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 9,260 | 1.148% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 584,785 | 72.529% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 82,864 | 10.277% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 75,646 | 9.382% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 53,724 | 6.663% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 9,287 | 1.152% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 580,915 | 72.049% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 82,956 | 10.289% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 75,527 | 9.367% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 57,594 | 7.143% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -35,16 +35,22 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 9.162% | 8.162% | 18.936% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Official Snowball direct | 9.162% | 8.162% | 18.936% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `russianRadixor` | 89.671 | 3.886 | 122.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 60.522 | 5.310 | 82.7 | 0.675 | Light Russian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[RUSSIAN]` | 106.031 | 9.287 | 145.0 | 1.182 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 137.512 | 10.801 | 188.0 | 1.534 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `russianRadixor` | 90.151 | 1.796 | 123.2 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 59.456 | 2.102 | 81.3 | 0.660 | Light Russian suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[RUSSIAN]` | 102.353 | 1.669 | 139.9 | 1.135 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 138.597 | 4.727 | 189.5 | 1.537 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -58,30 +64,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `RU_RU` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `RU_RU` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/ru_ru/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `ru-ru-default`, loaded from classpath resource `org/egothor/stemmer/models/ru-ru-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.989827** among 4 deterministic stemmers. The runner-up is `SNOWBALL RUSSIAN LUCENE FILTER` at 0.834876, a difference of 0.154951. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989852** among 4 deterministic stemmers. The runner-up is `SNOWBALL RUSSIAN DIRECT` at 0.834854, a difference of 0.154998. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.990188** among 4 deterministic stemmers. The runner-up is `SNOWBALL RUSSIAN LUCENE FILTER` at 0.834565, a difference of 0.155624. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.990213** among 4 deterministic stemmers. The runner-up is `SNOWBALL RUSSIAN DIRECT` at 0.834542, a difference of 0.155670. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989827|155850 / 295576291016 (0.000053%)|266302 / 13089505 (2.034470%)|0.986313|0.983806|0.983814|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.834876|3785790 / 295576291016 (0.001281%)|4322616 / 13089505 (33.023525%)|0.692485|0.683786|0.683923|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.834867|3782908 / 295576291016 (0.001280%)|4322849 / 13089505 (33.025305%)|0.692603|0.683851|0.683989|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.617692|321183 / 295576291016 (0.000109%)|10008438 / 13089505 (76.461547%)|0.577011|0.373649|0.461687|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.990188|0.000000%|1.962362%|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|0.834565|0.001215%|33.085867%|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|0.834556|0.001214%|33.087654%|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|0.616440|0.000059%|76.711890%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -89,10 +95,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987992|0.979655|0.999999|0.989827|0.999999|0.000001|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.698408|0.669765|0.999987|0.834876|0.999973|0.000027|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.698563|0.669747|0.999987|0.834867|0.999973|0.000027|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.905597|0.235385|0.999999|0.617692|0.999965|0.000035|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.980376|1.000000|0.990188|0.999999|0.000001|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.713518|0.669141|0.999988|0.834565|0.999973|0.000027|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.713680|0.669123|0.999988|0.834556|0.999973|0.000027|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.946958|0.232881|0.999999|0.616440|0.999965|0.000035|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -100,21 +106,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986313|0.983806|0.981311|0.968128|0.983815|0.983814|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.692485|0.683786|0.675304|0.519510|0.683936|0.683923|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.692603|0.683851|0.675318|0.519585|0.684003|0.683989|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.577011|0.373649|0.276278|0.229747|0.461696|0.461687|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.983805|0.997699|0.997274|0.997487|0.997487|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.683773|0.974131|0.953674|0.963794|0.963794|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.683838|0.974180|0.953661|0.963811|0.963811|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.373638|0.994311|0.870888|0.928516|0.928516|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996013|0.990091|0.984239|0.980376|0.990140|0.990139|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.704178|0.690618|0.677570|0.527438|0.690974|0.690960|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.704300|0.690684|0.677584|0.527515|0.691043|0.691029|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.586986|0.373828|0.274241|0.229882|0.469605|0.469596|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -122,62 +117,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|12823203|155850|266302|295576135166|155850 / 295576291016|266302 / 13089505|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|8766889|3785790|4322616|295572505226|3785790 / 295576291016|4322616 / 13089505|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|8766656|3782908|4322849|295572508108|3782908 / 295576291016|4322849 / 13089505|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|3081067|321183|10008438|295575969833|321183 / 295576291016|10008438 / 13089505|
|
||||
|1|Radixor|PRIMARY_OUTPUT|12781761|0|255845|288279885172|0 / 288279885172|255845 / 13037606|
|
||||
|2|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|8724001|3502741|4313605|288276382431|3502741 / 288279885172|4313605 / 13037606|
|
||||
|3|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|8723768|3499880|4313838|288276385292|3499880 / 288279885172|4313838 / 13037606|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|3036212|170067|10001394|288279715105|170067 / 288279885172|10001394 / 13037606|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 295576291016 (0.000000%)|13 / 13089505 (0.000099%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000100%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0.999999|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|0.999999|0.999999|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|13089492|0|13|295576291016|0 / 295576291016|13 / 13089505|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 288279885172|13 / 13037606|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|434710 / 295576291016 (0.000147%)|13 / 13089505 (0.000099%)|0.974119|0.983665|0.983796|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000100%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -185,7 +158,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.967857|0.999999|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|0.999999|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -193,15 +166,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.974119|0.983665|0.993401|0.967856|0.983797|0.983796|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|0.999999|0.999999|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -209,7 +174,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|13089492|434710|13|295575856306|434710 / 295576291016|13 / 13089505|
|
||||
|1|Radixor|ALL_CANDIDATES|13037593|0|13|288279885172|0 / 288279885172|13 / 13037606|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -219,22 +184,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|266289|155850|278860|19162|2.492190%|4|788492|
|
||||
|Radixor|255832|0|0|9613|1.265979%|4|769106|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989852|155850 / 295000681652 (0.000053%)|265613 / 13087126 (2.029575%)|0.986322|0.983830|0.983838|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.834854|3782908 / 295000681652 (0.001282%)|4322407 / 13087126 (33.027931%)|0.692561|0.683815|0.683953|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.834854|3782908 / 295000681652 (0.001282%)|4322407 / 13087126 (33.027931%)|0.692561|0.683815|0.683953|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.617630|318921 / 295000681652 (0.000108%)|10008238 / 13087126 (76.473918%)|0.577038|0.373540|0.461703|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.990213|0.000000%|1.957434%|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|0.834542|0.001216%|33.090302%|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|0.834542|0.001216%|33.090302%|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|0.616378|0.000058%|76.724356%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -242,10 +207,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987991|0.979704|0.999999|0.989852|0.999999|0.000001|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.698516|0.669721|0.999987|0.834854|0.999973|0.000027|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.698516|0.669721|0.999987|0.834854|0.999973|0.000027|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.906139|0.235261|0.999999|0.617630|0.999965|0.000035|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.980426|1.000000|0.990213|0.999999|0.000001|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.713634|0.669097|0.999988|0.834542|0.999973|0.000027|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.713634|0.669097|0.999988|0.834542|0.999973|0.000027|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.947585|0.232756|0.999999|0.616378|0.999965|0.000035|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -253,21 +218,10 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986322|0.983830|0.981350|0.968175|0.983839|0.983838|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.692561|0.683815|0.675288|0.519544|0.683967|0.683953|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.692561|0.683815|0.675288|0.519544|0.683967|0.683953|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.577038|0.373540|0.276152|0.229664|0.461713|0.461703|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.983829|0.997697|0.997321|0.997509|0.997509|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.683802|0.974149|0.953634|0.963782|0.963782|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.683802|0.974149|0.953634|0.963782|0.963782|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.373528|0.994350|0.870767|0.928464|0.928464|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.996023|0.990116|0.984279|0.980426|0.990164|0.990164|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|0.704259|0.690648|0.677554|0.527474|0.691007|0.690993|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|0.704259|0.690648|0.677554|0.527474|0.691007|0.690993|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|0.587020|0.373716|0.274113|0.229798|0.469634|0.469625|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -275,62 +229,40 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|12821513|155850|265613|295000525802|155850 / 295000681652|265613 / 13087126|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|8764719|3782908|4322407|294996898744|3782908 / 295000681652|4322407 / 13087126|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|8764719|3782908|4322407|294996898744|3782908 / 295000681652|4322407 / 13087126|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|3078888|318921|10008238|295000362731|318921 / 295000681652|10008238 / 13087126|
|
||||
|1|Radixor|PRIMARY_OUTPUT|12780071|0|255156|287711428009|0 / 287711428009|255156 / 13035227|
|
||||
|2|SNOWBALL RUSSIAN DIRECT|PRIMARY_OUTPUT|8721831|3499880|4313396|287707928129|3499880 / 287711428009|4313396 / 13035227|
|
||||
|3|SNOWBALL RUSSIAN LUCENE FILTER|PRIMARY_OUTPUT|8721831|3499880|4313396|287707928129|3499880 / 287711428009|4313396 / 13035227|
|
||||
|4|RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|PRIMARY_OUTPUT|3034033|167825|10001194|287711260184|167825 / 287711428009|10001194 / 13035227|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 295000681652 (0.000000%)|0 / 13087126 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|13087126|0|0|295000681652|0 / 295000681652|0 / 13087126|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 287711428009|0 / 13035227|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999999|434710 / 295000681652 (0.000147%)|0 / 13087126 (0.000000%)|0.974115|0.983663|0.983794|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -338,7 +270,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.967851|1.000000|0.999999|0.999999|0.999999|0.000001|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -346,15 +278,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.974115|0.983663|0.993401|0.967851|0.983794|0.983794|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -362,7 +286,7 @@ This mode contains **6 result rows**, **4 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|13087126|434710|0|295000246942|434710 / 295000681652|0 / 13087126|
|
||||
|1|Radixor|ALL_CANDIDATES|13035227|0|0|287711428009|0 / 287711428009|0 / 13035227|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -372,19 +296,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|265613|155850|278860|18991|2.472358%|4|787549|
|
||||
|Radixor|255156|0|0|9442|1.244687%|4|768163|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -393,16 +317,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `RU_RU`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `ES_ES` | 65,059 | 926,393 | 120,121 | 806,272 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `es-es-default` | `1.0.0` | `ES_ES` | 65,059 | 926,393 | 120,121 | 806,272 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **926,393**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **926,393**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 5,367 | 0.579% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 524,682 | 56.637% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 240,872 | 26.001% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 130,089 | 14.043% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 25,383 | 2.740% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 8,534 | 0.921% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 522,685 | 56.422% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 243,410 | 26.275% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 124,386 | 13.427% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 27,378 | 2.955% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -38,19 +38,25 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 4.889% | 4.287% | 8.932% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Official Snowball direct | 4.889% | 4.287% | 8.930% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `spanishRadixor` | 78.919 | 7.253 | 97.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 2079.041 | 193.548 | 2578.6 | 26.344 | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 45.596 | 4.639 | 56.6 | 0.578 | Minimal Spanish suffix reducer; narrow baseline. |
|
||||
| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 42.003 | 1.683 | 52.1 | 0.532 | Light Spanish suffix stemmer. |
|
||||
| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 93.734 | 6.247 | 116.3 | 1.188 | Plural-oriented Spanish suffix reducer. |
|
||||
| Official Snowball direct | `snowballDirect[SPANISH]` | 171.995 | 11.035 | 213.3 | 2.179 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 211.138 | 17.940 | 261.9 | 2.675 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `spanishRadixor` | 81.605 | 1.347 | 101.2 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 2033.430 | 14.863 | 2522.0 | 24.918 | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 42.144 | 1.556 | 52.3 | 0.516 | Minimal Spanish suffix reducer; narrow baseline. |
|
||||
| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 44.479 | 1.291 | 55.2 | 0.545 | Light Spanish suffix stemmer. |
|
||||
| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 96.537 | 3.418 | 119.7 | 1.183 | Plural-oriented Spanish suffix reducer. |
|
||||
| Official Snowball direct | `snowballDirect[SPANISH]` | 172.151 | 7.261 | 213.5 | 2.110 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 201.697 | 9.363 | 250.2 | 2.472 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -64,33 +70,33 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `ES_ES` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `ES_ES` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/es_es/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `es-es-default`, loaded from classpath resource `org/egothor/stemmer/models/es-es-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.989295** among 7 deterministic stemmers. The runner-up is `SNOWBALL SPANISH LUCENE FILTER` at 0.652614, a difference of 0.336680. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989429** among 7 deterministic stemmers. The runner-up is `SNOWBALL SPANISH DIRECT` at 0.652720, a difference of 0.336709. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.989448** among 7 deterministic stemmers. The runner-up is `SNOWBALL SPANISH LUCENE FILTER` at 0.652438, a difference of 0.337010. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989580** among 7 deterministic stemmers. The runner-up is `SNOWBALL SPANISH DIRECT` at 0.652542, a difference of 0.337038. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989295|288483 / 379567318110 (0.000076%)|898652 / 41973336 (2.141007%)|0.990105|0.985755|0.985780|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.652614|2230481 / 379567318110 (0.000588%)|29161643 / 41973336 (69.476591%)|0.627151|0.449411|0.509848|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.652614|2228819 / 379567318110 (0.000587%)|29161649 / 41973336 (69.476605%)|0.627192|0.449424|0.509876|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.615102|536192 / 379567318110 (0.000141%)|32310860 / 41973336 (76.979490%)|0.583708|0.370408|0.466992|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.514823|147956 / 379567318110 (0.000039%)|40729019 / 41973336 (97.035458%)|0.130864|0.057387|0.162762|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.503874|58578 / 379567318110 (0.000015%)|41648091 / 41973336 (99.225115%)|0.037377|0.015357|0.081026|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.501768|47859 / 379567318110 (0.000013%)|41824873 / 41973336 (99.646292%)|0.017361|0.007041|0.051714|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.989448|0.000000%|2.110334%|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|0.652438|0.000414%|69.511918%|
|
||||
|3|SNOWBALL SPANISH DIRECT|0.652438|0.000413%|69.511932%|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|0.615028|0.000068%|76.994273%|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|0.514565|0.000009%|97.087060%|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|0.503764|0.000002%|99.247265%|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|0.501678|0.000001%|99.664470%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -98,13 +104,13 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.993026|0.978590|0.999999|0.989295|0.999997|0.000003|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.851718|0.305234|0.999994|0.652614|0.999917|0.000083|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.851812|0.305234|0.999994|0.652614|0.999917|0.000083|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.947425|0.230205|0.999999|0.615102|0.999913|0.000087|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.893731|0.029645|1.000000|0.514823|0.999892|0.000108|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.847383|0.007749|1.000000|0.503874|0.999890|0.000110|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.756222|0.003537|1.000000|0.501768|0.999890|0.000110|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.978897|1.000000|0.989448|0.999998|0.000002|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.895438|0.304881|0.999996|0.652438|0.999915|0.000085|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.895510|0.304881|0.999996|0.652438|0.999915|0.000085|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.975281|0.230057|0.999999|0.615028|0.999910|0.000090|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.974423|0.029129|1.000000|0.514565|0.999887|0.000113|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.979154|0.007527|1.000000|0.503764|0.999885|0.000115|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.970596|0.003355|1.000000|0.501678|0.999884|0.000116|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -112,27 +118,13 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990105|0.985755|0.981443|0.971910|0.985781|0.985780|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.627151|0.449411|0.350170|0.289832|0.509876|0.509848|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.627192|0.449424|0.350173|0.289843|0.509904|0.509876|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.583708|0.370408|0.271278|0.227301|0.467014|0.466992|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.130864|0.057387|0.036752|0.029541|0.162773|0.162762|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.037377|0.015357|0.009664|0.007738|0.081032|0.081026|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.017361|0.007041|0.004416|0.003533|0.051719|0.051714|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.985753|0.995418|0.993266|0.994341|0.994341|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.449379|0.981386|0.852461|0.912391|0.912391|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.449392|0.981406|0.852463|0.912401|0.912401|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.370381|0.993314|0.790558|0.880414|0.880414|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.057381|0.993824|0.756690|0.859195|0.859195|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.015355|0.995442|0.723731|0.838115|0.838115|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.007040|0.995635|0.710610|0.829316|0.829316|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995707|0.989336|0.983046|0.978897|0.989392|0.989391|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.645406|0.454882|0.351206|0.294400|0.522496|0.522469|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.645436|0.454891|0.351208|0.294407|0.522517|0.522490|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.591847|0.372295|0.271557|0.228724|0.473678|0.473655|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.130091|0.056568|0.036142|0.029107|0.168477|0.168467|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.036514|0.014940|0.009391|0.007526|0.085851|0.085846|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.016548|0.006687|0.004191|0.003355|0.057067|0.057063|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -140,71 +132,46 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|41074684|288483|898652|379567029627|288483 / 379567318110|898652 / 41973336|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|12811693|2230481|29161643|379565087629|2230481 / 379567318110|29161643 / 41973336|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|12811687|2228819|29161649|379565089291|2228819 / 379567318110|29161649 / 41973336|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|9662476|536192|32310860|379566781918|536192 / 379567318110|32310860 / 41973336|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|1244317|147956|40729019|379567170154|147956 / 379567318110|40729019 / 41973336|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|325245|58578|41648091|379567259532|58578 / 379567318110|41648091 / 41973336|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|148463|47859|41824873|379567270251|47859 / 379567318110|41824873 / 41973336|
|
||||
|1|Radixor|PRIMARY_OUTPUT|41053986|0|885054|360919543590|0 / 360919543590|885054 / 41939040|
|
||||
|2|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|12786409|1493087|29152631|360918050503|1493087 / 360919543590|29152631 / 41939040|
|
||||
|3|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|12786403|1491944|29152637|360918051646|1491944 / 360919543590|29152637 / 41939040|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|9648381|244539|32290659|360919299051|244539 / 360919543590|32290659 / 41939040|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|1221659|32066|40717381|360919511524|32066 / 360919543590|40717381 / 41939040|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|315690|6721|41623350|360919536869|6721 / 360919543590|41623350 / 41939040|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|140718|4263|41798322|360919539327|4263 / 360919543590|41798322 / 41939040|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999993|2 / 379567318110 (0.000000%)|626 / 41973336 (0.001491%)|0.999997|0.999993|0.999993|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.620065|416345 / 379567318110 (0.000110%)|31894218 / 41973336 (75.986855%)|0.600268|0.384195|0.480192|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|0.000062%|76.009935%|
|
||||
|Radixor|0.000000%|0.001493%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0.999985|1.000000|0.999993|1.000000|0.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.960331|0.240131|0.999999|0.620065|0.999915|0.000085|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|0.999997|0.999993|0.999988|0.999985|0.999993|0.999993|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.600268|0.384195|0.282504|0.237773|0.480214|0.480192|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|41972710|2|626|379567318108|2 / 379567318110|626 / 41973336|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|10079118|416345|31894218|379566901765|416345 / 379567318110|31894218 / 41973336|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|223500 / 360919543590|31877837 / 41939040|
|
||||
|Radixor|0 / 360919543590|626 / 41939040|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999991|1349800 / 379567318110 (0.000356%)|626 / 41973336 (0.001491%)|0.974915|0.984168|0.984289|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.620065|888077 / 379567318110 (0.000234%)|31894218 / 41973336 (75.986855%)|0.587073|0.380771|0.469749|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.999993|<0.000001%|0.001493%|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|0.619950|0.000073%|76.009935%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -212,8 +179,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.968843|0.999985|0.999996|0.999991|0.999996|0.000004|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.919024|0.240131|0.999998|0.620065|0.999914|0.000086|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999959|0.999985|1.000000|0.999993|1.000000|0.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.974467|0.239901|0.999999|0.619950|0.999911|0.000089|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -221,17 +188,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.974915|0.984168|0.993598|0.968829|0.984291|0.984289|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.587073|0.380771|0.281759|0.235156|0.469773|0.469749|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999964|0.999972|0.999980|0.999944|0.999972|0.999972|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.604361|0.385016|0.282490|0.238402|0.483503|0.483480|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -239,8 +197,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|41972710|1349800|626|379565968310|1349800 / 379567318110|626 / 41973336|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|10079118|888077|31894218|379566430033|888077 / 379567318110|31894218 / 41973336|
|
||||
|1|Radixor|ALL_CANDIDATES|41938414|1737|626|360919541853|1737 / 360919543590|626 / 41939040|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|10061203|263629|31877837|360919279961|263629 / 360919543590|31877837 / 41939040|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -250,26 +208,26 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|416642|119847|351885|17877|2.051686%|5|890999|
|
||||
|Radixor|898026|288481|1061317|42637|4.893313%|21|916797|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|412822|21039|19090|11309|1.331001%|5|861853|
|
||||
|Radixor|884428|0|1737|20967|2.467690%|21|871404|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.989429|276044 / 377860669765 (0.000073%)|885033 / 41863370 (2.114099%)|0.990385|0.986031|0.986056|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.652720|2201196 / 377860669765 (0.000583%)|29076352 / 41863370 (69.455354%)|0.627946|0.449839|0.510450|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.652720|2201196 / 377860669765 (0.000583%)|29076352 / 41863370 (69.455354%)|0.627946|0.449839|0.510450|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.614999|531181 / 377860669765 (0.000141%)|32234855 / 41863370 (77.000144%)|0.583531|0.370163|0.466854|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.514832|146613 / 377860669765 (0.000039%)|40621522 / 41863370 (97.033569%)|0.130949|0.057424|0.162875|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.503877|57716 / 377860669765 (0.000015%)|41538714 / 41863370 (99.224487%)|0.037409|0.015370|0.081139|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.501770|47148 / 377860669765 (0.000012%)|41715144 / 41863370 (99.645929%)|0.017379|0.007049|0.051824|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.989580|<0.000001%|2.084022%|
|
||||
|2|SNOWBALL SPANISH DIRECT|0.652542|0.000410%|69.491126%|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|0.652542|0.000410%|69.491126%|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|0.614924|0.000068%|77.015229%|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|0.514575|0.000009%|97.085003%|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|0.503767|0.000002%|99.246590%|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|0.501679|0.000001%|99.664108%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -277,13 +235,13 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.993309|0.978859|0.999999|0.989429|0.999997|0.000003|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.853138|0.305446|0.999994|0.652720|0.999917|0.000083|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.853138|0.305446|0.999994|0.652720|0.999917|0.000083|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.947717|0.229999|0.999999|0.614999|0.999913|0.000087|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.894406|0.029664|1.000000|0.514832|0.999892|0.000108|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.849058|0.007755|1.000000|0.503877|0.999890|0.000110|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.758678|0.003541|1.000000|0.501770|0.999889|0.000111|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.999999|0.979160|1.000000|0.989580|0.999998|0.000002|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.896551|0.305089|0.999996|0.652542|0.999915|0.000085|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.896551|0.305089|0.999996|0.652542|0.999915|0.000085|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.975224|0.229848|0.999999|0.614924|0.999910|0.000090|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.974539|0.029150|1.000000|0.514575|0.999887|0.000113|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.979521|0.007534|1.000000|0.503767|0.999884|0.000116|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.971230|0.003359|1.000000|0.501679|0.999884|0.000116|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -291,27 +249,13 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990385|0.986031|0.981715|0.972447|0.986057|0.986056|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.627946|0.449839|0.350441|0.290188|0.510478|0.510450|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.627946|0.449839|0.350441|0.290188|0.510478|0.510450|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.583531|0.370163|0.271053|0.227117|0.466876|0.466854|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.130949|0.057424|0.036775|0.029561|0.162886|0.162875|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.037409|0.015370|0.009672|0.007744|0.081145|0.081139|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.017379|0.007049|0.004421|0.003537|0.051829|0.051824|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986029|0.995464|0.993323|0.994392|0.994392|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.449806|0.981469|0.852556|0.912482|0.912482|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.449806|0.981469|0.852556|0.912482|0.912482|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.370136|0.993362|0.790500|0.880396|0.880396|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.057417|0.993866|0.756725|0.859234|0.859234|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.015368|0.995484|0.723753|0.838145|0.838145|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.007048|0.995676|0.710626|0.829341|0.829341|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995761|0.989470|0.983258|0.979159|0.989525|0.989523|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|0.646055|0.455257|0.351461|0.294714|0.522999|0.522972|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.646055|0.455257|0.351461|0.294714|0.522999|0.522972|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|0.591553|0.372016|0.271323|0.228513|0.473448|0.473426|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.130175|0.056607|0.036167|0.029128|0.168546|0.168536|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|0.036546|0.014953|0.009400|0.007533|0.085906|0.085901|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.016565|0.006695|0.004195|0.003359|0.057116|0.057113|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -319,71 +263,46 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|40978337|276044|885033|377860393721|276044 / 377860669765|885033 / 41863370|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|12787018|2201196|29076352|377858468569|2201196 / 377860669765|29076352 / 41863370|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|12787018|2201196|29076352|377858468569|2201196 / 377860669765|29076352 / 41863370|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|9628515|531181|32234855|377860138584|531181 / 377860669765|32234855 / 41863370|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|1241848|146613|40621522|377860523152|146613 / 377860669765|40621522 / 41863370|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|324656|57716|41538714|377860612049|57716 / 377860669765|41538714 / 41863370|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|148226|47148|41715144|377860622617|47148 / 377860669765|41715144 / 41863370|
|
||||
|1|Radixor|PRIMARY_OUTPUT|40958710|34|871756|359407144881|34 / 359407144915|871756 / 41830466|
|
||||
|2|SNOWBALL SPANISH DIRECT|PRIMARY_OUTPUT|12762004|1472547|29068462|359405672368|1472547 / 359407144915|29068462 / 41830466|
|
||||
|3|SNOWBALL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|12762004|1472547|29068462|359405672368|1472547 / 359407144915|29068462 / 41830466|
|
||||
|4|HUNSPELL SPANISH LUCENE FILTER|PRIMARY_OUTPUT|9614637|244260|32215829|359406900655|244260 / 359407144915|32215829 / 41830466|
|
||||
|5|SPANISH LUCENE SPANISH LIGHT STEM FILTER|PRIMARY_OUTPUT|1219357|31857|40611109|359407113058|31857 / 359407144915|40611109 / 41830466|
|
||||
|6|SPANISH LUCENE SPANISH PLURAL STEM FILTER|PRIMARY_OUTPUT|315155|6589|41515311|359407138326|6589 / 359407144915|41515311 / 41830466|
|
||||
|7|SPANISH LUCENE SPANISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|140505|4162|41689961|359407140753|4162 / 359407144915|41689961 / 41830466|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 377860669765 (0.000000%)|0 / 41863370 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.619928|412198 / 377860669765 (0.000109%)|31822108 / 41863370 (76.014205%)|0.600000|0.383864|0.479978|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|0.000062%|76.037484%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.960568|0.239858|0.999999|0.619928|0.999915|0.000085|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|0.600000|0.383864|0.282205|0.237519|0.480000|0.479978|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|41863370|0|0|377860669765|0 / 377860669765|0 / 41863370|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ANY_CANDIDATE|10041262|412198|31822108|377860257567|412198 / 377860669765|31822108 / 41863370|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|223274 / 359407144915|31806834 / 41830466|
|
||||
|Radixor|0 / 359407144915|0 / 41830466|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999998|1255381 / 377860669765 (0.000332%)|0 / 41863370 (0.000000%)|0.976572|0.985228|0.985334|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.619928|878949 / 377860669765 (0.000233%)|31822108 / 41863370 (76.014205%)|0.586905|0.380469|0.469606|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|<0.000001%|0.000000%|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|0.619812|0.000073%|76.037484%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -391,8 +310,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.970885|1.000000|0.999997|0.999998|0.999997|0.000003|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.919512|0.239858|0.999998|0.619928|0.999913|0.000087|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999987|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.974405|0.239625|0.999999|0.619812|0.999911|0.000089|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -400,17 +319,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.976572|0.985228|0.994038|0.970885|0.985335|0.985334|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.586905|0.380469|0.281467|0.234926|0.469630|0.469606|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999989|0.999993|0.999997|0.999987|0.999993|0.999993|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|0.603992|0.384656|0.282183|0.238126|0.483210|0.483187|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -418,8 +328,8 @@ This mode contains **11 result rows**, **7 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|41863370|1255381|0|377859414384|1255381 / 377860669765|0 / 41863370|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|10041262|878949|31822108|377859790816|878949 / 377860669765|31822108 / 41863370|
|
||||
|1|Radixor|ALL_CANDIDATES|41830466|560|0|359407144355|560 / 359407144915|0 / 41830466|
|
||||
|2|HUNSPELL SPANISH LUCENE FILTER|ALL_CANDIDATES|10023632|263289|31806834|359406881626|263289 / 359407144915|31806834 / 41830466|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -429,20 +339,20 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|412747|118983|347768|17807|2.048262%|5|888962|
|
||||
|Radixor|885033|276044|979337|42403|4.877434%|21|914127|
|
||||
|HUNSPELL SPANISH LUCENE FILTER|408995|20986|19029|11287|1.331204%|5|860048|
|
||||
|Radixor|871756|34|526|20911|2.466272%|21|869542|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -451,16 +361,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `ES_ES`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `SV_SE` | 12,371 | 110,468 | 24,731 | 85,737 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `sv-se-default` | `1.0.0` | `SV_SE` | 12,371 | 110,468 | 24,731 | 85,737 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **110,468**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **110,468**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 502 | 0.454% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 14,268 | 12.916% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 66,796 | 60.466% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 25,745 | 23.305% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 3,157 | 2.858% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 711 | 0.644% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 14,126 | 12.787% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 68,749 | 62.234% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 23,583 | 21.348% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 3,299 | 2.986% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -36,17 +36,23 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Official Snowball direct | 40.068% | 37.512% | 48.926% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
| Lucene SnowballFilter | 38.785% | 35.839% | 48.999% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `swedishRadixor` | 5.489 | 0.355 | 64.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.630 | 0.130 | 54.0 | 0.843 | Minimal Swedish suffix reducer. |
|
||||
| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.876 | 0.328 | 56.9 | 0.888 | Light Swedish suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[SWEDISH]` | 7.517 | 0.072 | 87.7 | 1.370 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 9.793 | 0.338 | 114.2 | 1.784 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `swedishRadixor` | 5.476 | 0.081 | 63.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.741 | 0.086 | 55.3 | 0.866 | Minimal Swedish suffix reducer. |
|
||||
| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.893 | 0.053 | 57.1 | 0.893 | Light Swedish suffix stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[SWEDISH]` | 7.606 | 0.555 | 88.7 | 1.389 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 10.295 | 0.749 | 120.1 | 1.880 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -60,31 +66,31 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `SV_SE` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `SV_SE` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/sv_se/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `sv-se-default`, loaded from classpath resource `org/egothor/stemmer/models/sv-se-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.974636** among 5 deterministic stemmers. The runner-up is `SNOWBALL SWEDISH DIRECT` at 0.807534, a difference of 0.167101. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.974584** among 5 deterministic stemmers. The runner-up is `SNOWBALL SWEDISH DIRECT` at 0.807599, a difference of 0.166985. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.977619** among 5 deterministic stemmers. The runner-up is `SNOWBALL SWEDISH DIRECT` at 0.808543, a difference of 0.169076. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.977573** among 5 deterministic stemmers. The runner-up is `SNOWBALL SWEDISH DIRECT` at 0.808611, a difference of 0.168961. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.974636|24473 / 4812155436 (0.000509%)|19546 / 385342 (5.072377%)|0.939665|0.943246|0.943260|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.807534|67105 / 4812155436 (0.001394%)|148325 / 385342 (38.491781%)|0.739832|0.687540|0.692339|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.799307|64262 / 4812155436 (0.001335%)|154666 / 385342 (40.137333%)|0.736940|0.678180|0.684227|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.796072|40227 / 4812155436 (0.000836%)|157161 / 385342 (40.784809%)|0.781991|0.698068|0.709491|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.783685|45941 / 4812155436 (0.000955%)|166707 / 385342 (43.262089%)|0.757232|0.672808|0.684713|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.977619|0.000000%|4.476263%|
|
||||
|2|SNOWBALL SWEDISH DIRECT|0.808543|0.000821%|38.290570%|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|0.800222|0.000775%|39.954747%|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|0.797907|0.000439%|40.418073%|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|0.785227|0.000534%|42.954113%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -92,11 +98,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.937292|0.949276|0.999995|0.974636|0.999991|0.000009|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.779348|0.615082|0.999986|0.807534|0.999955|0.000045|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.782117|0.598627|0.999987|0.799307|0.999955|0.000045|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.850127|0.592152|0.999992|0.796072|0.999959|0.000041|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.826360|0.567379|0.999990|0.783685|0.999956|0.000044|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.955237|1.000000|0.977619|0.999996|0.000004|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.863080|0.617094|0.999992|0.808543|0.999960|0.000040|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.866630|0.600453|0.999992|0.800222|0.999959|0.000041|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.919176|0.595819|0.999996|0.797907|0.999962|0.000038|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.899588|0.570459|0.999995|0.785227|0.999959|0.000041|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -104,23 +110,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.939665|0.943246|0.946855|0.892588|0.943265|0.943260|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.739832|0.687540|0.642152|0.523856|0.692361|0.692339|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.736940|0.678180|0.628098|0.513065|0.684249|0.684227|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.781991|0.698068|0.630412|0.536179|0.709510|0.709491|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.757232|0.672808|0.605321|0.506941|0.684733|0.684713|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.943241|0.992631|0.993395|0.993013|0.993013|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.687518|0.984860|0.942685|0.963311|0.963311|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.678157|0.985207|0.939659|0.961894|0.961894|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.698048|0.988493|0.944582|0.966038|0.966038|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.672787|0.986795|0.942303|0.964036|0.964036|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990715|0.977106|0.963866|0.955237|0.977362|0.977361|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.799353|0.719647|0.654396|0.562070|0.729796|0.729777|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.796053|0.709394|0.639751|0.549660|0.721367|0.721348|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.829176|0.722989|0.640913|0.566158|0.740043|0.740026|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.806522|0.698179|0.615497|0.536309|0.716364|0.716347|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -128,63 +122,41 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|365796|24473|19546|4812130963|24473 / 4812155436|19546 / 385342|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|237017|67105|148325|4812088331|67105 / 4812155436|148325 / 385342|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|230676|64262|154666|4812091174|64262 / 4812155436|154666 / 385342|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|228181|40227|157161|4812115209|40227 / 4812155436|157161 / 385342|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|218635|45941|166707|4812109495|45941 / 4812155436|166707 / 385342|
|
||||
|1|Radixor|PRIMARY_OUTPUT|362653|0|16994|4529284143|0 / 4529284143|16994 / 379647|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|234278|37166|145369|4529246977|37166 / 4529284143|145369 / 379647|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|227960|35082|151687|4529249061|35082 / 4529284143|151687 / 379647|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|226201|19890|153446|4529264253|19890 / 4529284143|153446 / 379647|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|216573|24174|163074|4529259969|24174 / 4529284143|163074 / 379647|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 4812155436 (0.000000%)|0 / 385342 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|385342|0|0|4812155436|0 / 4812155436|0 / 385342|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 4529284143|0 / 379647|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999995|47848 / 4812155436 (0.000994%)|0 / 385342 (0.000000%)|0.909640|0.941544|0.943152|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -192,7 +164,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.889545|1.000000|0.999990|0.999995|0.999990|0.000010|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -200,15 +172,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.909640|0.941544|0.975768|0.889545|0.943157|0.943152|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -216,7 +180,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|385342|47848|0|4812107588|47848 / 4812155436|0 / 385342|
|
||||
|1|Radixor|ALL_CANDIDATES|379647|0|0|4529284143|0 / 4529284143|0 / 379647|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -226,23 +190,23 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19546|24473|23375|5767|5.878216%|5|104148|
|
||||
|Radixor|16994|0|0|2840|2.983789%|5|98108|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.974584|24473 / 4789911577 (0.000511%)|19546 / 384563 (5.082652%)|0.939544|0.943132|0.943146|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.807599|67105 / 4789911577 (0.001401%)|147975 / 384563 (38.478741%)|0.739645|0.687500|0.692274|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.799355|64262 / 4789911577 (0.001342%)|154316 / 384563 (40.127625%)|0.736744|0.678122|0.684143|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.795947|40227 / 4789911577 (0.000840%)|156939 / 384563 (40.809698%)|0.781694|0.697790|0.709212|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.783598|45941 / 4789911577 (0.000959%)|166437 / 384563 (43.279515%)|0.756945|0.672575|0.684469|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.977573|0.000000%|4.485467%|
|
||||
|2|SNOWBALL SWEDISH DIRECT|0.808611|0.000824%|38.276920%|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|0.800274|0.000778%|39.944519%|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|0.797785|0.000441%|40.442582%|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|0.785141|0.000536%|42.971167%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -250,11 +214,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.937167|0.949173|0.999995|0.974584|0.999991|0.000009|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.779037|0.615213|0.999986|0.807599|0.999955|0.000045|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.781800|0.598724|0.999987|0.799355|0.999954|0.000046|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.849816|0.591903|0.999992|0.795947|0.999959|0.000041|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.826025|0.567205|0.999990|0.783598|0.999956|0.000044|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.955145|1.000000|0.977573|0.999996|0.000004|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.862864|0.617231|0.999992|0.808611|0.999960|0.000040|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.866412|0.600555|0.999992|0.800274|0.999959|0.000041|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.918993|0.595574|0.999996|0.797785|0.999962|0.000038|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.899375|0.570288|0.999995|0.785141|0.999959|0.000041|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -262,23 +226,11 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.939544|0.943132|0.946748|0.892384|0.943151|0.943146|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.739645|0.687500|0.642223|0.523810|0.692296|0.692274|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.736744|0.678122|0.628142|0.512999|0.684165|0.684143|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.781694|0.697790|0.630152|0.535851|0.709231|0.709212|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.756945|0.672575|0.605126|0.506676|0.684489|0.684469|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.943127|0.992612|0.993378|0.992995|0.992995|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.687478|0.984821|0.942695|0.963298|0.963298|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.678100|0.985169|0.939661|0.961877|0.961877|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.697770|0.988463|0.944528|0.965996|0.965996|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.672553|0.986761|0.942265|0.964000|0.964000|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.990695|0.977058|0.963791|0.955145|0.977315|0.977314|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|0.799250|0.719665|0.654494|0.562091|0.729785|0.729766|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|0.795941|0.709393|0.639820|0.549658|0.721337|0.721319|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|0.828962|0.722752|0.640668|0.565867|0.739816|0.739800|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|0.806317|0.697987|0.615318|0.536083|0.716172|0.716155|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -286,63 +238,41 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|365017|24473|19546|4789887104|24473 / 4789911577|19546 / 384563|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|236588|67105|147975|4789844472|67105 / 4789911577|147975 / 384563|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|230247|64262|154316|4789847315|64262 / 4789911577|154316 / 384563|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|227624|40227|156939|4789871350|40227 / 4789911577|156939 / 384563|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|218126|45941|166437|4789865636|45941 / 4789911577|166437 / 384563|
|
||||
|1|Radixor|PRIMARY_OUTPUT|361874|0|16994|4507704713|0 / 4507704713|16994 / 378868|
|
||||
|2|SNOWBALL SWEDISH DIRECT|PRIMARY_OUTPUT|233849|37166|145019|4507667547|37166 / 4507704713|145019 / 378868|
|
||||
|3|SNOWBALL SWEDISH LUCENE FILTER|PRIMARY_OUTPUT|227531|35082|151337|4507669631|35082 / 4507704713|151337 / 378868|
|
||||
|4|SWEDISH LUCENE SWEDISH MINIMAL STEM FILTER|PRIMARY_OUTPUT|225644|19890|153224|4507684823|19890 / 4507704713|153224 / 378868|
|
||||
|5|SWEDISH LUCENE SWEDISH LIGHT STEM FILTER|PRIMARY_OUTPUT|216064|24174|162804|4507680539|24174 / 4507704713|162804 / 378868|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 4789911577 (0.000000%)|0 / 384563 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|384563|0|0|4789911577|0 / 4789911577|0 / 384563|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 4507704713|0 / 378868|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999995|47848 / 4789911577 (0.000999%)|0 / 384563 (0.000000%)|0.909473|0.941433|0.943047|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -350,7 +280,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.889346|1.000000|0.999990|0.999995|0.999990|0.000010|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -358,15 +288,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.909473|0.941433|0.975720|0.889346|0.943051|0.943047|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -374,7 +296,7 @@ This mode contains **7 result rows**, **5 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|384563|47848|0|4789863729|47848 / 4789911577|0 / 384563|
|
||||
|1|Radixor|ALL_CANDIDATES|378868|0|0|4507704713|0 / 4507704713|0 / 378868|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -384,19 +306,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|19546|24473|23375|5767|5.891848%|5|103921|
|
||||
|Radixor|16994|0|0|2840|2.990922%|5|97881|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -405,16 +327,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `SV_SE`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,21 +8,21 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `UK_UA` | 1,493 | 15,737 | 2,985 | 12,752 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `uk-ua-default` | `1.0.0` | `UK_UA` | 1,493 | 15,737 | 2,985 | 12,752 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **15,737**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **15,737**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 249 | 1.582% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 4,160 | 26.435% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 5,859 | 37.231% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 3,004 | 19.089% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 2,465 | 15.664% |
|
||||
| `AppendCharacterCommand` | Appends one character to the end of the word form. | 267 | 1.697% |
|
||||
| `BackwardCompoundCommand` | Applies a multi-step backward patch made from skip, delete, insert, and replace operations. | 4,156 | 26.409% |
|
||||
| `DeleteSuffixCommand` | Deletes one or more trailing characters from the word form. | 5,883 | 37.383% |
|
||||
| `PreserveCommand` | Returns the word form unchanged because it already matches the preferred root. | 2,962 | 18.822% |
|
||||
| `ReplaceLastCharacterCommand` | Replaces the final character of the word form. | 2,469 | 15.689% |
|
||||
|
||||
## Accuracy
|
||||
|
||||
@@ -35,16 +35,22 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene MorfologikFilter | 92.362% | 90.637% | 99.732% | Dictionary-based path; Morfologik can emit multiple terms. |
|
||||
| Morfologik direct | 92.362% | 90.637% | 99.732% | Direct dictionary lookup; first returned stem is used for quality when no ranking weight is exposed. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `ukrainianRadixor` | 0.682 | 0.057 | 53.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 43.527 | 1.207 | 3413.3 | 63.799 | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Morfologik direct | `ukrainianMorfologikDirect` | 8.680 | 0.073 | 680.7 | 12.723 | Direct Morfologik dictionary lookup; first returned stem is used for quality. |
|
||||
| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 14.575 | 0.248 | 1143.0 | 21.364 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||
| Radixor | `ukrainianRadixor` | 0.639 | 0.009 | 50.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 47.919 | 5.308 | 3757.8 | 74.957 | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||
| Morfologik direct | `ukrainianMorfologikDirect` | 8.662 | 0.105 | 679.3 | 13.550 | Direct Morfologik dictionary lookup; first returned stem is used for quality. |
|
||||
| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 15.367 | 0.219 | 1205.1 | 24.038 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -58,30 +64,30 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `UK_UA` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `UK_UA` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/uk_ua/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `uk-ua-default`, loaded from classpath resource `org/egothor/stemmer/models/uk-ua-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.995343** among 4 deterministic stemmers. The runner-up is `UKRAINIAN LUCENE MORFOLOGIK FILTER` at 0.928768, a difference of 0.066575. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.995342** among 4 deterministic stemmers. The runner-up is `UKRAINIAN LUCENE MORFOLOGIK FILTER` at 0.928751, a difference of 0.066591. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.995816** among 4 deterministic stemmers. The runner-up is `UKRAINIAN LUCENE MORFOLOGIK FILTER` at 0.928906, a difference of 0.066910. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.995815** among 4 deterministic stemmers. The runner-up is `UKRAINIAN LUCENE MORFOLOGIK FILTER` at 0.928888, a difference of 0.066926. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995343|880 / 101387550 (0.000868%)|608 / 65340 (0.930517%)|0.987406|0.988637|0.988632|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.928768|828 / 101387550 (0.000817%)|9308 / 65340 (14.245485%)|0.956896|0.917054|0.919223|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.928646|828 / 101387550 (0.000817%)|9324 / 65340 (14.269972%)|0.956832|0.916912|0.919090|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.885793|794 / 101387550 (0.000783%)|14924 / 65340 (22.840526%)|0.933008|0.865139|0.871499|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.995816|0.000000%|0.836852%|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.928906|0.000028%|14.218810%|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|0.928783|0.000028%|14.243378%|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|0.885789|0.000006%|22.842226%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -89,10 +95,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986588|0.990695|0.999991|0.995343|0.999985|0.000015|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.985438|0.857545|0.999992|0.928768|0.999900|0.000100|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.985434|0.857300|0.999992|0.928646|0.999900|0.000100|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.984495|0.771595|0.999992|0.885793|0.999845|0.000155|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.991631|1.000000|0.995816|0.999995|0.000005|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.999499|0.857812|1.000000|0.928906|0.999907|0.000093|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.999499|0.857566|1.000000|0.928783|0.999907|0.000093|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.999881|0.771578|1.000000|0.885789|0.999851|0.000149|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -100,21 +106,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987406|0.988637|0.989871|0.977529|0.988639|0.988632|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.956896|0.917054|0.880397|0.846814|0.919270|0.919223|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.956832|0.916912|0.880190|0.846572|0.919137|0.919090|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.933008|0.865139|0.806475|0.762331|0.871568|0.871499|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988630|0.997994|0.998266|0.998130|0.998130|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.917004|0.997990|0.971000|0.984310|0.984310|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.916862|0.997990|0.970876|0.984246|0.984246|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.865063|0.998114|0.949804|0.973360|0.973360|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998315|0.995798|0.993294|0.991631|0.995807|0.995804|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.967537|0.923251|0.882842|0.857443|0.925949|0.925906|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.967474|0.923109|0.882634|0.857198|0.925817|0.925774|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.944015|0.871018|0.808499|0.771507|0.878343|0.878277|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -122,80 +117,49 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|64732|880|608|101386670|880 / 101387550|608 / 65340|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|56032|828|9308|101386722|828 / 101387550|9308 / 65340|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|56016|828|9324|101386722|828 / 101387550|9324 / 65340|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|50416|794|14924|101386756|794 / 101387550|14924 / 65340|
|
||||
|1|Radixor|PRIMARY_OUTPUT|64580|0|545|100039050|0 / 100039050|545 / 65125|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|55865|28|9260|100039022|28 / 100039050|9260 / 65125|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|55849|28|9276|100039022|28 / 100039050|9276 / 65125|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|50249|6|14876|100039044|6 / 100039050|14876 / 65125|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 101387550 (0.000000%)|0 / 65340 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.962151|122 / 101387550 (0.000120%)|4946 / 65340 (7.569636%)|0.982323|0.959732|0.960413|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.962029|122 / 101387550 (0.000120%)|4962 / 65340 (7.594123%)|0.982267|0.959599|0.960286|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.927570|326 / 101387550 (0.000322%)|9465 / 65340 (14.485767%)|0.962884|0.919443|0.922008|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|0.000000%|14.533589%|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.000000%|7.594626%|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|0.000000%|7.619194%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.997984|0.924304|0.999999|0.962151|0.999950|0.000050|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.997983|0.924059|0.999999|0.962029|0.999950|0.000050|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.994199|0.855142|0.999997|0.927570|0.999903|0.000097|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.982323|0.959732|0.938156|0.922581|0.960438|0.960413|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.982267|0.959599|0.937954|0.922337|0.960310|0.960286|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.962884|0.919443|0.879752|0.850897|0.922053|0.922008|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|65340|0|0|101387550|0 / 101387550|0 / 65340|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|60394|122|4946|101387428|122 / 101387550|4946 / 65340|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|60378|122|4962|101387428|122 / 101387550|4962 / 65340|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|55875|326|9465|101387224|326 / 101387550|9465 / 65340|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|0 / 100039050|9465 / 65125|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|0 / 100039050|4946 / 65125|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|0 / 100039050|4962 / 65125|
|
||||
|Radixor|0 / 100039050|0 / 65125|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999993|1490 / 101387550 (0.001470%)|0 / 65340 (0.000000%)|0.982084|0.988727|0.988782|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.962145|1368 / 101387550 (0.001349%)|4946 / 65340 (7.569636%)|0.966650|0.950323|0.950669|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.962023|1368 / 101387550 (0.001349%)|4962 / 65340 (7.594123%)|0.966592|0.950191|0.950541|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.927565|1271 / 101387550 (0.001254%)|9465 / 65340 (14.485767%)|0.950501|0.912349|0.914347|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.962027|0.000059%|7.594626%|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|0.961904|0.000059%|7.619194%|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|0.927332|0.000047%|14.533589%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -203,10 +167,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.977705|1.000000|0.999985|0.999993|0.999985|0.000015|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.977850|0.924304|0.999987|0.962145|0.999938|0.000062|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.977845|0.924059|0.999987|0.962023|0.999938|0.000062|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.977759|0.855142|0.999987|0.927565|0.999894|0.000106|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.999021|0.924054|0.999999|0.962027|0.999950|0.000050|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.999020|0.923808|0.999999|0.961904|0.999950|0.000050|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.999156|0.854664|1.000000|0.927332|0.999905|0.000095|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -214,21 +178,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.982084|0.988727|0.995460|0.977705|0.988789|0.988782|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.966650|0.950323|0.934539|0.905349|0.950700|0.950669|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.966592|0.950191|0.934337|0.905109|0.950571|0.950541|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.950501|0.912349|0.877142|0.838825|0.914398|0.914347|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.983070|0.960076|0.938133|0.923217|0.960806|0.960782|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.983014|0.959943|0.937931|0.922972|0.960678|0.960654|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.966477|0.921279|0.880120|0.854048|0.924090|0.924046|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -236,10 +189,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|65340|1490|0|101386060|1490 / 101387550|0 / 65340|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|60394|1368|4946|101386182|1368 / 101387550|4946 / 65340|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|60378|1368|4962|101386182|1368 / 101387550|4962 / 65340|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|55875|1271|9465|101386279|1271 / 101387550|9465 / 65340|
|
||||
|1|Radixor|ALL_CANDIDATES|65125|0|0|100039050|0 / 100039050|0 / 65125|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|60179|59|4946|100038991|59 / 100039050|4946 / 65125|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|60163|59|4962|100038991|59 / 100039050|4962 / 65125|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|55660|47|9465|100039003|47 / 100039050|9465 / 65125|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -249,25 +202,25 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|5459|468|477|1322|9.280449%|6|15740|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|4362|706|540|2207|15.493155%|6|16937|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|4362|706|540|2207|15.493155%|6|16937|
|
||||
|Radixor|608|880|610|190|1.333801%|2|14435|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|5411|6|41|1259|8.897527%|6|15577|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|4314|28|31|2130|15.053004%|6|16748|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|4314|28|31|2130|15.053004%|6|16748|
|
||||
|Radixor|545|0|0|95|0.671378%|2|14245|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995342|880 / 101259406 (0.000869%)|608 / 65324 (0.930745%)|0.987403|0.988634|0.988629|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.928751|828 / 101259406 (0.000818%)|9308 / 65324 (14.248974%)|0.956884|0.917032|0.919202|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.928751|828 / 101259406 (0.000818%)|9308 / 65324 (14.248974%)|0.956884|0.917032|0.919202|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.885796|794 / 101259406 (0.000784%)|14920 / 65324 (22.839998%)|0.933007|0.865141|0.871500|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.995815|0.000000%|0.837058%|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.928888|0.000028%|14.222304%|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|0.928888|0.000028%|14.222304%|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|0.885791|0.000006%|22.841696%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -275,10 +228,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.986585|0.990693|0.999991|0.995342|0.999985|0.000015|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.985434|0.857510|0.999992|0.928751|0.999900|0.000100|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.985434|0.857510|0.999992|0.928751|0.999900|0.000100|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.984492|0.771600|0.999992|0.885796|0.999845|0.000155|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.991629|1.000000|0.995815|0.999995|0.000005|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.999499|0.857777|1.000000|0.928888|0.999907|0.000093|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.999499|0.857777|1.000000|0.928888|0.999907|0.000093|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.999881|0.771583|1.000000|0.885791|0.999851|0.000149|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -286,21 +239,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.987403|0.988634|0.989868|0.977524|0.988636|0.988629|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.956884|0.917032|0.880367|0.846777|0.919249|0.919202|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.956884|0.917032|0.880367|0.846777|0.919249|0.919202|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.933007|0.865141|0.806479|0.762334|0.871570|0.871500|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988627|0.997992|0.998264|0.998128|0.998128|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.916982|0.997988|0.970978|0.984298|0.984298|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.916982|0.997988|0.970978|0.984298|0.984298|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.865065|0.998113|0.949788|0.973351|0.973351|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.998315|0.995797|0.993292|0.991629|0.995806|0.995803|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|0.967528|0.923231|0.882812|0.857408|0.925930|0.925887|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|0.967528|0.923231|0.882812|0.857408|0.925930|0.925887|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|0.944017|0.871021|0.808503|0.771512|0.878346|0.878280|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -308,80 +250,49 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|64716|880|608|101258526|880 / 101259406|608 / 65324|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|56016|828|9308|101258578|828 / 101259406|9308 / 65324|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|56016|828|9308|101258578|828 / 101259406|9308 / 65324|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|50404|794|14920|101258612|794 / 101259406|14920 / 65324|
|
||||
|1|Radixor|PRIMARY_OUTPUT|64564|0|545|99911761|0 / 99911761|545 / 65109|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|PRIMARY_OUTPUT|55849|28|9260|99911733|28 / 99911761|9260 / 65109|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|PRIMARY_OUTPUT|55849|28|9260|99911733|28 / 99911761|9260 / 65109|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|PRIMARY_OUTPUT|50237|6|14872|99911755|6 / 99911761|14872 / 65109|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 101259406 (0.000000%)|0 / 65324 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.962142|122 / 101259406 (0.000120%)|4946 / 65324 (7.571490%)|0.982318|0.959722|0.960404|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.962142|122 / 101259406 (0.000120%)|4946 / 65324 (7.571490%)|0.982318|0.959722|0.960404|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.927552|326 / 101259406 (0.000322%)|9465 / 65324 (14.489315%)|0.962874|0.919422|0.921988|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|0.000000%|14.537161%|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.000000%|7.596492%|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|0.000000%|7.596492%|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.997983|0.924285|0.999999|0.962142|0.999950|0.000050|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.997983|0.924285|0.999999|0.962142|0.999950|0.000050|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.994198|0.855107|0.999997|0.927552|0.999903|0.000097|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|0.982318|0.959722|0.938141|0.922562|0.960428|0.960404|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|0.982318|0.959722|0.938141|0.922562|0.960428|0.960404|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|0.962874|0.919422|0.879722|0.850861|0.922033|0.921988|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|65324|0|0|101259406|0 / 101259406|0 / 65324|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ANY_CANDIDATE|60378|122|4946|101259284|122 / 101259406|4946 / 65324|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ANY_CANDIDATE|60378|122|4946|101259284|122 / 101259406|4946 / 65324|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ANY_CANDIDATE|55859|326|9465|101259080|326 / 101259406|9465 / 65324|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|0 / 99911761|9465 / 65109|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|0 / 99911761|4946 / 65109|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|0 / 99911761|4946 / 65109|
|
||||
|Radixor|0 / 99911761|0 / 65109|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999993|1490 / 101259406 (0.001471%)|0 / 65324 (0.000000%)|0.982079|0.988724|0.988779|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.962136|1368 / 101259406 (0.001351%)|4946 / 65324 (7.571490%)|0.966642|0.950311|0.950657|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.962136|1368 / 101259406 (0.001351%)|4946 / 65324 (7.571490%)|0.966642|0.950311|0.950657|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.927547|1271 / 101259406 (0.001255%)|9465 / 65324 (14.489315%)|0.950487|0.912326|0.914325|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.962017|0.000059%|7.596492%|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|0.962017|0.000059%|7.596492%|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|0.927314|0.000047%|14.537161%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -389,10 +300,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.977699|1.000000|0.999985|0.999993|0.999985|0.000015|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.977845|0.924285|0.999986|0.962136|0.999938|0.000062|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.977845|0.924285|0.999986|0.962136|0.999938|0.000062|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.977752|0.855107|0.999987|0.927547|0.999894|0.000106|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.999020|0.924035|0.999999|0.962017|0.999950|0.000050|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.999020|0.924035|0.999999|0.962017|0.999950|0.000050|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.999156|0.854628|1.000000|0.927314|0.999905|0.000095|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -400,21 +311,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.982079|0.988724|0.995459|0.977699|0.988787|0.988779|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.966642|0.950311|0.934522|0.905326|0.950688|0.950657|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.966642|0.950311|0.934522|0.905326|0.950688|0.950657|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.950487|0.912326|0.877111|0.838787|0.914376|0.914325|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|0.983065|0.960066|0.938118|0.923199|0.960796|0.960772|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|0.983065|0.960066|0.938118|0.923199|0.960796|0.960772|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|0.966468|0.921258|0.880089|0.854012|0.924071|0.924027|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -422,10 +322,10 @@ This mode contains **12 result rows**, **4 evaluated stemmers**, and **3 output
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|65324|1490|0|101257916|1490 / 101259406|0 / 65324|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|60378|1368|4946|101258038|1368 / 101259406|4946 / 65324|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|60378|1368|4946|101258038|1368 / 101259406|4946 / 65324|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|55859|1271|9465|101258135|1271 / 101259406|9465 / 65324|
|
||||
|1|Radixor|ALL_CANDIDATES|65109|0|0|99911761|0 / 99911761|0 / 65109|
|
||||
|2|UKRAINIAN LUCENE MORFOLOGIK FILTER|ALL_CANDIDATES|60163|59|4946|99911702|59 / 99911761|4946 / 65109|
|
||||
|3|UKRAINIAN MORFOLOGIK DIRECT|ALL_CANDIDATES|60163|59|4946|99911702|59 / 99911761|4946 / 65109|
|
||||
|4|HUNSPELL UKRAINIAN LUCENE FILTER|ALL_CANDIDATES|55644|47|9465|99911714|47 / 99911761|9465 / 65109|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -435,22 +335,22 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|5455|468|477|1321|9.279292%|6|15730|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|4362|706|540|2207|15.502950%|6|16928|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|4362|706|540|2207|15.502950%|6|16928|
|
||||
|Radixor|608|880|610|190|1.334645%|2|14426|
|
||||
|HUNSPELL UKRAINIAN LUCENE FILTER|5407|6|41|1258|8.896118%|6|15567|
|
||||
|UKRAINIAN LUCENE MORFOLOGIK FILTER|4314|28|31|2130|15.062584%|6|16739|
|
||||
|UKRAINIAN MORFOLOGIK DIRECT|4314|28|31|2130|15.062584%|6|16739|
|
||||
|Radixor|545|0|0|95|0.671805%|2|14236|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -459,16 +359,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `UK_UA`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -8,13 +8,13 @@ Radixor must not be read as simply "slower" when a narrow competitor has a lower
|
||||
|
||||
## Dictionary Corpus
|
||||
|
||||
| Resource | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `YI` | 802 | 4,300 | 1,524 | 2,776 |
|
||||
| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |
|
||||
| --- | --- | --- | ---: | ---: | ---: | ---: |
|
||||
| `yi-default` | `1.0.0` | `YI` | 802 | 4,300 | 1,524 | 2,776 |
|
||||
|
||||
## Radixor Patch Command Distribution
|
||||
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete language dictionary. The total number of preferred patch commands analyzed for this language is **4,300**.
|
||||
Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is **4,300**.
|
||||
|
||||
| Command class | Meaning | Word forms | Share |
|
||||
| --- | --- | ---: | ---: |
|
||||
@@ -33,15 +33,21 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun
|
||||
| Lucene SnowballFilter | 2.837% | 2.558% | 3.346% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||
| Official Snowball direct | 2.837% | 2.558% | 3.346% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Speed
|
||||
|
||||
Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fork, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower.
|
||||
|
||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||
| Radixor | `radixor[YIDDISH]` | 0.254 | 0.004 | 50.7 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.537 | 0.220 | 307.3 | 6.058 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.714 | 0.120 | 342.8 | 6.756 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
| Radixor | `radixor[YIDDISH]` | 0.249 | 0.001 | 89.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||
| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.574 | 0.066 | 567.1 | 6.330 | Official Snowball generated Java stemmer; direct API. |
|
||||
| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.849 | 0.079 | 665.9 | 7.434 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||
|
||||
|
||||
|
||||
|
||||
## Interpretation Notes
|
||||
|
||||
@@ -55,29 +61,29 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
||||
|
||||
## Stemming Quality
|
||||
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `YI` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.
|
||||
Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `YI` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.
|
||||
|
||||
`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).
|
||||
|
||||
### Evaluation Scope and Key Findings
|
||||
|
||||
The dictionary resource is `src/main/resources/yi/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
The default model is `yi-default`, loaded from classpath resource `org/egothor/stemmer/models/yi-default/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.
|
||||
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.988241** among 3 deterministic stemmers. The runner-up is `SNOWBALL YIDDISH DIRECT` at 0.890988, a difference of 0.097253. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.988241** among 3 deterministic stemmers. The runner-up is `SNOWBALL YIDDISH DIRECT` at 0.890988, a difference of 0.097253. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **ALL_WORDS:** `Radixor` ranks first by balanced accuracy at **0.989079** among 3 deterministic stemmers. The runner-up is `SNOWBALL YIDDISH DIRECT` at 0.891118, a difference of 0.097961. This rank does not imply leadership in throughput or every secondary metric.
|
||||
- **LOWERCASE_GROUPS_ONLY:** `Radixor` ranks first by balanced accuracy at **0.989079** among 3 deterministic stemmers. The runner-up is `SNOWBALL YIDDISH DIRECT` at 0.891118, a difference of 0.097961. This rank does not imply leadership in throughput or every secondary metric.
|
||||
### `ALL_WORDS`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988241|195 / 6392909 (0.003050%)|149 / 6344 (2.348676%)|0.970881|0.972986|0.972965|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.890988|1151 / 6392909 (0.018004%)|1382 / 6344 (21.784363%)|0.805624|0.796661|0.796600|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.890988|1151 / 6392909 (0.018004%)|1382 / 6344 (21.784363%)|0.805624|0.796661|0.796600|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.989079|0.000000%|2.184236%|
|
||||
|2|SNOWBALL YIDDISH DIRECT|0.891118|0.013211%|21.763216%|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|0.891118|0.013211%|21.763216%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -85,9 +91,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.969484|0.976513|0.999969|0.988241|0.999946|0.000054|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.811713|0.782156|0.999820|0.890988|0.999604|0.000396|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.811713|0.782156|0.999820|0.890988|0.999604|0.000396|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.978158|1.000000|0.989079|0.999978|0.000022|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.857267|0.782368|0.999868|0.891118|0.999648|0.000352|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.857267|0.782368|0.999868|0.891118|0.999648|0.000352|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -95,19 +101,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.970881|0.972986|0.975099|0.947393|0.972992|0.972965|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.805624|0.796661|0.787894|0.662041|0.796798|0.796600|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.805624|0.796661|0.787894|0.662041|0.796798|0.796600|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.972959|0.995691|0.996142|0.995917|0.995917|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.796462|0.982919|0.962014|0.972354|0.972354|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.796462|0.982919|0.962014|0.972354|0.972354|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995554|0.988958|0.982449|0.978158|0.989019|0.989008|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.841161|0.818107|0.796282|0.692200|0.818961|0.818787|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.841161|0.818107|0.796282|0.692200|0.818961|0.818787|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -115,61 +111,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6195|195|149|6392714|195 / 6392909|149 / 6344|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|4962|1151|1382|6391758|1151 / 6392909|1382 / 6344|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|4962|1151|1382|6391758|1151 / 6392909|1382 / 6344|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6180|0|138|6229428|0 / 6229428|138 / 6318|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|4943|823|1375|6228605|823 / 6229428|1375 / 6318|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|4943|823|1375|6228605|823 / 6229428|1375 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 6392909 (0.000000%)|0 / 6344 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|6344|0|0|6392909|0 / 6392909|0 / 6344|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 6229428|0 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999970|389 / 6392909 (0.006085%)|0 / 6344 (0.000000%)|0.953240|0.970253|0.970653|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -177,7 +151,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.942225|1.000000|0.999939|0.999970|0.999939|0.000061|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -185,15 +159,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.953240|0.970253|0.987885|0.942225|0.970683|0.970653|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -201,7 +167,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|6344|389|0|6392520|389 / 6392909|0 / 6344|
|
||||
|1|Radixor|ALL_CANDIDATES|6318|0|0|6229428|0 / 6229428|0 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -211,21 +177,21 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|149|195|194|89|2.487423%|3|3676|
|
||||
|Radixor|138|0|0|43|1.217441%|3|3578|
|
||||
|
||||
### `LOWERCASE_GROUPS_ONLY`
|
||||
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.
|
||||
|
||||
#### `PRIMARY_OUTPUT` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.988241|195 / 6392909 (0.003050%)|149 / 6344 (2.348676%)|0.970881|0.972986|0.972965|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.890988|1151 / 6392909 (0.018004%)|1382 / 6344 (21.784363%)|0.805624|0.796661|0.796600|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.890988|1151 / 6392909 (0.018004%)|1382 / 6344 (21.784363%)|0.805624|0.796661|0.796600|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|0.989079|0.000000%|2.184236%|
|
||||
|2|SNOWBALL YIDDISH DIRECT|0.891118|0.013211%|21.763216%|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|0.891118|0.013211%|21.763216%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -233,9 +199,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.969484|0.976513|0.999969|0.988241|0.999946|0.000054|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.811713|0.782156|0.999820|0.890988|0.999604|0.000396|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.811713|0.782156|0.999820|0.890988|0.999604|0.000396|
|
||||
|1|Radixor|PRIMARY_OUTPUT|1.000000|0.978158|1.000000|0.989079|0.999978|0.000022|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.857267|0.782368|0.999868|0.891118|0.999648|0.000352|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.857267|0.782368|0.999868|0.891118|0.999648|0.000352|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -243,19 +209,9 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.970881|0.972986|0.975099|0.947393|0.972992|0.972965|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.805624|0.796661|0.787894|0.662041|0.796798|0.796600|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.805624|0.796661|0.787894|0.662041|0.796798|0.796600|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.972959|0.995691|0.996142|0.995917|0.995917|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.796462|0.982919|0.962014|0.972354|0.972354|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.796462|0.982919|0.962014|0.972354|0.972354|
|
||||
|1|Radixor|PRIMARY_OUTPUT|0.995554|0.988958|0.982449|0.978158|0.989019|0.989008|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|0.841161|0.818107|0.796282|0.692200|0.818961|0.818787|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|0.841161|0.818107|0.796282|0.692200|0.818961|0.818787|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -263,61 +219,39 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6195|195|149|6392714|195 / 6392909|149 / 6344|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|4962|1151|1382|6391758|1151 / 6392909|1382 / 6344|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|4962|1151|1382|6391758|1151 / 6392909|1382 / 6344|
|
||||
|1|Radixor|PRIMARY_OUTPUT|6180|0|138|6229428|0 / 6229428|138 / 6318|
|
||||
|2|SNOWBALL YIDDISH DIRECT|PRIMARY_OUTPUT|4943|823|1375|6228605|823 / 6229428|1375 / 6318|
|
||||
|3|SNOWBALL YIDDISH LUCENE FILTER|PRIMARY_OUTPUT|4943|823|1375|6228605|823 / 6229428|1375 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ANY_CANDIDATE` ranking
|
||||
#### `ANY_CANDIDATE` oracle bounds
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|0 / 6392909 (0.000000%)|0 / 6344 (0.000000%)|1.000000|1.000000|1.000000|
|
||||
<div class="quality-summary quality-summary--oracle" markdown="1">
|
||||
|
||||
| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |
|
||||
|---|---:|---:|
|
||||
|Radixor|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Classification metrics</summary>
|
||||
<details class="quality-details" markdown="1"><summary>Oracle-bound pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Pair-relation metrics</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|n/a|n/a|n/a|n/a|n/a|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Raw pair counts</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ANY_CANDIDATE|6344|0|0|6392909|0 / 6392909|0 / 6344|
|
||||
| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |
|
||||
|---|---:|---:|
|
||||
|Radixor|0 / 6229428|0 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
#### `ALL_CANDIDATES` ranking
|
||||
|
||||
<div class="quality-table quality-table--compact" role="region" aria-label="Compact stemming-quality ranking; scroll horizontally for additional columns" tabindex="0" markdown="1">
|
||||
<div class="quality-summary" markdown="1">
|
||||
|
||||
| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.999970|389 / 6392909 (0.006085%)|0 / 6344 (0.000000%)|0.953240|0.970253|0.970653|
|
||||
| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |
|
||||
|---:|---|---:|---:|---:|
|
||||
|1|Radixor|1.000000|0.000000%|0.000000%|
|
||||
|
||||
</div>
|
||||
|
||||
@@ -325,7 +259,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.942225|1.000000|0.999939|0.999970|0.999939|0.000061|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|0.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -333,15 +267,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | F0.5 | F1 | F2 | Jaccard | Fowlkes–Mallows | MCC |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|0.953240|0.970253|0.987885|0.942225|0.970683|0.970653|
|
||||
|
||||
</details>
|
||||
|
||||
<details class="quality-details" markdown="1"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>
|
||||
|
||||
| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|n/a|n/a|n/a|n/a|n/a|
|
||||
|1|Radixor|ALL_CANDIDATES|1.000000|1.000000|1.000000|1.000000|1.000000|1.000000|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -349,7 +275,7 @@ This mode contains **5 result rows**, **3 evaluated stemmers**, and **3 output p
|
||||
|
||||
| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |
|
||||
|---:|---|---|---:|---:|---:|---:|---:|---:|
|
||||
|1|Radixor|ALL_CANDIDATES|6344|389|0|6392520|389 / 6392909|0 / 6344|
|
||||
|1|Radixor|ALL_CANDIDATES|6318|0|0|6229428|0 / 6229428|0 / 6318|
|
||||
|
||||
</details>
|
||||
|
||||
@@ -359,19 +285,19 @@ Alternative candidates are capability analyses, not replacements for the determi
|
||||
|
||||
| Stemmer | Under pairs repaired | Best-case over pairs avoided | All-candidate collisions added | Multi-candidate forms | Multi-candidate share | Maximum candidates | Total candidate assignments |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|
|
||||
|Radixor|149|195|194|89|2.487423%|3|3676|
|
||||
|Radixor|138|0|0|43|1.217441%|3|3578|
|
||||
|
||||
### Output Policies and Metric Definitions
|
||||
|
||||
`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.
|
||||
Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.
|
||||
|
||||
For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.
|
||||
For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.
|
||||
|
||||
- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.
|
||||
- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.
|
||||
- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.
|
||||
- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.
|
||||
- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.
|
||||
- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.
|
||||
- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.
|
||||
- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.
|
||||
- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.
|
||||
- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.
|
||||
@@ -380,16 +306,17 @@ For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs
|
||||
- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.
|
||||
- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.
|
||||
|
||||
Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.
|
||||
Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.
|
||||
|
||||
### Provenance
|
||||
|
||||
- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`
|
||||
- Source SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- Evaluation command: `./gradlew stemmingQuality`
|
||||
- Source SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Evaluation command: `./gradlew stemmingQuality --no-daemon`
|
||||
- Dictionary language: `YI`
|
||||
- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`
|
||||
- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`
|
||||
- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV
|
||||
- Model ID, version, and SHA-256: recorded in every CSV row
|
||||
- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)
|
||||
|
||||
<!-- STEMMING-QUALITY:END -->
|
||||
|
||||
@@ -2,27 +2,27 @@
|
||||
|
||||
The table below describes the Radixor resources used to build speed and quality corpora. `Total tokens` is the complete dictionary token count used by quality benchmarks. `Already-root tokens` counts fields where the token is already equal to the line root. `Changed tokens` is the speed workload before the minimum-size repeat rule.
|
||||
|
||||
| Language resource | Dictionary rows | Total tokens | Already-root tokens | Changed tokens | Speed timing tokens |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| `cs_cz` | 5,113 | 56,612 | 10,049 | 46,563 | 46,563 |
|
||||
| `da_dk` | 4,179 | 32,256 | 8,356 | 23,900 | 23,900 |
|
||||
| `de_de` | 39,315 | 213,440 | 73,799 | 139,641 | 139,641 |
|
||||
| `es_es` | 65,059 | 926,393 | 120,121 | 806,272 | 806,272 |
|
||||
| `fa_ir` | 69 | 3,770 | 138 | 3,632 | 5,000 |
|
||||
| `fi_fi` | 57,027 | 1,865,215 | 110,525 | 1,754,690 | 1,754,690 |
|
||||
| `fr_fr` | 59,240 | 474,110 | 108,141 | 365,969 | 365,969 |
|
||||
| `he_il` | 2,358 | 61,071 | 4,715 | 56,356 | 56,356 |
|
||||
| `hu_hu` | 19,406 | 935,713 | 38,775 | 896,938 | 896,938 |
|
||||
| `it_it` | 10,009 | 337,546 | 20,004 | 317,542 | 317,542 |
|
||||
| `nb_no` | 17,929 | 90,757 | 33,376 | 57,381 | 57,381 |
|
||||
| `nl_nl` | 4,992 | 31,466 | 9,981 | 21,485 | 21,485 |
|
||||
| `nn_no` | 4,688 | 19,651 | 6,089 | 13,562 | 13,562 |
|
||||
| `pl_pl` | 9,990 | 132,308 | 19,957 | 112,351 | 112,351 |
|
||||
| `pt_pt` | 4,001 | 215,490 | 8,002 | 207,488 | 207,488 |
|
||||
| `ru_ru` | 37,410 | 806,279 | 74,808 | 731,471 | 731,471 |
|
||||
| `sv_se` | 12,371 | 110,468 | 24,731 | 85,737 | 85,737 |
|
||||
| `uk_ua` | 1,493 | 15,737 | 2,985 | 12,752 | 12,752 |
|
||||
| `us_uk` | 396,939 | 1,004,374 | 793,874 | 210,500 | 210,500 |
|
||||
| `yi` | 802 | 4,300 | 1,524 | 2,776 | 5,000 |
|
||||
| Default model ID | Version | SHA-256 | Language | Dictionary rows | Total tokens | Already-root tokens | Changed tokens | Speed timing tokens |
|
||||
| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| `cs-cz-default` | `1.0.0` | `62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4` | `CS_CZ` | 5,113 | 56,612 | 10,049 | 46,563 | 46,563 |
|
||||
| `da-dk-default` | `1.0.0` | `3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90` | `DA_DK` | 4,179 | 32,256 | 8,356 | 23,900 | 23,900 |
|
||||
| `nl-nl-default` | `1.0.0` | `c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e` | `NL_NL` | 4,992 | 31,466 | 9,981 | 21,485 | 21,485 |
|
||||
| `us-uk-default` | `1.0.0` | `8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460` | `US_UK` | 396,939 | 1,004,374 | 793,874 | 210,500 | 210,500 |
|
||||
| `fi-fi-default` | `1.0.0` | `ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f` | `FI_FI` | 57,027 | 1,865,215 | 110,525 | 1,754,690 | 1,754,690 |
|
||||
| `fr-fr-default` | `1.0.0` | `a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9` | `FR_FR` | 59,240 | 474,110 | 108,141 | 365,969 | 365,969 |
|
||||
| `de-de-default` | `1.0.0` | `cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5` | `DE_DE` | 54,092 | 333,036 | 90,535 | 242,501 | 242,501 |
|
||||
| `he-il-default` | `1.0.0` | `9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256` | `HE_IL` | 2,358 | 61,071 | 4,715 | 56,356 | 56,356 |
|
||||
| `hu-hu-default` | `1.0.0` | `359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e` | `HU_HU` | 19,406 | 935,713 | 38,775 | 896,938 | 896,938 |
|
||||
| `it-it-default` | `1.0.0` | `5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12` | `IT_IT` | 10,009 | 337,546 | 20,004 | 317,542 | 317,542 |
|
||||
| `nb-no-default` | `1.0.0` | `f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9` | `NB_NO` | 17,929 | 90,757 | 33,376 | 57,381 | 57,381 |
|
||||
| `nn-no-default` | `1.0.0` | `900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398` | `NN_NO` | 4,688 | 19,651 | 6,089 | 13,562 | 13,562 |
|
||||
| `fa-ir-default` | `1.0.0` | `b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1` | `FA_IR` | 69 | 3,770 | 138 | 3,632 | 5,000 |
|
||||
| `pl-pl-unimorph` | `1.0.0` | `8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721` | `PL_PL` | 9,990 | 132,308 | 19,957 | 112,351 | 112,351 |
|
||||
| `pt-pt-default` | `1.0.0` | `7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d` | `PT_PT` | 4,001 | 215,490 | 8,002 | 207,488 | 207,488 |
|
||||
| `ru-ru-default` | `1.0.0` | `df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40` | `RU_RU` | 37,410 | 806,279 | 74,808 | 731,471 | 731,471 |
|
||||
| `es-es-default` | `1.0.0` | `7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721` | `ES_ES` | 65,059 | 926,393 | 120,121 | 806,272 | 806,272 |
|
||||
| `sv-se-default` | `1.0.0` | `d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c` | `SV_SE` | 12,371 | 110,468 | 24,731 | 85,737 | 85,737 |
|
||||
| `uk-ua-default` | `1.0.0` | `cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae` | `UK_UA` | 1,493 | 15,737 | 2,985 | 12,752 | 12,752 |
|
||||
| `yi-default` | `1.0.0` | `f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3` | `YI` | 802 | 4,300 | 1,524 | 2,776 | 5,000 |
|
||||
|
||||
Speed benchmarks process the complete changed-token dictionary sequence for the language. Only resources with fewer than 5,000 changed tokens are repeated to reach the minimum timing size; larger resources are not sampled or truncated.
|
||||
|
||||
@@ -6,16 +6,16 @@ This benchmark is the clearest demonstration of the Radixor quality/speed envelo
|
||||
|
||||
| Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 28.578 | 7.571 | 135.8 |
|
||||
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 26.612 | 9.227 | 126.4 |
|
||||
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 23.331 | 8.106 | 110.8 |
|
||||
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 22.362 | 1.957 | 106.2 |
|
||||
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 16.497 | 2.026 | 78.4 |
|
||||
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 16.035 | 0.986 | 76.2 |
|
||||
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 16.459 | 0.664 | 78.2 |
|
||||
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 19.566 | 0.758 | 92.9 |
|
||||
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 14.616 | 0.487 | 69.4 |
|
||||
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 18.093 | 3.147 | 86.0 |
|
||||
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 20.627 | 2.117 | 98.0 |
|
||||
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 21.713 | 2.104 | 103.2 |
|
||||
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 17.408 | 1.438 | 82.7 |
|
||||
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 16.946 | 1.531 | 80.5 |
|
||||
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 15.735 | 1.278 | 74.8 |
|
||||
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 14.714 | 1.089 | 69.9 |
|
||||
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 15.090 | 1.254 | 71.7 |
|
||||
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 13.773 | 1.071 | 65.4 |
|
||||
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 15.396 | 2.497 | 73.1 |
|
||||
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 16.970 | 2.847 | 80.6 |
|
||||
|
||||
## Column Meanings
|
||||
|
||||
|
||||
@@ -4,53 +4,85 @@ The values below are environment-specific and must not be read as universal perf
|
||||
|
||||
| Item | Value |
|
||||
| --- | --- |
|
||||
| Benchmark date | 2026-07-06 (Europe/Prague) |
|
||||
| Focused comparison command family | `./gradlew jmh -Pjmh.includes='.*StemmerComparisonBenchmark.*' --no-daemon` |
|
||||
| English coverage command | `./gradlew jmh -Pjmh.includes='.*EnglishRadixorDictionaryCoverageBenchmark.*' --no-daemon` |
|
||||
| Speed result reports | `build/reports/jmh/stemmer-comparison-2026-07-06.csv`, `build/reports/jmh/stemmer-comparison-2026-07-06.txt`, `build/reports/jmh/english-coverage-2026-07-06.csv`, and `build/reports/jmh/english-coverage-2026-07-06.txt` |
|
||||
| Accuracy result reports | `build/reports/jmh/stemmer-comparison-2026-07-06.csv`, `build/reports/jmh/english-coverage-2026-07-06.csv`, and deterministic Radixor exact-root accounting over the same bundled language corpora |
|
||||
| Final comparison JMH scope | Stemmer comparison benchmarks only; internal `FrequencyTrie*` microbenchmarks were not run |
|
||||
| Coverage JMH scope | English Radixor dictionary coverage benchmark only |
|
||||
| Benchmark date | 2026-07-23 (Europe/Prague) |
|
||||
| Corpus command | `./gradlew benchmarkCorpusReport --no-daemon` |
|
||||
| Exact-root accuracy command | Direct JMH execution of the four `*BenchmarkQuality` classes selected in `stemmer-accuracy-2026-07-23.txt`; timing scores are discarded |
|
||||
| Stemming-quality command | `./gradlew stemmingQuality --no-daemon` |
|
||||
| Published speed command | `tools/run-published-speed-benchmarks.sh 2026-07-23` |
|
||||
| Published speed run interval | 2026-07-23 12:58:50 to 15:15:43 Europe/Prague (2 h 16 min 53 s, including idle intervals and both JMH suites) |
|
||||
| Stabilization intervals | 120 s before the main speed matrix; 60 s between the main matrix and coverage-speed suite |
|
||||
| Corpus and command report | `build/reports/jmh/benchmark-corpora.csv` |
|
||||
| Exact-root reports | `build/reports/jmh/stemmer-accuracy-2026-07-23.csv` and `.txt` |
|
||||
| Speed reports | `build/reports/jmh/stemmer-speed-2026-07-23.csv` and `.txt` |
|
||||
| English coverage accuracy reports | `build/reports/jmh/english-coverage-accuracy-2026-07-23.csv` and `.txt` |
|
||||
| English coverage speed reports | `build/reports/jmh/english-coverage-speed-2026-07-23.csv` and `.txt` |
|
||||
| Stemming-quality reports | `build/reports/stemming-quality/stemming-quality.csv` and `.md` |
|
||||
| Environment report | `build/reports/jmh/performance-environment-2026-07-23.txt` |
|
||||
| Selected speed methods | `build/reports/jmh/published-speed-benchmarks-2026-07-23.txt` |
|
||||
| Comparison scope | Same-language methods used by the 20 language pages; `PolishPolimorfStemmerComparisonBenchmark`, all quality methods, the separate CISTEM gold-standard experiment, and internal trie microbenchmarks are excluded |
|
||||
| Model scope | Exactly the 20 IDs declared by `Language.defaultModelId()`; Polish uses `pl-pl-unimorph`, and `pl-pl-polimorf` is not measured |
|
||||
| Core base commit | `1f1b03c6a8d36a0918b92ebde698e5379a2a5946` |
|
||||
| Measured source state | `release@4.0.0-dirty`; exact tracked changes and untracked-source checksums are retained as `measured-source-2026-07-23.patch` and `measured-untracked-2026-07-23.sha256` |
|
||||
| JMH version | 1.37 |
|
||||
| Speed benchmark mode | Average time, `time/op` |
|
||||
| Score unit | `ns/op` |
|
||||
| Speed warmup | 3 iterations, 1 s each |
|
||||
| Speed measurement | 5 iterations, 1 s each |
|
||||
| Accuracy warmup | 3 JMH warmup iterations were applied by the Gradle invocation; timing scores from quality methods are not interpreted |
|
||||
| Accuracy measurement | 5 JMH measurement samples; documentation uses deterministic auxiliary counter ratios from the same report |
|
||||
| Fork count in generated report files | 1 |
|
||||
| Default fork policy for accuracy-only benchmark classes | `@Fork(0)` for future default runs because accuracy counters are deterministic and not interpreted as speed |
|
||||
| Thread count | 1 |
|
||||
| Score unit | `ns/op`; language pages additionally derive `ms/op` and `ns/token` |
|
||||
| Speed warmup | 5 iterations, 1 s each, independently in every fork |
|
||||
| Speed measurement | 10 iterations, 1 s each, independently in every fork |
|
||||
| Speed forks | 3 independent JVM forks |
|
||||
| Speed threads | 1 |
|
||||
| Speed fork heap | Fixed `-Xms6g -Xmx6g` |
|
||||
| Reported uncertainty | JMH `Score Error (99.9%)` over 30 measured samples |
|
||||
| Observed relative uncertainty | Main speed matrix: maximum 11.945%, with 6 of 102 rows above 10%; coverage-speed curve: maximum 16.775%, with 3 of 10 rows above 10%; no published row exceeded 20% |
|
||||
| Deterministic measurements | Corpus, patch-command distribution, exact-root counters, coverage accuracy, and pairwise stemming quality are evaluated without interpreting runtime scores; no warmup is required |
|
||||
| JVM reported by JMH | JDK 25.0.3, OpenJDK 64-Bit Server VM, 25.0.3+9 |
|
||||
| Java runtime | OpenJDK Runtime Environment, Red Hat build 25.0.3+9 |
|
||||
| JVM invoker | `/usr/lib/jvm/java-25-openjdk/bin/java` |
|
||||
| Operating system | Fedora Linux 44 (MATE-Compiz) |
|
||||
| Kernel | Linux 7.0.12-201.fc44.x86_64 |
|
||||
| Kernel | Linux 7.1.4-200.fc44.x86_64 |
|
||||
| Architecture | x86_64 |
|
||||
| CPU | AMD Ryzen 5 8600G w/ Radeon 760M Graphics |
|
||||
| Physical cores | 6 |
|
||||
| Logical CPUs | 12 |
|
||||
| Physical / logical CPUs | 6 / 12 |
|
||||
| CPU frequency policy | `amd-pstate-epp`; governor `performance` on every logical CPU; EPP `performance`; boost enabled |
|
||||
| CPU affinity | Scheduler default; no explicit pinning |
|
||||
| Installed memory | 60 GiB reported by the operating system |
|
||||
| Pre-run idle state | Load average 0.25 / 0.36 / 0.71 after the 120 s idle interval; CPU Tctl 40.2 degrees Celsius; swap unused |
|
||||
| End-of-run state | Load average 1.16 / 1.28 / 1.32; CPU Tctl 60.5 degrees Celsius |
|
||||
| Power and idle policy | Developer workstation on stable power; screensaver, suspend, and hibernation disabled |
|
||||
| Concurrent project work | None during the published speed and coverage-speed run |
|
||||
|
||||
The workstation is not a hard real-time system. Normal kernel and desktop background activity was not removed, so the three independent forks and the published 99.9% error interval remain essential parts of result interpretation. Initial/final load and temperature sensor readings are stored in the environment report.
|
||||
|
||||
## Contracted Trie Baseline
|
||||
|
||||
All Radixor rows in the refreshed benchmark tables use contracted compiled patch tries. During compilation, a subtree whose reachable entries all resolve to the same preferred patch command is represented as an accepting leaf. Runtime lookup can therefore stop as soon as that leaf is reached, which reduces depth in uniform regions while preserving the preferred result used by `get()`.
|
||||
All Radixor rows use contracted compiled patch tries. During compilation, a subtree whose reachable entries all resolve to the same preferred patch command is represented as an accepting leaf. Runtime lookup can therefore stop as soon as that leaf is reached while preserving the preferred result used by `get()`.
|
||||
|
||||
## Model And Source Identity
|
||||
|
||||
`benchmark-corpora.csv` records the model ID, independent artifact version, and descriptor SHA-256 for every language. Every stemming-quality CSV row repeats the same three fields. The performance environment report additionally records checksums of the executable JMH JAR, runtime classpath manifest, corpus report, quality report, measured source patch, and untracked-source manifest.
|
||||
|
||||
The JMH runtime classpath contains the optional model artifact because it is a separately testable project dependency. It is not selected by any published benchmark. The selected-method manifest rejects `PolishPolimorf`, and the corpus/quality publication validators reject any non-default Polish model.
|
||||
|
||||
## Report Files
|
||||
|
||||
Generated local report files for this benchmark update:
|
||||
|
||||
- `build/reports/jmh/stemmer-comparison-2026-07-06.csv`
|
||||
- `build/reports/jmh/stemmer-comparison-2026-07-06.txt`
|
||||
- `build/reports/jmh/english-coverage-2026-07-06.csv`
|
||||
- `build/reports/jmh/english-coverage-2026-07-06.txt`
|
||||
- `build/reports/jmh/benchmark-corpora.csv`
|
||||
- `build/reports/jmh/stemmer-accuracy-2026-07-23.csv`
|
||||
- `build/reports/jmh/stemmer-accuracy-2026-07-23.txt`
|
||||
- `build/reports/jmh/stemmer-speed-2026-07-23.csv`
|
||||
- `build/reports/jmh/stemmer-speed-2026-07-23.txt`
|
||||
- `build/reports/jmh/english-coverage-accuracy-2026-07-23.csv`
|
||||
- `build/reports/jmh/english-coverage-accuracy-2026-07-23.txt`
|
||||
- `build/reports/jmh/english-coverage-speed-2026-07-23.csv`
|
||||
- `build/reports/jmh/english-coverage-speed-2026-07-23.txt`
|
||||
- `build/reports/jmh/performance-environment-2026-07-23.txt`
|
||||
- `build/reports/stemming-quality/stemming-quality.csv`
|
||||
- `build/reports/stemming-quality/stemming-quality.md`
|
||||
- `build/reports/stemming-quality/metric-correlations-pearson.csv`
|
||||
- `build/reports/stemming-quality/metric-correlations-spearman.csv`
|
||||
|
||||
JMH TXT and CSV reports are still published as benchmark artifacts. They are not converted into a Porter speed badge.
|
||||
The versioned documentation snapshot under `docs/benchmarks/data/` preserves the complete stemming-quality matrix. Machine-specific JMH reports remain build artifacts.
|
||||
|
||||
## Published Metrics
|
||||
|
||||
The historical English Radixor versus Porter performance badge is no longer generated. `tools/generate-pages-badges.py` now produces only coverage and mutation badge endpoint JSON files:
|
||||
|
||||
- `coverage-badge.json`
|
||||
- `pitest-badge.json`
|
||||
|
||||
The README therefore no longer presents a single Porter speed ratio. Benchmark interpretation now uses both speed and quality, because a narrow or aggressive stemmer can be fast while disagreeing with the dictionary root much more often than Radixor.
|
||||
The historical English Radixor versus Porter performance badge is retired. `tools/generate-pages-badges.py` produces only coverage and mutation badge endpoint JSON files. Benchmark interpretation uses both speed and quality because a narrow or aggressive stemmer can be fast while disagreeing with the dictionary root much more often than Radixor.
|
||||
|
||||
@@ -8,18 +8,24 @@ The authoritative Radixor language universe is the reconciliation of registered
|
||||
|
||||
Model identity is part of the candidate identity. Default Polish means `pl-pl-unimorph`; optional PoliMorf means `pl-pl-polimorf`. Results for those inputs must not be combined or relabeled, and historical snapshots cannot acquire a newer model identity retroactively.
|
||||
|
||||
Within one language and dictionary mode, every adapter receives the same original included forms. Exact duplicates are removed only within one dictionary row. Identical surface forms in different rows remain distinct entries. Candidate strings use exact `String.equals`, with no evaluation-only lowercasing, normalization, accent removal, or gold-label-aware selection. Adapter preprocessing and lifecycle match the JMH comparison path.
|
||||
Within one language and dictionary mode, every adapter receives the same original included forms. A distinct surface string is one evaluated item even when it occurs in several rows; those occurrences become multiple gold-group memberships. Candidate strings use exact `String.equals`, with no evaluation-only lowercasing, normalization, accent removal, or gold-label-aware selection. Adapter preprocessing and lifecycle match the JMH comparison path.
|
||||
|
||||
## Gold-standard pairs
|
||||
|
||||
Every usable dictionary row is a gold-standard equivalence group. An unordered pair from the same row is positive; a pair from different rows is negative. For group size `n`, `C2(n) = n * (n - 1) / 2`.
|
||||
Every usable dictionary row contributes one gold-standard group. The groups form an overlapping cover rather than an exclusive partition: one surface form may belong to several groups. For two distinct forms `u` and `v` with membership sets `G(u)` and `G(v)`:
|
||||
|
||||
- `TP = underPossiblePairs - underErrorPairs`: same-group pairs correctly related.
|
||||
- `FN = underErrorPairs`: same-group pairs incorrectly separated.
|
||||
- `FP = overErrorPairs`: different-group pairs incorrectly related.
|
||||
- `TN = overPossiblePairs - overErrorPairs`: different-group pairs correctly separated.
|
||||
```text
|
||||
goldRelated(u, v) = (G(u) intersection G(v) is not empty)
|
||||
```
|
||||
|
||||
Under-stemming is the false-negative relation among same-group pairs. Over-stemming is the false-positive relation among different-group pairs. Their percentages use different denominators and must not be added or averaged without an explicitly defined composite.
|
||||
A pair is counted once even if it shares several groups. Gold-negative pairs have disjoint membership sets. Thus:
|
||||
|
||||
- `TP = underPossiblePairs - underErrorPairs`: gold-related pairs correctly related.
|
||||
- `FN = underErrorPairs`: gold-related pairs incorrectly separated.
|
||||
- `FP = overErrorPairs`: gold-negative pairs incorrectly related.
|
||||
- `TN = overPossiblePairs - overErrorPairs`: gold-negative pairs correctly separated.
|
||||
|
||||
Under-stemming is Paice's Understemming Index (UI), the false-negative rate among gold-related pairs. Over-stemming is Paice's Overstemming Index (OI), the false-positive rate among gold-negative pairs. The original Paice formulation assumes disjoint lemma groups; this evaluator explicitly generalizes the pair relation to overlapping membership. Their percentages use different denominators and must not be added or averaged without an explicitly defined composite.
|
||||
|
||||
## Dictionary-processing modes
|
||||
|
||||
@@ -30,9 +36,9 @@ Under-stemming is the false-negative relation among same-group pairs. Over-stemm
|
||||
|
||||
`PRIMARY_OUTPUT` uses the adapter's deterministic primary stem. It defines a strict predicted partition and is the principal direct comparison between implementations.
|
||||
|
||||
`ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound. Same-group pairs succeed when candidate sets intersect. Different-group pairs avoid an error whenever a non-colliding candidate selection exists. Selection may differ between pairs, so this policy is not deterministic runtime behaviour and may not correspond to one globally realizable assignment.
|
||||
`ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound. Gold-related pairs succeed when candidate sets intersect. Gold-negative pairs avoid an error whenever a non-colliding candidate selection exists. Selection may differ between pairs, so this policy is not deterministic runtime behaviour and may not correspond to one globally realizable assignment. Because its positive and negative decisions use different oracle conditions, it does not define one confusion matrix; TP/FP/FN/TN and all confusion-derived scores are therefore `n/a`. Its separate under/over error counts and denominators remain defined.
|
||||
|
||||
`ALL_CANDIDATES` treats every returned candidate as active. Two forms are related when their candidate sets intersect. Alternatives can recover same-group relationships while introducing cross-group collisions. This overlapping relation need not be transitive or form a partition.
|
||||
`ALL_CANDIDATES` treats every returned candidate as active. Two forms are related when their candidate sets intersect. Alternatives can recover gold-positive relationships while introducing gold-negative collisions. This overlapping relation need not be transitive or form a partition.
|
||||
|
||||
Candidate-aware policies are reported as capability analyses. They are not mixed into the principal `PRIMARY_OUTPUT` ranking.
|
||||
|
||||
@@ -42,11 +48,11 @@ Undefined denominators produce `n/a`, never zero, `NaN`, or infinity. Metrics ar
|
||||
|
||||
| Metric | Formula | Range and interpretation | Sensitivity and applicability |
|
||||
| --- | --- | --- | --- |
|
||||
| Under-stemming rate | `FN / (TP + FN)` | `[0, 1]`; lower is better. False-negative rate over same-group pairs. | Sensitive to splitting large gold groups. All policies. |
|
||||
| Over-stemming rate | `FP / (TN + FP)` | `[0, 1]`; lower is better. False-positive rate over different-group pairs. | The denominator is usually very large. All policies. |
|
||||
| Precision | `TP / (TP + FP)` | `[0, 1]`; higher is better. Fraction of predicted relations that are gold-positive. | Penalizes over-stemming. All policies, with oracle-assisted interpretation for `ANY_CANDIDATE`. |
|
||||
| Recall | `TP / (TP + FN)` | `[0, 1]`; higher is better. Fraction of gold-positive pairs recovered. | Equivalent to one minus the under-stemming rate. All policies. |
|
||||
| Specificity | `TN / (TN + FP)` | `[0, 1]`; higher is better. Fraction of negative pairs separated. | Sensitive to cross-group collisions. All policies. |
|
||||
| Under-stemming rate | `FN / (TP + FN)` | `[0, 1]`; lower is better. False-negative rate over gold-related pairs. | Sensitive to splitting large gold groups. All policies. |
|
||||
| Over-stemming rate | `FP / (TN + FP)` | `[0, 1]`; lower is better. False-positive rate over gold-negative pairs. | The denominator is usually very large. All policies. |
|
||||
| Precision | `TP / (TP + FP)` | `[0, 1]`; higher is better. Fraction of predicted relations that are gold-positive. | Penalizes over-stemming. `PRIMARY_OUTPUT` and `ALL_CANDIDATES`. |
|
||||
| Recall | `TP / (TP + FN)` | `[0, 1]`; higher is better. Fraction of gold-positive pairs recovered. | Equivalent to one minus the under-stemming rate. `PRIMARY_OUTPUT` and `ALL_CANDIDATES`. |
|
||||
| Specificity | `TN / (TN + FP)` | `[0, 1]`; higher is better. Fraction of negative pairs separated. | Sensitive to false conflations. `PRIMARY_OUTPUT` and `ALL_CANDIDATES`. |
|
||||
| Balanced accuracy | `(recall + specificity) / 2` | `[0, 1]`; higher is better. Equal weight for positive and negative classes. | Primary navigation metric; less dominated by TN than ordinary accuracy, but not uniquely authoritative. |
|
||||
| Pairwise accuracy | `(TP + TN) / (TP + TN + FP + FN)` | `[0, 1]`; higher is better. | Can be dominated by the very large TN class and is not the default ranking metric. |
|
||||
| Pairwise error rate | `(FP + FN) / (TP + TN + FP + FN)` | `[0, 1]`; lower is better. | Also sensitive to the number of negative pairs. |
|
||||
@@ -59,17 +65,9 @@ Undefined denominators produce `n/a`, never zero, `NaN`, or infinity. Metrics ar
|
||||
|
||||
The general F-beta formula is `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`.
|
||||
|
||||
## Partition-only metrics
|
||||
## Inapplicable partition metrics
|
||||
|
||||
These metrics apply only to `PRIMARY_OUTPUT`. Candidate relations are not forced into artificial partitions.
|
||||
|
||||
- Adjusted Rand Index is the Rand agreement corrected for agreement expected from the gold/predicted contingency-table marginals. Its usual range is `[-1, 1]`, with `1` indicating identical partitions.
|
||||
- Homogeneity is `1 - H(gold | predicted) / H(gold)`, in `[0, 1]`; each predicted cluster ideally contains one gold group.
|
||||
- Completeness is `1 - H(predicted | gold) / H(predicted)`, in `[0, 1]`; each gold group ideally maps to one predicted cluster.
|
||||
- V-measure is the harmonic mean of homogeneity and completeness, in `[0, 1]`.
|
||||
- Normalized mutual information uses arithmetic-mean entropy normalization: `MI / ((H(gold) + H(predicted)) / 2)`, in `[0, 1]` under this implementation.
|
||||
|
||||
Entropy zero cases follow the evaluator's explicit perfect/undefined conventions. Language tables render inapplicable candidate-policy values as `n/a`.
|
||||
Standard Adjusted Rand Index, homogeneity, completeness, V-measure, and normalized mutual information are not calculated. Their ordinary contingency-table definitions require every item to have one exclusive gold label. Assigning an arbitrary single label or duplicating a multi-membership form would change the scientific question and reintroduce the counting defect this methodology avoids. A future overlapping-clustering index would require a separately specified random model and interpretation; it must not be labelled as ordinary ARI or NMI.
|
||||
|
||||
## Aggregation and ranking
|
||||
|
||||
@@ -81,4 +79,4 @@ Multiple metrics and Pearson/Spearman correlation datasets are published because
|
||||
|
||||
## Limitations
|
||||
|
||||
Dictionary groups encode the available annotation, not every linguistic distinction. Homographs may occur in different groups, singleton rows contribute no under-stemming pair, and group size affects pair counts. `ANY_CANDIDATE` is optimistic; `ALL_CANDIDATES` measures an overlapping graph; neither is a deterministic global assignment. Results characterize the tested versions, adapters, dictionaries, and preprocessing, not every deployment or domain.
|
||||
Dictionary groups encode the available annotation, not every linguistic distinction. Homographs and polyfunctional forms may have several memberships, singleton rows contribute no relation by themselves, and group size affects pair counts. `ANY_CANDIDATE` is optimistic; `ALL_CANDIDATES` measures an overlapping graph; neither is a deterministic global assignment. Results characterize the tested versions, adapters, dictionaries, and preprocessing, not every deployment or domain.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
The stemmer comparison suite measures Radixor and Java stemmers on the same language and deterministic Radixor model dictionary-derived data. Published Radixor rows in this refresh use contracted compiled patch tries, where uniform preferred-command subtrees are collapsed into accepting leaves before the trie is frozen for lookup. For each language, the registered default model resource stores the expected root as the first tab-separated field on a line and its surface forms on the same line. Every single-token field on that line can therefore be paired with the same expected root.
|
||||
|
||||
Published stemmer comparison results must come only from benchmark classes matching `.*StemmerComparisonBenchmark.*`. Internal `FrequencyTrie*` microbenchmarks are not part of those results.
|
||||
Published speed results come only from the exact method selection retained in `published-speed-benchmarks-2026-07-23.txt`. Internal `FrequencyTrie*` microbenchmarks, quality methods, the CISTEM gold-standard experiment, and the optional `PolishPolimorfStemmerComparisonBenchmark` are not part of those results.
|
||||
|
||||
## Benchmark Passes
|
||||
|
||||
@@ -13,6 +13,8 @@ There are two distinct benchmark passes:
|
||||
|
||||
Timing corpora are generated once per JMH JVM and kept in memory as shared `{token, expectedRoot}` arrays. Corpus construction, dictionary loading, trie loading, table loading, and analyzer construction are setup work and are not included in measured benchmark methods.
|
||||
|
||||
The deterministic and timed workloads are executed separately. Corpus statistics, patch-command counts, exact-root counters, coverage accuracy, and pairwise quality do not use or interpret warmup or runtime scores. Published speed and coverage-speed methods use three independent forks, five one-second warmup iterations and ten one-second measurement iterations per fork, one benchmark thread, and a fixed 6 GiB heap.
|
||||
|
||||
Performance is interpreted as average time per input token:
|
||||
|
||||
```text
|
||||
@@ -37,10 +39,10 @@ For right-to-left Radixor languages, patch application uses the traversal direct
|
||||
|
||||
## Quality Metric
|
||||
|
||||
The quality pass reports exact-root agreement against the expected root from the Radixor dictionary line. It writes to the normal JMH report files:
|
||||
The quality pass reports exact-root agreement against the expected root from the default-model dictionary line. External-stemmer counters are written to:
|
||||
|
||||
- `build/reports/jmh/jmh-results.csv`
|
||||
- `build/reports/jmh/jmh-results.txt`
|
||||
- `build/reports/jmh/stemmer-accuracy-2026-07-23.csv`
|
||||
- `build/reports/jmh/stemmer-accuracy-2026-07-23.txt`
|
||||
|
||||
Accuracy is computed from standard JMH secondary rows:
|
||||
|
||||
@@ -54,7 +56,7 @@ rootPreservedPercent = rootPreservedMatches / rootEvaluatedTokens * 100
|
||||
|
||||
Morfologik can emit multiple terms for one input token. The quality benchmark uses the first emitted term for exact-root accounting when no ranking weight is exposed. Throughput benchmarks for Morfologik TokenFilter paths consume all emitted terms.
|
||||
|
||||
Quality reports use JMH auxiliary counter rows. Exact-root accounting is deterministic for a fixed corpus and stemmer, so repeated measurement samples duplicate the same counters; documentation uses the counter ratios and does not interpret quality benchmark timing scores.
|
||||
External-stemmer quality reports use JMH auxiliary counter rows from one deterministic evaluation. Radixor exact-root counts are computed directly while the default-model corpus and preferred patch commands are audited, so all 20 default models have the same coverage even where no older JMH quality adapter existed. Documentation uses counter ratios and does not interpret quality benchmark timing scores.
|
||||
|
||||
Pairwise over-stemming, under-stemming, candidate-aware policies, balanced accuracy, and partition comparison are a separate analytical evaluation. See [Linguistic Quality Methodology](linguistic-quality.md); exact-root accuracy must not be interpreted as the complement of pairwise under-stemming.
|
||||
Pairwise over-stemming, under-stemming, candidate-aware policies, and relation metrics are a separate analytical evaluation. See [Linguistic Quality Methodology](linguistic-quality.md); exact-root accuracy must not be interpreted as the complement of pairwise under-stemming.
|
||||
Default rows use `Language.defaultModelId()`. Optional variants require a separate model field; `pl-pl-unimorph` and `pl-pl-polimorf` must never share an ambiguous Polish label. The benchmark runtime receives each resource exactly once from its individual model JAR through direct JMH runtime dependencies. See [Model Selection and Loading](../../model-selection-and-loading.md).
|
||||
|
||||
@@ -4,14 +4,12 @@
|
||||
|
||||
- Machine-readable CSV: [stemming-quality.csv](../data/stemming-quality.csv)
|
||||
- SHA-256 record: [stemming-quality.sha256](../data/stemming-quality.sha256)
|
||||
- SHA-256: `5a93a6ab60e46489737cd649eb1ac48182114b9038f7f20195ab9d1c1fc0dd28`
|
||||
- SHA-256: `edf16b07be8a535943ddf37caeb8807755c95e9e1fb13244145f28be74b491d8`
|
||||
- Complete scenarios: 308
|
||||
- Authoritative language universe: 20 languages
|
||||
- Language-page scenarios: 302 across 19 existing benchmark pages
|
||||
- Language-page scenarios: 308 across 20 benchmark pages
|
||||
|
||||
The six remaining scenarios are the three Radixor policies in two modes for `HE_IL`. Hebrew is present in the complete result snapshot but has no existing language benchmark page.
|
||||
|
||||
The CSV contains raw TP, FP, FN, and TN counts; raw over/under numerators and denominators; candidate statistics; relation metrics; and partition-only metrics. Documentation is regenerated from this file rather than manually transcribed.
|
||||
The CSV contains the model ID, independent model version, descriptor SHA-256, raw pair counts, raw over/under numerators and denominators, candidate statistics, and relation metrics. Reserved partition-metric columns remain empty because the gold standard is an overlapping cover. Documentation is regenerated from this file rather than manually transcribed. Publication fails when any row uses a model other than the language's registered default.
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -24,6 +22,10 @@ The CSV contains raw TP, FP, FN, and TN counts; raw over/under numerators and de
|
||||
mkdocs build --strict --config-file build/mkdocs/mkdocs.yml
|
||||
```
|
||||
|
||||
For an immediate local preview, `mkdocs serve` works directly from the repository root. The checked-in
|
||||
model catalog makes that source tree complete. After changing model metadata or model bytes, refresh it
|
||||
with `./gradlew publishModelCatalogDocumentation`; verification rejects a stale checked-in catalog.
|
||||
|
||||
`stemmingQuality` performs the expensive complete evaluation and is intentionally not attached to `test` or `check`. It prepares JMH third-party dependencies automatically and writes:
|
||||
|
||||
- `build/reports/stemming-quality/stemming-quality.csv`
|
||||
@@ -57,19 +59,20 @@ The Pages workflow publishes that staged documentation together with Javadoc, JU
|
||||
|
||||
## Performance benchmark reproduction
|
||||
|
||||
The JMH comparison command family is:
|
||||
The current speed and coverage-speed command is:
|
||||
|
||||
```bash
|
||||
./gradlew jmh -Pjmh.includes='.*StemmerComparisonBenchmark.*' --no-daemon
|
||||
./gradlew writeJmhRuntimeClasspath --no-daemon
|
||||
tools/run-published-speed-benchmarks.sh 2026-07-23
|
||||
```
|
||||
|
||||
The exact JMH configuration, hardware, operating system, and JDK captured for the published performance tables are listed in [Environment and reports](environment.md). Quality and performance reports are separate datasets and are not combined into an undocumented scalar.
|
||||
The runner refuses to start unless every CPU uses the `performance` governor, materializes the exact selected benchmark list, rejects quality/Polimorf/gold-standard methods, and requires the Hebrew speed path. It records hardware, JVM, source-state, JAR, classpath, corpus, quality, load, temperature, and governor provenance before running. The exact JMH configuration is listed in [Environment and reports](environment.md). Quality and performance reports are separate datasets and are not combined into an undocumented scalar.
|
||||
|
||||
## Recorded and unavailable provenance
|
||||
|
||||
The performance documentation records its 2026-07-06 environment, JDK 25.0.3, operating system, and hardware. The quality CSV records the evaluated identifiers and counts but does not embed the Radixor Git revision, generation date, JDK, operating system, model ID, dictionary content hash, or immutable upstream revisions for every downloaded source. These fields are explicitly unavailable for this historical snapshot and are not reconstructed from filesystem timestamps. In particular, the snapshot predates the optional PoliMorf integration and must not be relabeled as `pl-pl-polimorf`.
|
||||
The performance documentation records its 2026-07-23 environment, JDK 25.0.3, operating system, hardware, base revision, exact dirty patch, untracked-source checksums, executable JMH JAR checksum, and model descriptor checksums. The quality CSV embeds model identity and checksum in every row; run date, core source state, JVM, OS, and hardware are shared provenance on the environment page.
|
||||
|
||||
Dependency versions that are reproducible from repository configuration include Apache Lucene 10.5.0, Morfologik 2.1.9, the Ukrainian dictionary artifact 4.9.1, and JMH 1.37. Other upstream branches or downloaded dictionary revisions should be pinned and embedded in a future result schema.
|
||||
Exact immutable upstream revisions were not recorded for every legacy UniMorph import. That limitation remains explicit in model descriptors and cannot be repaired from filesystem timestamps. Dependency versions reproducible from repository configuration include Apache Lucene 10.5.0, Morfologik 2.1.9, the Ukrainian dictionary artifact 4.9.1, and JMH 1.37.
|
||||
|
||||
## Correlation and audit data
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ The JMH adapter registry is authoritative for evaluated implementations and lang
|
||||
|
||||
| Family or implementation | Upstream / attribution | Tested version or revision | Evaluated scope | Output capability and adapter behaviour | Interpretation notes |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Radixor | Egothor / Radixor project | Current repository revision; exact revision was not embedded in the quality CSV | All 20 reconciled default model languages; 19 have benchmark pages | Deterministic preferred patch via `get`; ranked distinct alternatives via `getAll`; primary is always included | Model-dictionary-derived compiled patch trie. Default rows use each language's stable default model ID. |
|
||||
| Radixor | Egothor / Radixor project | Base commit and measured working-tree state recorded on the environment page | All 20 reconciled default model languages; all 20 have benchmark pages | Deterministic preferred patch via `get`; ranked distinct alternatives via `getAll`; primary is always included | Model-dictionary-derived compiled patch trie. Default rows use each language's stable default model ID. |
|
||||
| Apache Lucene language stem filters | Apache Lucene project | 10.5.0 | Adapter-declared language-specific subsets | TokenFilter lifecycle and language normalization match JMH; normally single-output | Light, minimal, possessive, and language stem filters deliberately implement different scopes. Narrow scope is not a defect. |
|
||||
| Apache Lucene SnowballFilter | Apache Lucene project using Snowball algorithms | Lucene 10.5.0 | Snowball-supported subset of Radixor languages | Single primary token emitted through the Lucene TokenFilter path | Includes TokenStream overhead and required normalization. |
|
||||
| Official Snowball Java | Snowball project | Repository preparation downloads the configured upstream Java distribution; an immutable revision was not recorded in the quality CSV | Same-language adapter subset | Direct generated Java API; single output | Rule-based suffix algorithms provide broad baselines rather than dictionary-root guarantees. |
|
||||
|
||||
3
docs/builds.md
Normal file
3
docs/builds.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Historical Builds
|
||||
|
||||
The Pages publication workflow replaces this local placeholder with the retained build index.
|
||||
25
docs/stemmer-model-catalog.md
Normal file
25
docs/stemmer-model-catalog.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Published Stemmer Model Catalog
|
||||
|
||||
| Model ID | Language | Default | Coordinates | Version | Source | Repository | Source version | Revision | Revision status | License | Attribution | SHA-256 | Bytes |
|
||||
|---|---|---:|---|---:|---|---|---|---|---|---|---|---|---:|
|
||||
| cs-cz-default | CS_CZ | true | org.egothor:radixor-model-cs-cz-default | 1.0.0 | UniMorph | https://github.com/unimorph/ces | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Witold Kieraś is credited for the separate MorfFlex-CZ conversion | 62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4 | 142365 |
|
||||
| da-dk-default | DA_DK | true | org.egothor:radixor-model-da-dk-default | 1.0.0 | UniMorph | https://github.com/unimorph/dan | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90 | 73030 |
|
||||
| de-de-default | DE_DE | true | org.egothor:radixor-model-de-de-default | 1.0.0 | UniMorph | https://github.com/unimorph/deu | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5 | 838450 |
|
||||
| es-es-default | ES_ES | true | org.egothor:radixor-model-es-es-default | 1.0.0 | UniMorph | https://github.com/unimorph/spa | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | 7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721 | 2269280 |
|
||||
| fa-ir-default | FA_IR | true | org.egothor:radixor-model-fa-ir-default | 1.0.0 | UniMorph | https://github.com/unimorph/fas | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1 | 8934 |
|
||||
| fi-fi-default | FI_FI | true | org.egothor:radixor-model-fi-fi-default | 1.0.0 | UniMorph | https://github.com/unimorph/fin | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f | 4867450 |
|
||||
| fr-fr-default | FR_FR | true | org.egothor:radixor-model-fr-fr-default | 1.0.0 | UniMorph | https://github.com/unimorph/fra | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9 | 1117956 |
|
||||
| he-il-default | HE_IL | true | org.egothor:radixor-model-he-il-default | 1.0.0 | UniMorph | https://github.com/unimorph/heb | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Omer Goldman (annotator); Wiktionary contributors | 9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256 | 179776 |
|
||||
| hu-hu-default | HU_HU | true | org.egothor:radixor-model-hu-hu-default | 1.0.0 | UniMorph | https://github.com/unimorph/hun | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Christo Kirov, Ryan Cotterell, and Khuyagbaatar Batsuren (conversion); Judit Ács and Gábor Bella (validation); English Wiktionary contributors | 359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e | 2346297 |
|
||||
| it-it-default | IT_IT | true | org.egothor:radixor-model-it-it-default | 1.0.0 | UniMorph | https://github.com/unimorph/ita | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12 | 841105 |
|
||||
| nb-no-default | NB_NO | true | org.egothor:radixor-model-nb-no-default | 1.0.0 | UniMorph | https://github.com/unimorph/nob | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9 | 205645 |
|
||||
| nl-nl-default | NL_NL | true | org.egothor:radixor-model-nl-nl-default | 1.0.0 | UniMorph | https://github.com/unimorph/nld | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e | 70600 |
|
||||
| nn-no-default | NN_NO | true | org.egothor:radixor-model-nn-no-default | 1.0.0 | UniMorph | https://github.com/unimorph/nno | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398 | 51244 |
|
||||
| pl-pl-polimorf | PL_PL | false | org.egothor:radixor-model-pl-pl-polimorf | 1.0.0 | PoliMorf 2.1 | https://github.com/morfologik/morfologik-stemming | 2.1 | 6e63b53 | recorded | BSD-2-Clause | Copyright (c) 2016, Marcin Miłkowski | 4fe4bf5e6c22c1beea5b3d57f1ce4c9ea5aac1ed8ab24c616fb06df745e40d15 | 12624997 |
|
||||
| pl-pl-unimorph | PL_PL | true | org.egothor:radixor-model-pl-pl-unimorph | 1.0.0 | UniMorph | https://github.com/unimorph/pol | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; SGJP authors Marcin Woliński, Zygmunt Saloni, Robert Wołosz, Włodzimierz Gruszczyński, Danuta Skowrońska, and Zbigniew Bronk; Witold Kieraś (conversion); Wiktionary contributors | 8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721 | 334951 |
|
||||
| pt-pt-default | PT_PT | true | org.egothor:radixor-model-pt-pt-default | 1.0.0 | UniMorph | https://github.com/unimorph/por | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d | 509965 |
|
||||
| ru-ru-default | RU_RU | true | org.egothor:radixor-model-ru-ru-default | 1.0.0 | UniMorph | https://github.com/unimorph/rus | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40 | 2414507 |
|
||||
| sv-se-default | SV_SE | true | org.egothor:radixor-model-sv-se-default | 1.0.0 | UniMorph | https://github.com/unimorph/swe | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and English Wiktionary contributors | d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c | 256300 |
|
||||
| uk-ua-default | UK_UA | true | org.egothor:radixor-model-uk-ua-default | 1.0.0 | UniMorph | https://github.com/unimorph/ukr | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph; Witold Kieraś and Maria Shvedova are credited for the separate VESUM conversion; Wiktionary contributors | cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae | 47300 |
|
||||
| us-uk-default | US_UK | true | org.egothor:radixor-model-us-uk-default | 1.0.0 | UniMorph | https://github.com/unimorph/eng | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph and Wikipedia contributors | 8c79122993499e437ea8b54b620832dca29019298f281c1f3132f4d1be885460 | 2713666 |
|
||||
| yi-default | YI | true | org.egothor:radixor-model-yi-default | 1.0.0 | UniMorph | https://github.com/unimorph/yid | not-recorded-in-legacy-import | not-recorded-in-legacy-import | not-recorded-in-legacy-import | CC-BY-SA-3.0 | UniMorph | f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3 | 12222 |
|
||||
@@ -189,4 +189,8 @@ Local validation for PoliMorf 1.0.0 is:
|
||||
|
||||
## Documentation and troubleshooting
|
||||
|
||||
`prepareMkDocsSource` generates the catalog only at `build/mkdocs-source/stemmer-model-catalog.md`; generated Markdown and rendered site content are not tracked. For runtime failures, dependency inspection, ClassLoader isolation, and fat-JAR guidance, see [Model Selection and Loading](model-selection-and-loading.md#troubleshooting).
|
||||
`publishModelCatalogDocumentation` updates the checked-in catalog used by a direct local `mkdocs serve`.
|
||||
`prepareMkDocsSource` independently regenerates the same catalog under `build/mkdocs-source/` for the
|
||||
publication workflow, and `verifyModelCatalogDocumentation` fails when the two copies differ. Rendered
|
||||
site content remains untracked. For runtime failures, dependency inspection, ClassLoader isolation, and
|
||||
fat-JAR guidance, see [Model Selection and Loading](model-selection-and-loading.md#troubleshooting).
|
||||
|
||||
@@ -10,13 +10,13 @@ The authoritative Radixor universe is the validated one-to-one reconciliation of
|
||||
|
||||
Default Polish evaluation is therefore `Radixor` with model `pl-pl-unimorph`. A future PoliMorf evaluation is a distinct `Radixor` / `pl-pl-polimorf` row. Evaluation classpaths receive individual models through direct non-production Gradle dependencies; ordinary applications inherit none of them from the core.
|
||||
|
||||
Complete PoliMorf trie construction and deterministic stemming smoke fixtures are runtime-verified separately. That functional verification is not a linguistic-quality measurement and does not justify rewriting the historical quality snapshot.
|
||||
Complete PoliMorf trie construction and deterministic stemming smoke fixtures are runtime-verified separately. That functional verification is not a linguistic-quality measurement and does not enter the current default-model quality snapshot.
|
||||
|
||||
The expected matrix is constructed before evaluation from stemmer, language, dictionary mode, and supported output policy. Generation fails on missing, duplicate, unexpected, or stale keys.
|
||||
|
||||
## Dictionary groups and modes
|
||||
|
||||
Every usable parsed row is one gold-standard group. Exact duplicate strings are removed only within that row; identical forms in different rows remain distinct. `ALL_WORDS` preserves every valid form. `LOWERCASE_GROUPS_ONLY` excludes a complete group containing an uppercase or titlecase Unicode code point. Retained words are not lowercased or normalized by the evaluator.
|
||||
Every usable parsed row contributes one gold-standard group. A distinct surface form is one evaluated item and may belong to several groups. `ALL_WORDS` preserves every valid form. `LOWERCASE_GROUPS_ONLY` excludes a complete group containing an uppercase or titlecase Unicode code point. Retained words are not lowercased or normalized by the evaluator.
|
||||
|
||||
## Output policies
|
||||
|
||||
@@ -24,9 +24,9 @@ Every usable parsed row is one gold-standard group. Exact duplicate strings are
|
||||
|
||||
For multi-output adapters, `C(w)` is the immutable, sorted, exactly deduplicated candidate set. It is non-null, non-empty, contains no null, and contains the primary output. Radixor obtains alternatives through `getAll`. The repository's Morphologik lookups can return distinct lemma strings and are multi-output. Configured Hunspell filters can emit several stems at one token position. Other adapters emit only primary rows.
|
||||
|
||||
`ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound. A same-group pair succeeds when its sets intersect. A cross-group pair is an error only when both sets are the same singleton; otherwise unequal candidates can be selected for that pair. Choices may vary between pairs and need not form one realizable global assignment.
|
||||
`ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound. A gold-related pair succeeds when its sets intersect. A gold-negative pair is an error only when both sets are the same singleton; otherwise unequal candidates can be selected for that pair. Choices may vary between pairs and need not form one realizable global assignment.
|
||||
|
||||
`ALL_CANDIDATES` activates every candidate. Two forms are related when their sets intersect, for both same-group and cross-group pairs. This relation can overlap and need not be transitive. A pair sharing several candidates is counted once.
|
||||
`ALL_CANDIDATES` activates every candidate. Two forms are predicted as related when their sets intersect, for both gold-related and gold-negative pairs. This relation can overlap and need not be transitive. A pair sharing several candidates is counted once.
|
||||
|
||||
The evaluator verifies:
|
||||
|
||||
@@ -40,14 +40,14 @@ ALL over >= PRIMARY over
|
||||
|
||||
## Pair definitions and efficient counting
|
||||
|
||||
For `C2(n) = n(n-1)/2`:
|
||||
For the unique form population `W`:
|
||||
|
||||
```text
|
||||
underPossible = sum_g C2(n_g)
|
||||
overPossible = C2(N) - sum_g C2(n_g)
|
||||
underPossible = |{{u,v} subset W : G(u) intersection G(v) is not empty}|
|
||||
overPossible = C2(|W|) - underPossible
|
||||
```
|
||||
|
||||
Under-stemming counts unrelated same-group pairs. Over-stemming counts related cross-group pairs. Primary output uses global and per-group stem frequencies. Candidate sets are canonical signatures counted globally and per group. An inverted candidate-to-signature index discovers intersections, and signature pairs shared through several candidates are deduplicated. `ANY_CANDIDATE` over-stemming uses only equal singleton signatures. All pair arithmetic uses checked `long` operations; complete production word pairs are never enumerated.
|
||||
For each distinct form `w`, let `G(w)` be its set of included dictionary groups. Two forms are gold-related exactly when their membership sets intersect. Under-stemming counts gold-related pairs that the output relation separates. Over-stemming counts gold-negative pairs that the output relation conflates. A form is processed once, and a pair sharing several groups is counted once. Primary output uses global and per-group stem frequencies with explicit overlap corrections. Candidate sets are canonical signatures counted globally and per group. An inverted candidate-to-signature index discovers intersections, and signature pairs shared through several candidates are deduplicated. `ANY_CANDIDATE` over-stemming uses only equal singleton signatures. All pair arithmetic uses checked `long` operations; complete production word pairs are never enumerated.
|
||||
|
||||
## Confusion and aggregate metrics
|
||||
|
||||
@@ -58,9 +58,9 @@ FP = overError
|
||||
TN = overPossible - overError
|
||||
```
|
||||
|
||||
Under-stemming is `FN/(TP+FN)` and over-stemming is `FP/(TN+FP)`; their denominators differ. The CSV also publishes precision, recall, specificity, accuracy, balanced accuracy, F0.5, F1, F2, Jaccard, Fowlkes-Mallows, Matthews correlation coefficient, and pairwise error rate. F0.5 emphasizes precision and over-stemming, F1 balances precision and recall, and F2 emphasizes recall and under-stemming. Accuracy and error rate can be dominated by the large cross-group true-negative population. Metrics use raw counts, not rounded rates. Zero denominators produce `n/a` in Markdown and empty CSV fields.
|
||||
Under-stemming is Paice UI `FN/(TP+FN)` and over-stemming is Paice OI `FP/(TN+FP)`; their denominators differ. This is an explicit pairwise generalization of Paice's disjoint lemma groups to the overlapping gold relation above. For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, the CSV also publishes TP/FP/FN/TN, precision, recall, specificity, accuracy, balanced accuracy, F0.5, F1, F2, Jaccard, Fowlkes-Mallows, Matthews correlation coefficient, and pairwise error rate. `ANY_CANDIDATE` uses different optimistic oracle conditions for positive and negative pairs and therefore has no single confusion matrix; its TP/FP/FN/TN and aggregate classification fields are empty, while its explicit under/over numerators and denominators remain available. Metrics use raw counts, not rounded rates. Zero denominators produce `n/a` in Markdown and empty CSV fields.
|
||||
|
||||
Only `PRIMARY_OUTPUT` receives partition metrics: Adjusted Rand Index, homogeneity, completeness, V-measure, and normalized mutual information with arithmetic-mean entropy normalization. Candidate policies remain inapplicable rather than being forced into artificial partitions.
|
||||
Standard Adjusted Rand Index, homogeneity, completeness, V-measure, and normalized mutual information are not calculated. Their ordinary contingency-table definitions require an exclusive gold partition, while these dictionary groups form an overlapping cover.
|
||||
|
||||
Micro summaries sum confusion counts before calculation. Macro summaries average defined language values and retain coverage counts. Common-language comparisons use the exact language intersection and never score unsupported languages as zero. Rankings are separated by policy and metric; the default F0.5 choice is navigation, not a universal scientific preference.
|
||||
|
||||
@@ -79,9 +79,9 @@ Exact textual accuracy and pairwise grouping use different denominators. One err
|
||||
|
||||
Optional properties are `stemmingQualityLanguage`, `stemmingQualityStemmer`, `stemmingQualityMode`, `stemmingQualityOutputPolicy`, `stemmingQualityRankMetric`, `stemmingQualityAudit`, and `stemmingQualityAuditLimit`. Policies are `PRIMARY_OUTPUT`, `ANY_CANDIDATE`, and `ALL_CANDIDATES`. Filtered reports carry `-filtered` and cannot overwrite complete output.
|
||||
|
||||
Generated files under `build/reports/stemming-quality/` include `stemming-quality.md`, `stemming-quality.csv`, `metric-correlations-pearson.csv`, `metric-correlations-spearman.csv`, and optional audit Markdown.
|
||||
Generated files under `build/reports/stemming-quality/` include `stemming-quality.md`, `stemming-quality.csv`, `metric-correlations-pearson.csv`, `metric-correlations-spearman.csv`, and optional audit Markdown. Every CSV scenario records the exact dictionary model ID, independent model version, and descriptor SHA-256.
|
||||
|
||||
## Limitations
|
||||
|
||||
These measurements evaluate agreement with the available dictionary grouping. They do not capture every semantic, morphological, downstream, or dataset-specific property. `ANY_CANDIDATE` is optimistic and may not be globally realizable. `ALL_CANDIDATES` measures an overlap graph rather than a partition. Language coverage must remain visible in cross-stemmer comparisons. No single published metric establishes universal superiority; multiple metrics and their correlations are provided for transparent scientific assessment.
|
||||
Historical checked-in quality results retain their original inputs and claims. The optional PoliMorf model is not attributed to snapshots that predate it. See [Model Selection and Loading](model-selection-and-loading.md) and the generated [model catalog](stemmer-model-catalog.md).
|
||||
The checked-in quality snapshot is regenerated from all 20 current default models. The optional PoliMorf model is not part of it and must not be attributed to the default Polish results. See [Model Selection and Loading](model-selection-and-loading.md) and the generated [model catalog](stemmer-model-catalog.md).
|
||||
|
||||
@@ -87,6 +87,7 @@ nav:
|
||||
- Finnish: benchmarks/languages/finnish.md
|
||||
- French: benchmarks/languages/french.md
|
||||
- German: benchmarks/languages/german.md
|
||||
- Hebrew: benchmarks/languages/hebrew.md
|
||||
- Hungarian: benchmarks/languages/hungarian.md
|
||||
- Italian: benchmarks/languages/italian.md
|
||||
- Norwegian Bokmal: benchmarks/languages/norwegian-bokmal.md
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
/*******************************************************************************
|
||||
* Copyright (C) 2026, Leo Galambos
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
* modification, are permitted provided that the following conditions are met:
|
||||
*
|
||||
* 1. Redistributions of source code must retain the above copyright notice,
|
||||
* this list of conditions and the following disclaimer.
|
||||
*
|
||||
* 2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
* this list of conditions and the following disclaimer in the documentation
|
||||
* and/or other materials provided with the distribution.
|
||||
*
|
||||
* 3. Neither the name of the copyright holder nor the names of its contributors
|
||||
* may be used to endorse or promote products derived from this software
|
||||
* without specific prior written permission.
|
||||
*
|
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||
* POSSIBILITY OF SUCH DAMAGE.
|
||||
******************************************************************************/
|
||||
package org.egothor.stemmer.benchmark;
|
||||
|
||||
import java.io.BufferedReader;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.io.InputStreamReader;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.TreeMap;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
import org.egothor.stemmer.CaseProcessingMode;
|
||||
import org.egothor.stemmer.CompiledPatchCommand;
|
||||
import org.egothor.stemmer.FrequencyTrie;
|
||||
import org.egothor.stemmer.ReductionMode;
|
||||
import org.egothor.stemmer.StemmerDictionaryParser;
|
||||
import org.egothor.stemmer.StemmerModelDescriptor;
|
||||
import org.egothor.stemmer.StemmerModelRegistry;
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader;
|
||||
|
||||
/**
|
||||
* Writes deterministic corpus and preferred patch-command counts for every
|
||||
* registered default model.
|
||||
*
|
||||
* <p>
|
||||
* This application performs setup-time analysis only; it does not publish or
|
||||
* interpret runtime performance. Optional model variants are excluded by
|
||||
* resolving every entry through
|
||||
* {@link StemmerModelRegistry#requireDefault(StemmerPatchTrieLoader.Language)}.
|
||||
* </p>
|
||||
*/
|
||||
public final class BenchmarkCorpusReportApplication {
|
||||
|
||||
/**
|
||||
* Utility class.
|
||||
*/
|
||||
private BenchmarkCorpusReportApplication() {
|
||||
throw new AssertionError("No instances.");
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes one UTF-8 CSV report.
|
||||
*
|
||||
* @param arguments one output-file path
|
||||
* @throws IOException if model discovery, dictionary parsing, trie loading, or
|
||||
* report writing fails
|
||||
*/
|
||||
public static void main(final String[] arguments) throws IOException {
|
||||
if (arguments.length != 1) {
|
||||
throw new IllegalArgumentException("Expected one corpus-report output path.");
|
||||
}
|
||||
|
||||
final Path output = Path.of(arguments[0]);
|
||||
final Path parent = output.toAbsolutePath().getParent();
|
||||
if (parent != null) {
|
||||
Files.createDirectories(parent);
|
||||
}
|
||||
|
||||
final StringBuilder csv = new StringBuilder(16_384);
|
||||
csv.append("Language,Model ID,Model version,Model SHA-256,Dictionary rows,Total tokens,Already-root tokens,Changed tokens,")
|
||||
.append("Speed timing tokens,All exact matches,Changed exact matches,Root preserved matches,")
|
||||
.append("Command class,Command count\n");
|
||||
final StemmerModelRegistry registry = StemmerModelRegistry.fromContextClassLoader();
|
||||
for (StemmerPatchTrieLoader.Language language : StemmerPatchTrieLoader.Language.values()) {
|
||||
appendLanguage(csv, registry, language);
|
||||
}
|
||||
Files.writeString(output, csv, StandardCharsets.UTF_8);
|
||||
System.out.println("Benchmark corpus report: " + output.toAbsolutePath());
|
||||
}
|
||||
|
||||
/**
|
||||
* Appends all command-class rows for one default model.
|
||||
*
|
||||
* @param csv destination
|
||||
* @param registry discovered model registry
|
||||
* @param language language to analyze
|
||||
* @throws IOException if the model cannot be parsed or loaded
|
||||
*/
|
||||
private static void appendLanguage(final StringBuilder csv, final StemmerModelRegistry registry,
|
||||
final StemmerPatchTrieLoader.Language language) throws IOException {
|
||||
final StemmerModelDescriptor descriptor = registry.requireDefault(language);
|
||||
final int dictionaryRows = countDictionaryRows(descriptor);
|
||||
final LanguageBenchmarkCorpus.Corpus corpus = LanguageBenchmarkCorpus.createFullCorpus(language);
|
||||
final String[] tokens = corpus.tokens();
|
||||
final String[] expectedRoots = corpus.expectedRoots();
|
||||
long alreadyRootTokens = 0;
|
||||
for (int index = 0; index < tokens.length; index++) {
|
||||
if (Objects.equals(tokens[index], expectedRoots[index])) {
|
||||
alreadyRootTokens++;
|
||||
}
|
||||
}
|
||||
final long changedTokens = tokens.length - alreadyRootTokens;
|
||||
final int timingTokens = LanguageBenchmarkCorpus.createChangedCorpus(language).tokens().length;
|
||||
|
||||
final FrequencyTrie<CompiledPatchCommand> trie = StemmerPatchTrieLoader.loadCompiled(language, true,
|
||||
ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);
|
||||
final RadixorBenchmarkStemmer stemmer = new RadixorBenchmarkStemmer(trie);
|
||||
final Map<String, Long> commandCounts = new TreeMap<>();
|
||||
long allExactMatches = 0;
|
||||
long changedExactMatches = 0;
|
||||
long rootPreservedMatches = 0;
|
||||
for (int index = 0; index < tokens.length; index++) {
|
||||
final String token = tokens[index];
|
||||
final String expectedRoot = expectedRoots[index];
|
||||
final CompiledPatchCommand command = trie.getNormalizedString(token);
|
||||
final String commandClass = command == null ? "NoCommand" : command.getClass().getSimpleName();
|
||||
commandCounts.merge(commandClass, 1L, Math::addExact);
|
||||
final String actualRoot = stemmer.stem(token);
|
||||
if (Objects.equals(actualRoot, expectedRoot)) {
|
||||
allExactMatches++;
|
||||
if (Objects.equals(token, expectedRoot)) {
|
||||
rootPreservedMatches++;
|
||||
} else {
|
||||
changedExactMatches++;
|
||||
}
|
||||
}
|
||||
}
|
||||
for (Map.Entry<String, Long> commandCount : commandCounts.entrySet()) {
|
||||
csv.append(language).append(',')
|
||||
.append(descriptor.id()).append(',')
|
||||
.append(descriptor.version()).append(',')
|
||||
.append(descriptor.sha256()).append(',')
|
||||
.append(dictionaryRows).append(',')
|
||||
.append(tokens.length).append(',')
|
||||
.append(alreadyRootTokens).append(',')
|
||||
.append(changedTokens).append(',')
|
||||
.append(timingTokens).append(',')
|
||||
.append(allExactMatches).append(',')
|
||||
.append(changedExactMatches).append(',')
|
||||
.append(rootPreservedMatches).append(',')
|
||||
.append(commandCount.getKey()).append(',')
|
||||
.append(commandCount.getValue()).append('\n');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Counts valid logical rows in one default dictionary.
|
||||
*
|
||||
* @param descriptor model descriptor
|
||||
* @return parsed dictionary-row count
|
||||
* @throws IOException if the dictionary cannot be opened or parsed
|
||||
*/
|
||||
private static int countDictionaryRows(final StemmerModelDescriptor descriptor) throws IOException {
|
||||
final ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader();
|
||||
final ClassLoader classLoader = contextClassLoader == null
|
||||
? BenchmarkCorpusReportApplication.class.getClassLoader()
|
||||
: contextClassLoader;
|
||||
final InputStream resource = classLoader.getResourceAsStream(descriptor.resource());
|
||||
if (resource == null) {
|
||||
throw new IOException("Dictionary resource is missing for model " + descriptor.id() + ": "
|
||||
+ descriptor.resource() + ".");
|
||||
}
|
||||
final int[] rows = {0};
|
||||
try (InputStream raw = resource;
|
||||
GZIPInputStream gzip = new GZIPInputStream(raw);
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(gzip, StandardCharsets.UTF_8))) {
|
||||
StemmerDictionaryParser.parse(reader, descriptor.resource(), CaseProcessingMode.LOWERCASE_WITH_LOCALE_ROOT,
|
||||
(stem, variants, lineNumber) -> rows[0] = Math.addExact(rows[0], 1));
|
||||
}
|
||||
return rows[0];
|
||||
}
|
||||
}
|
||||
@@ -37,12 +37,15 @@ import java.io.InputStreamReader;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.EnumMap;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Locale;
|
||||
import java.util.Objects;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
import org.egothor.stemmer.StemmerModelDescriptor;
|
||||
import org.egothor.stemmer.StemmerModelRegistry;
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader;
|
||||
|
||||
/**
|
||||
@@ -75,6 +78,11 @@ final class LanguageBenchmarkCorpus {
|
||||
private static final Map<StemmerPatchTrieLoader.Language, Corpus> CHANGED_TIMING_CORPORA =
|
||||
new EnumMap<>(StemmerPatchTrieLoader.Language.class);
|
||||
|
||||
/**
|
||||
* Shared changed-token timing corpora keyed by explicit bundled model ID.
|
||||
*/
|
||||
private static final Map<String, Corpus> MODEL_CHANGED_TIMING_CORPORA = new HashMap<>();
|
||||
|
||||
/**
|
||||
* Shared complete corpora keyed by bundled Radixor language.
|
||||
*/
|
||||
@@ -107,6 +115,25 @@ final class LanguageBenchmarkCorpus {
|
||||
return createChangedCorpus(language).tokens();
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a deterministic changed-token timing corpus from an explicitly
|
||||
* selected bundled model dictionary.
|
||||
*
|
||||
* <p>
|
||||
* Only token/root pairs where the token differs from the expected root are
|
||||
* included. Smaller changed-token resources are repeated in stable order until
|
||||
* the timing corpus reaches 5,000 tokens.
|
||||
* </p>
|
||||
*
|
||||
* @param modelId exact bundled model identifier
|
||||
* @return token array containing changed-token dictionary entries, repeated
|
||||
* only when the changed-token resource is smaller than 5,000 tokens
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
static String[] createTokens(final String modelId) throws IOException {
|
||||
return createChangedCorpus(modelId).tokens();
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a deterministic changed-token timing corpus from a bundled language
|
||||
* dictionary.
|
||||
@@ -119,6 +146,18 @@ final class LanguageBenchmarkCorpus {
|
||||
return cachedChangedCorpus(language);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a deterministic changed-token timing corpus from an explicitly
|
||||
* selected bundled model dictionary.
|
||||
*
|
||||
* @param modelId exact bundled model identifier
|
||||
* @return changed-token corpus with expected roots
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
static Corpus createChangedCorpus(final String modelId) throws IOException {
|
||||
return cachedChangedCorpus(modelId);
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a deterministic full-dictionary timing corpus and expected root
|
||||
* array from a bundled language dictionary.
|
||||
@@ -220,6 +259,29 @@ final class LanguageBenchmarkCorpus {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns a cached changed-token timing corpus, creating it once per JVM when
|
||||
* necessary.
|
||||
*
|
||||
* @param modelId exact bundled model identifier
|
||||
* @return changed-token timing corpus
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
private static Corpus cachedChangedCorpus(final String modelId) throws IOException {
|
||||
Objects.requireNonNull(modelId, "modelId");
|
||||
|
||||
synchronized (LanguageBenchmarkCorpus.class) {
|
||||
final Corpus existing = MODEL_CHANGED_TIMING_CORPORA.get(modelId);
|
||||
if (existing != null) {
|
||||
return existing;
|
||||
}
|
||||
|
||||
final Corpus created = buildChangedTimingCorpus(modelId, MINIMUM_TIMING_TOKEN_COUNT);
|
||||
MODEL_CHANGED_TIMING_CORPORA.put(modelId, created);
|
||||
return created;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a deterministic timing corpus from a bundled language dictionary.
|
||||
*
|
||||
@@ -263,11 +325,41 @@ final class LanguageBenchmarkCorpus {
|
||||
private static Corpus buildChangedTimingCorpus(final StemmerPatchTrieLoader.Language language,
|
||||
final int minimumTokenCount) throws IOException {
|
||||
Objects.requireNonNull(language, "language");
|
||||
return buildChangedTimingCorpus(readCandidates(language, Integer.MAX_VALUE), language.toString(),
|
||||
minimumTokenCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a deterministic changed-token timing corpus from an explicitly
|
||||
* selected bundled model dictionary.
|
||||
*
|
||||
* @param modelId exact bundled model identifier
|
||||
* @param minimumTokenCount minimum token count for timing
|
||||
* @return changed-token corpus with expected roots
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
private static Corpus buildChangedTimingCorpus(final String modelId, final int minimumTokenCount)
|
||||
throws IOException {
|
||||
Objects.requireNonNull(modelId, "modelId");
|
||||
return buildChangedTimingCorpus(readCandidates(modelId, Integer.MAX_VALUE), modelId, minimumTokenCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a deterministic changed-token timing corpus from parsed entries.
|
||||
*
|
||||
* @param allCandidates all valid dictionary entries
|
||||
* @param sourceLabel human-readable source label for diagnostics
|
||||
* @param minimumTokenCount minimum token count for timing
|
||||
* @return changed-token corpus with expected roots
|
||||
*/
|
||||
private static Corpus buildChangedTimingCorpus(final List<Entry> allCandidates, final String sourceLabel,
|
||||
final int minimumTokenCount) {
|
||||
Objects.requireNonNull(allCandidates, "allCandidates");
|
||||
Objects.requireNonNull(sourceLabel, "sourceLabel");
|
||||
if (minimumTokenCount < 1) {
|
||||
throw new IllegalArgumentException("minimumTokenCount must be at least 1.");
|
||||
}
|
||||
|
||||
final List<Entry> allCandidates = readCandidates(language, Integer.MAX_VALUE);
|
||||
final List<Entry> changedCandidates = new ArrayList<>(allCandidates.size());
|
||||
for (Entry entry : allCandidates) {
|
||||
if (!Objects.equals(entry.token(), entry.root())) {
|
||||
@@ -276,7 +368,7 @@ final class LanguageBenchmarkCorpus {
|
||||
}
|
||||
if (changedCandidates.isEmpty()) {
|
||||
throw new IllegalStateException("No changed-token benchmark corpus tokens were available for "
|
||||
+ language + ".");
|
||||
+ sourceLabel + ".");
|
||||
}
|
||||
|
||||
final int timingTokenCount = Math.max(changedCandidates.size(), minimumTokenCount);
|
||||
@@ -332,8 +424,35 @@ final class LanguageBenchmarkCorpus {
|
||||
*/
|
||||
private static List<Entry> readCandidates(final StemmerPatchTrieLoader.Language language, final int maximumTokenCount)
|
||||
throws IOException {
|
||||
final String resourcePath = org.egothor.stemmer.StemmerModelRegistry.fromContextClassLoader()
|
||||
.requireDefault(language).resource();
|
||||
final StemmerModelDescriptor descriptor = StemmerModelRegistry.fromContextClassLoader()
|
||||
.requireDefault(language);
|
||||
return readCandidatesFromResource(descriptor.resource(), maximumTokenCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads token candidates from an explicitly selected bundled compressed
|
||||
* dictionary.
|
||||
*
|
||||
* @param modelId exact bundled model identifier
|
||||
* @param maximumTokenCount maximum token count to read
|
||||
* @return deterministic candidate list
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
private static List<Entry> readCandidates(final String modelId, final int maximumTokenCount) throws IOException {
|
||||
final StemmerModelDescriptor descriptor = StemmerModelRegistry.fromContextClassLoader().require(modelId);
|
||||
return readCandidatesFromResource(descriptor.resource(), maximumTokenCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads token candidates from a bundled compressed dictionary resource.
|
||||
*
|
||||
* @param resourcePath classpath resource path
|
||||
* @param maximumTokenCount maximum token count to read
|
||||
* @return deterministic candidate list
|
||||
* @throws IOException if the resource cannot be read
|
||||
*/
|
||||
private static List<Entry> readCandidatesFromResource(final String resourcePath, final int maximumTokenCount)
|
||||
throws IOException {
|
||||
final InputStream resource = StemmerPatchTrieLoader.class.getClassLoader().getResourceAsStream(resourcePath);
|
||||
if (resource == null) {
|
||||
throw new IllegalStateException("Missing bundled benchmark resource " + resourcePath + ".");
|
||||
|
||||
@@ -143,6 +143,11 @@ public class MultiLanguageStemmerComparisonBenchmark {
|
||||
*/
|
||||
private LanguageState french;
|
||||
|
||||
/**
|
||||
* Hebrew benchmark state.
|
||||
*/
|
||||
private LanguageState hebrew;
|
||||
|
||||
/**
|
||||
* Hungarian benchmark state.
|
||||
*/
|
||||
@@ -196,6 +201,7 @@ public class MultiLanguageStemmerComparisonBenchmark {
|
||||
this.persian = load(StemmerPatchTrieLoader.Language.FA_IR);
|
||||
this.finnish = load(StemmerPatchTrieLoader.Language.FI_FI);
|
||||
this.french = load(StemmerPatchTrieLoader.Language.FR_FR);
|
||||
this.hebrew = load(StemmerPatchTrieLoader.Language.HE_IL);
|
||||
this.hungarian = load(StemmerPatchTrieLoader.Language.HU_HU);
|
||||
this.italian = load(StemmerPatchTrieLoader.Language.IT_IT);
|
||||
this.norwegianBokmal = load(StemmerPatchTrieLoader.Language.NB_NO);
|
||||
@@ -600,6 +606,17 @@ public class MultiLanguageStemmerComparisonBenchmark {
|
||||
runRadixor(sharedState.french, blackhole);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs Radixor over the Hebrew corpus.
|
||||
*
|
||||
* @param sharedState shared benchmark state
|
||||
* @param blackhole result sink
|
||||
*/
|
||||
@Benchmark
|
||||
public void hebrewRadixor(final SharedState sharedState, final Blackhole blackhole) {
|
||||
runRadixor(sharedState.hebrew, blackhole);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs Lucene FrenchLightStemFilter over the French corpus.
|
||||
*
|
||||
|
||||
@@ -0,0 +1,213 @@
|
||||
/*******************************************************************************
|
||||
* Copyright (C) 2026, Leo Galambos
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
* modification, are permitted provided that the following conditions are met:
|
||||
*
|
||||
* 1. Redistributions of source code must retain the above copyright notice,
|
||||
* this list of conditions and the following disclaimer.
|
||||
*
|
||||
* 2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
* this list of conditions and the following disclaimer in the documentation
|
||||
* and/or other materials provided with the distribution.
|
||||
*
|
||||
* 3. Neither the name of the copyright holder nor the names of its contributors
|
||||
* may be used to endorse or promote products derived from this software
|
||||
* without specific prior written permission.
|
||||
*
|
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||
* POSSIBILITY OF SUCH DAMAGE.
|
||||
******************************************************************************/
|
||||
package org.egothor.stemmer.benchmark;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
import org.apache.lucene.analysis.TokenStream;
|
||||
import org.apache.lucene.analysis.morfologik.MorfologikFilter;
|
||||
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
|
||||
import org.egothor.stemmer.CompiledPatchCommand;
|
||||
import org.egothor.stemmer.FrequencyTrie;
|
||||
import org.egothor.stemmer.ReductionMode;
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader;
|
||||
import org.openjdk.jmh.annotations.Benchmark;
|
||||
import org.openjdk.jmh.annotations.BenchmarkMode;
|
||||
import org.openjdk.jmh.annotations.Fork;
|
||||
import org.openjdk.jmh.annotations.Level;
|
||||
import org.openjdk.jmh.annotations.Measurement;
|
||||
import org.openjdk.jmh.annotations.Mode;
|
||||
import org.openjdk.jmh.annotations.OutputTimeUnit;
|
||||
import org.openjdk.jmh.annotations.Scope;
|
||||
import org.openjdk.jmh.annotations.Setup;
|
||||
import org.openjdk.jmh.annotations.State;
|
||||
import org.openjdk.jmh.annotations.Warmup;
|
||||
import org.openjdk.jmh.infra.Blackhole;
|
||||
|
||||
/**
|
||||
* Compares the two Polish production stemmer paths over the PoliMorf-backed
|
||||
* Radixor dictionary workload.
|
||||
*
|
||||
* <p>
|
||||
* Each benchmark operation processes the same changed-token corpus derived from
|
||||
* {@code pl-pl-polimorf}. The Radixor method uses the explicit
|
||||
* {@code pl-pl-polimorf} runtime model, while the Lucene method uses the public
|
||||
* {@link MorfologikFilter} path.
|
||||
* </p>
|
||||
*/
|
||||
@BenchmarkMode(Mode.AverageTime)
|
||||
@OutputTimeUnit(TimeUnit.NANOSECONDS)
|
||||
@Warmup(iterations = 3, time = 1, timeUnit = TimeUnit.SECONDS)
|
||||
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
|
||||
@Fork(value = 1, jvmArgsAppend = { "-Xmx6g" })
|
||||
public class PolishPolimorfStemmerComparisonBenchmark {
|
||||
|
||||
/**
|
||||
* Explicit Polish PoliMorf model identifier.
|
||||
*/
|
||||
private static final String POLIMORF_MODEL_ID = "pl-pl-polimorf";
|
||||
|
||||
/**
|
||||
* Shared PoliMorf corpus and Radixor trie state.
|
||||
*/
|
||||
@State(Scope.Benchmark)
|
||||
public static class SharedState {
|
||||
|
||||
/**
|
||||
* Shared deterministic changed-token dictionary corpus.
|
||||
*/
|
||||
private String[] tokens;
|
||||
|
||||
/**
|
||||
* Radixor benchmark adapter over the PoliMorf model.
|
||||
*/
|
||||
private RadixorBenchmarkStemmer radixorStemmer;
|
||||
|
||||
/**
|
||||
* Initializes the PoliMorf corpus and Radixor trie before measurement.
|
||||
*
|
||||
* @throws IOException if the corpus or trie cannot be loaded
|
||||
*/
|
||||
@Setup(Level.Trial)
|
||||
public void setUp() throws IOException {
|
||||
this.tokens = LanguageBenchmarkCorpus.createTokens(POLIMORF_MODEL_ID);
|
||||
final FrequencyTrie<CompiledPatchCommand> trie = StemmerPatchTrieLoader.loadCompiled(POLIMORF_MODEL_ID,
|
||||
true, ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);
|
||||
this.radixorStemmer = new RadixorBenchmarkStemmer(trie);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-thread Lucene Morfologik filter state.
|
||||
*/
|
||||
@State(Scope.Thread)
|
||||
public static class LuceneFilterState {
|
||||
|
||||
/**
|
||||
* Reusable Morfologik token-filter pipeline.
|
||||
*/
|
||||
private final FilterPipeline polishMorfologik = new FilterPipeline(MorfologikFilter::new);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs Radixor with the {@code pl-pl-polimorf} model over the PoliMorf corpus.
|
||||
*
|
||||
* @param sharedState shared benchmark state
|
||||
* @param blackhole result sink
|
||||
*/
|
||||
@Benchmark
|
||||
public void polishPolimorfRadixor(final SharedState sharedState, final Blackhole blackhole) {
|
||||
final String[] tokens = sharedState.tokens;
|
||||
final RadixorBenchmarkStemmer stemmer = sharedState.radixorStemmer;
|
||||
|
||||
for (String token : tokens) {
|
||||
blackhole.consume(stemmer.stem(token));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs Lucene MorfologikFilter over the PoliMorf-derived corpus.
|
||||
*
|
||||
* @param sharedState shared benchmark state
|
||||
* @param filterState reusable filter state
|
||||
* @param blackhole result sink
|
||||
* @throws IOException if Lucene token streaming fails
|
||||
*/
|
||||
@Benchmark
|
||||
public void polishLuceneMorfologikFilter(final SharedState sharedState, final LuceneFilterState filterState,
|
||||
final Blackhole blackhole) throws IOException {
|
||||
filterState.polishMorfologik.run(sharedState.tokens, blackhole);
|
||||
}
|
||||
|
||||
/**
|
||||
* Factory for a Lucene filter under test.
|
||||
*/
|
||||
private interface FilterFactory {
|
||||
|
||||
/**
|
||||
* Creates a token stream wrapping the supplied benchmark input stream.
|
||||
*
|
||||
* @param input input token stream
|
||||
* @return filter stream
|
||||
*/
|
||||
TokenStream create(TokenStream input);
|
||||
}
|
||||
|
||||
/**
|
||||
* Reusable input stream, filter stream, and term attribute for one Lucene
|
||||
* benchmark method.
|
||||
*/
|
||||
private static final class FilterPipeline {
|
||||
|
||||
/**
|
||||
* Reusable benchmark input stream.
|
||||
*/
|
||||
private final BenchmarkTokenStream input;
|
||||
|
||||
/**
|
||||
* Lucene filter output stream.
|
||||
*/
|
||||
private final TokenStream output;
|
||||
|
||||
/**
|
||||
* Term attribute consumed by the benchmark.
|
||||
*/
|
||||
private final CharTermAttribute termAttribute;
|
||||
|
||||
/**
|
||||
* Creates one reusable filter pipeline.
|
||||
*
|
||||
* @param factory filter factory
|
||||
*/
|
||||
private FilterPipeline(final FilterFactory factory) {
|
||||
this.input = new BenchmarkTokenStream(new String[0]);
|
||||
this.output = factory.create(this.input);
|
||||
this.termAttribute = this.output.addAttribute(CharTermAttribute.class);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs the filter over one token corpus and consumes all emitted terms.
|
||||
*
|
||||
* @param tokens token corpus
|
||||
* @param blackhole result sink
|
||||
* @throws IOException if Lucene token streaming fails
|
||||
*/
|
||||
private void run(final String[] tokens, final Blackhole blackhole) throws IOException {
|
||||
this.input.setTokens(tokens);
|
||||
this.output.reset();
|
||||
while (this.output.incrementToken()) {
|
||||
blackhole.consume(this.termAttribute.toString());
|
||||
}
|
||||
this.output.end();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -41,6 +41,9 @@ import org.egothor.stemmer.StemmerPatchTrieLoader.Language;
|
||||
|
||||
/** Authoritative analytical view of the candidate matrix defined by the JMH quality benchmark. */
|
||||
public final class QualityStemmerMatrix {
|
||||
/** Optional Polish PoliMorf model included as an explicit non-default comparison. */
|
||||
private static final String POLISH_POLIMORF_MODEL_ID = "pl-pl-polimorf";
|
||||
|
||||
/** Utility class. */
|
||||
private QualityStemmerMatrix() {
|
||||
throw new AssertionError("No instances.");
|
||||
@@ -81,6 +84,15 @@ public final class QualityStemmerMatrix {
|
||||
@Override public boolean supportsMultipleOutputs() { return true; }
|
||||
}))
|
||||
.forEach(candidates::add);
|
||||
final List<Candidate> defaultPolishCandidates = candidates.stream()
|
||||
.filter(candidate -> candidate.language() == Language.PL_PL)
|
||||
.toList();
|
||||
candidates.add(new Candidate("POLISH_POLIMORF_RADIXOR", Language.PL_PL,
|
||||
POLISH_POLIMORF_MODEL_ID, POLISH_POLIMORF_MODEL_ID,
|
||||
() -> adapt(StemmerComparisonBenchmarkQuality.createRadixorQualityStemmer(POLISH_POLIMORF_MODEL_ID))));
|
||||
defaultPolishCandidates
|
||||
.forEach(candidate -> candidates.add(new Candidate(candidate.name(), candidate.language(),
|
||||
POLISH_POLIMORF_MODEL_ID, POLISH_POLIMORF_MODEL_ID, candidate.factory)));
|
||||
return List.copyOf(candidates);
|
||||
}
|
||||
|
||||
@@ -102,13 +114,23 @@ public final class QualityStemmerMatrix {
|
||||
public static final class Candidate {
|
||||
private final String name;
|
||||
private final Language language;
|
||||
private final String resultLanguage;
|
||||
private final String dictionaryModelId;
|
||||
private final StemmerFactory factory;
|
||||
|
||||
/** Creates an immutable facade over one benchmark candidate. */
|
||||
private Candidate(final String name, final Language language,
|
||||
final StemmerFactory factory) {
|
||||
this(name, language, language.name(), language.defaultModelId(), factory);
|
||||
}
|
||||
|
||||
/** Creates an immutable facade over one benchmark candidate and dictionary model. */
|
||||
private Candidate(final String name, final Language language, final String resultLanguage,
|
||||
final String dictionaryModelId, final StemmerFactory factory) {
|
||||
this.name = Objects.requireNonNull(name, "name");
|
||||
this.language = Objects.requireNonNull(language, "language");
|
||||
this.resultLanguage = Objects.requireNonNull(resultLanguage, "resultLanguage");
|
||||
this.dictionaryModelId = Objects.requireNonNull(dictionaryModelId, "dictionaryModelId");
|
||||
this.factory = Objects.requireNonNull(factory, "factory");
|
||||
}
|
||||
|
||||
@@ -122,6 +144,16 @@ public final class QualityStemmerMatrix {
|
||||
return this.language;
|
||||
}
|
||||
|
||||
/** @return stable report language or model label */
|
||||
public String resultLanguage() {
|
||||
return this.resultLanguage;
|
||||
}
|
||||
|
||||
/** @return exact dictionary model used as the gold-standard grouping source */
|
||||
public String dictionaryModelId() {
|
||||
return this.dictionaryModelId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a scenario-confined adapter using exactly the JMH factory and preprocessing path.
|
||||
*
|
||||
|
||||
@@ -643,6 +643,19 @@ public class StemmerComparisonBenchmarkQuality {
|
||||
return radixor(createRadixorStemmer(language));
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the authoritative multi-output Radixor adapter for an explicitly
|
||||
* selected runtime model.
|
||||
*
|
||||
* @param modelId exact model identifier
|
||||
* @return scenario-confined adapter using the JMH invocation path
|
||||
* @throws IOException if the compiled dictionary cannot be loaded
|
||||
*/
|
||||
static CandidateStemmer createRadixorQualityStemmer(final String modelId) throws IOException {
|
||||
return radixor(new RadixorBenchmarkStemmer(StemmerPatchTrieLoader.loadCompiled(
|
||||
modelId, true, ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Exact-root agreement counters for one quality operation.
|
||||
*
|
||||
|
||||
@@ -42,6 +42,8 @@ import java.util.Objects;
|
||||
import java.util.zip.GZIPInputStream;
|
||||
|
||||
import org.egothor.stemmer.CaseProcessingMode;
|
||||
import org.egothor.stemmer.StemmerModelDescriptor;
|
||||
import org.egothor.stemmer.StemmerModelRegistry;
|
||||
import org.egothor.stemmer.StemmerDictionaryParser;
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader.Language;
|
||||
|
||||
@@ -58,10 +60,21 @@ public final class BundledGoldStandardLoader {
|
||||
*/
|
||||
public static List<GoldStandardGroup> load(final Language language) throws IOException {
|
||||
Objects.requireNonNull(language, "language");
|
||||
final String resource = org.egothor.stemmer.StemmerModelRegistry.fromContextClassLoader()
|
||||
.requireDefault(language).resource();
|
||||
return loadModel(StemmerModelRegistry.fromContextClassLoader().requireDefault(language).id());
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses one explicitly selected compressed UTF-8 model dictionary with case preserved.
|
||||
* @param modelId exact model identifier
|
||||
* @return immutable groups in source-row order
|
||||
* @throws IOException if the resource is absent, malformed, or unreadable
|
||||
*/
|
||||
public static List<GoldStandardGroup> loadModel(final String modelId) throws IOException {
|
||||
Objects.requireNonNull(modelId, "modelId");
|
||||
final StemmerModelDescriptor descriptor = StemmerModelRegistry.fromContextClassLoader().require(modelId);
|
||||
final String resource = descriptor.resource();
|
||||
final List<GoldStandardGroup> groups = new ArrayList<>();
|
||||
try (InputStream raw = openResource(language, resource); InputStream gzip = new GZIPInputStream(raw);
|
||||
try (InputStream raw = openResource(modelId, resource); InputStream gzip = new GZIPInputStream(raw);
|
||||
BufferedReader reader = new BufferedReader(new InputStreamReader(gzip, StandardCharsets.UTF_8))) {
|
||||
StemmerDictionaryParser.parse(reader, resource, CaseProcessingMode.AS_IS, (stem, variants, row) -> {
|
||||
final List<String> forms = new ArrayList<>(variants.length + 1);
|
||||
@@ -70,7 +83,7 @@ public final class BundledGoldStandardLoader {
|
||||
try {
|
||||
groups.add(new GoldStandardGroup(row, forms));
|
||||
} catch (IllegalArgumentException exception) {
|
||||
throw new IOException("Invalid dictionary group for language " + language + ", resource "
|
||||
throw new IOException("Invalid dictionary group for model " + modelId + ", resource "
|
||||
+ resource + ", row " + row + ": " + exception.getMessage(), exception);
|
||||
}
|
||||
});
|
||||
@@ -79,10 +92,10 @@ public final class BundledGoldStandardLoader {
|
||||
}
|
||||
|
||||
/** Opens one required classpath resource with a precise language diagnostic. */
|
||||
private static InputStream openResource(final Language language, final String resource) throws IOException {
|
||||
private static InputStream openResource(final String modelId, final String resource) throws IOException {
|
||||
final InputStream input = Thread.currentThread().getContextClassLoader().getResourceAsStream(resource);
|
||||
if (input == null) {
|
||||
throw new IOException("Dictionary resource is missing for language " + language + ": " + resource + ".");
|
||||
throw new IOException("Dictionary resource is missing for model " + modelId + ": " + resource + ".");
|
||||
}
|
||||
return input;
|
||||
}
|
||||
|
||||
@@ -57,20 +57,8 @@ final class CandidateAwareEvaluator {
|
||||
if (policy == OutputPolicy.PRIMARY_OUTPUT) {
|
||||
throw new IllegalArgumentException("Candidate-aware evaluation requires ANY_CANDIDATE or ALL_CANDIDATES.");
|
||||
}
|
||||
final List<GoldStandardGroup> includedGroups = groups.stream().filter(group -> mode.includes(group.forms())).toList();
|
||||
final List<String> forms = new ArrayList<>();
|
||||
final List<Integer> groupIndexes = new ArrayList<>();
|
||||
long singletonRows = 0;
|
||||
long pairRows = 0;
|
||||
long underPossible = 0;
|
||||
for (int groupIndex = 0; groupIndex < includedGroups.size(); groupIndex++) {
|
||||
final GoldStandardGroup group = includedGroups.get(groupIndex);
|
||||
if (group.forms().size() == 1) { singletonRows = add(singletonRows, 1, "singleton rows"); }
|
||||
else { pairRows = add(pairRows, 1, "rows contributing under-stemming pairs"); }
|
||||
underPossible = add(underPossible, QualityEvaluator.chooseTwo(group.forms().size()), "under denominator");
|
||||
for (String form : group.forms()) { forms.add(form); groupIndexes.add(groupIndex); }
|
||||
}
|
||||
final String[] input = forms.toArray(String[]::new);
|
||||
final GoldStandardCover cover = GoldStandardCover.create(groups, mode);
|
||||
final String[] input = cover.forms().toArray(String[]::new);
|
||||
final String[] primary = stemmer.stem(input);
|
||||
final List<List<String>> rawCandidates = stemmer.stemCandidates(input);
|
||||
if (primary == null || primary.length != input.length || rawCandidates == null || rawCandidates.size() != input.length) {
|
||||
@@ -83,22 +71,31 @@ final class CandidateAwareEvaluator {
|
||||
long multipleCandidates = 0;
|
||||
long maximumCandidates = 0;
|
||||
long assignments = 0;
|
||||
final Signature[] formSignatures = new Signature[input.length];
|
||||
for (int index = 0; index < input.length; index++) {
|
||||
final Signature signature = signature(rawCandidates.get(index), primary[index], stemmerName, language,
|
||||
mode, policy, includedGroups.get(groupIndexes.get(index)).rowNumber(), input[index]);
|
||||
mode, policy, cover.representativeRow(index), input[index]);
|
||||
formSignatures[index] = signature;
|
||||
final int size = signature.candidates().size();
|
||||
if (size == 1) { oneCandidate = add(oneCandidate, 1, "single-candidate forms"); }
|
||||
else { multipleCandidates = add(multipleCandidates, 1, "multi-candidate forms"); }
|
||||
maximumCandidates = Math.max(maximumCandidates, size);
|
||||
assignments = add(assignments, size, "candidate assignments");
|
||||
distinctCandidates.addAll(signature.candidates());
|
||||
counts.computeIfAbsent(signature, ignored -> new SignatureCount()).increment(groupIndexes.get(index));
|
||||
counts.computeIfAbsent(signature, ignored -> new SignatureCount()).incrementTotal();
|
||||
}
|
||||
for (int groupIndex = 0; groupIndex < cover.groups().size(); groupIndex++) {
|
||||
for (String form : cover.groups().get(groupIndex).forms()) {
|
||||
counts.get(formSignatures[cover.indexOf(form)]).incrementGroup(groupIndex);
|
||||
}
|
||||
}
|
||||
|
||||
final List<Map.Entry<Signature, SignatureCount>> signatures = new ArrayList<>(counts.entrySet());
|
||||
signatures.sort(Map.Entry.comparingByKey());
|
||||
long sameGroupRelated = 0;
|
||||
long crossGroupRelated = 0;
|
||||
long globallyRelated = 0;
|
||||
long forcedSameGroupRelated = 0;
|
||||
long globallyForcedRelated = 0;
|
||||
final Map<String, List<Integer>> inverted = new HashMap<>();
|
||||
for (int index = 0; index < signatures.size(); index++) {
|
||||
final Map.Entry<Signature, SignatureCount> entry = signatures.get(index);
|
||||
@@ -107,10 +104,13 @@ final class CandidateAwareEvaluator {
|
||||
sameWithin = add(sameWithin, QualityEvaluator.chooseTwo(groupCount), "same-signature group pairs");
|
||||
}
|
||||
sameGroupRelated = add(sameGroupRelated, sameWithin, "same-group related pairs");
|
||||
if (policy == OutputPolicy.ALL_CANDIDATES || entry.getKey().candidates().size() == 1) {
|
||||
crossGroupRelated = add(crossGroupRelated,
|
||||
subtract(QualityEvaluator.chooseTwo(entry.getValue().total()), sameWithin, "same-signature cross pairs"),
|
||||
"cross-group related pairs");
|
||||
globallyRelated = add(globallyRelated, QualityEvaluator.chooseTwo(entry.getValue().total()),
|
||||
"globally related pairs");
|
||||
if (entry.getKey().candidates().size() == 1) {
|
||||
forcedSameGroupRelated = add(forcedSameGroupRelated, sameWithin,
|
||||
"forced same-group related pairs");
|
||||
globallyForcedRelated = add(globallyForcedRelated,
|
||||
QualityEvaluator.chooseTwo(entry.getValue().total()), "globally forced related pairs");
|
||||
}
|
||||
for (String candidate : entry.getKey().candidates()) {
|
||||
inverted.computeIfAbsent(candidate, ignored -> new ArrayList<>()).add(index);
|
||||
@@ -134,20 +134,58 @@ final class CandidateAwareEvaluator {
|
||||
}
|
||||
final long total = multiply(left.total(), right.total(), "different-signature pairs");
|
||||
sameGroupRelated = add(sameGroupRelated, same, "same-group related pairs");
|
||||
if (policy == OutputPolicy.ALL_CANDIDATES) {
|
||||
crossGroupRelated = add(crossGroupRelated, subtract(total, same, "different-signature cross pairs"),
|
||||
"cross-group related pairs");
|
||||
globallyRelated = add(globallyRelated, total, "globally related pairs");
|
||||
}
|
||||
for (GoldStandardCover.DuplicateRelation duplicate : cover.duplicateRelations()) {
|
||||
final Signature left = formSignatures[duplicate.leftFormIndex()];
|
||||
final Signature right = formSignatures[duplicate.rightFormIndex()];
|
||||
if (intersects(left, right)) {
|
||||
sameGroupRelated = subtract(sameGroupRelated, duplicate.extraOccurrences(),
|
||||
"duplicate same-group candidate relations");
|
||||
}
|
||||
if (isForcedCollision(left, right)) {
|
||||
forcedSameGroupRelated = subtract(forcedSameGroupRelated, duplicate.extraOccurrences(),
|
||||
"duplicate forced same-group candidate relations");
|
||||
}
|
||||
}
|
||||
final long wordCount = input.length;
|
||||
final long underPossible = cover.relatedPairs();
|
||||
final long overPossible = subtract(QualityEvaluator.chooseTwo(wordCount), underPossible, "over denominator");
|
||||
final long underError = subtract(underPossible, sameGroupRelated, "candidate under errors");
|
||||
final long overError = policy == OutputPolicy.ALL_CANDIDATES
|
||||
? subtract(globallyRelated, sameGroupRelated, "all-candidate over errors")
|
||||
: subtract(globallyForcedRelated, forcedSameGroupRelated, "any-candidate over errors");
|
||||
return new QualityResult(stemmerName, language, mode, policy,
|
||||
includedGroups.size(), wordCount, singletonRows, pairRows, oneCandidate, multipleCandidates,
|
||||
maximumCandidates, assignments, distinctCandidates.size(), crossGroupRelated, overPossible,
|
||||
cover.groups().size(), wordCount, cover.singletonRows(), cover.pairRows(),
|
||||
oneCandidate, multipleCandidates, maximumCandidates, assignments,
|
||||
distinctCandidates.size(), overError, overPossible,
|
||||
underError, underPossible, null);
|
||||
}
|
||||
|
||||
/** Returns whether two canonical candidate sets intersect. */
|
||||
private static boolean intersects(final Signature left, final Signature right) {
|
||||
int leftIndex = 0;
|
||||
int rightIndex = 0;
|
||||
while (leftIndex < left.candidates().size() && rightIndex < right.candidates().size()) {
|
||||
final int comparison = left.candidates().get(leftIndex).compareTo(right.candidates().get(rightIndex));
|
||||
if (comparison == 0) {
|
||||
return true;
|
||||
}
|
||||
if (comparison < 0) {
|
||||
leftIndex++;
|
||||
} else {
|
||||
rightIndex++;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Returns whether every independent selection forces the same output. */
|
||||
private static boolean isForcedCollision(final Signature left, final Signature right) {
|
||||
return left.candidates().size() == 1 && right.candidates().size() == 1
|
||||
&& left.candidates().getFirst().equals(right.candidates().getFirst());
|
||||
}
|
||||
|
||||
/** Canonicalizes and validates one adapter candidate collection. */
|
||||
private static Signature signature(final List<String> raw, final String primary, final String stemmer,
|
||||
final String language, final ProcessingMode mode, final OutputPolicy policy,
|
||||
@@ -182,11 +220,12 @@ final class CandidateAwareEvaluator {
|
||||
return Integer.compare(candidates.size(), other.candidates.size());
|
||||
}
|
||||
}
|
||||
/** Aggregated global and per-group frequency of one signature. */
|
||||
/** Aggregated unique-form and per-group membership frequency of one signature. */
|
||||
private static final class SignatureCount {
|
||||
private long total;
|
||||
private final Map<Integer, Long> byGroup = new HashMap<>();
|
||||
/** Adds one word occurrence. */ private void increment(final int group) { total = add(total, 1, "signature frequency"); byGroup.merge(group, 1L, (left, right) -> add(left, right, "signature group frequency")); }
|
||||
/** Adds one unique word form. */ private void incrementTotal() { total = add(total, 1, "signature frequency"); }
|
||||
/** Adds membership in one gold group. */ private void incrementGroup(final int group) { byGroup.merge(group, 1L, (left, right) -> add(left, right, "signature group frequency")); }
|
||||
/** @return global signature frequency */ private long total() { return total; }
|
||||
/** @return mutable internally owned per-group frequencies */ private Map<Integer, Long> byGroup() { return byGroup; }
|
||||
}
|
||||
|
||||
@@ -109,6 +109,24 @@ final class CandidateAwareEvaluatorTest {
|
||||
assertEquals(2, any.underErrorPairs()); assertEquals(any.underErrorPairs(), all.underErrorPairs());
|
||||
}
|
||||
|
||||
/** Verifies candidate relations over a gold cover with shared forms. */
|
||||
@Test @DisplayName("Candidate evaluation deduplicates forms and overlapping gold relations")
|
||||
void overlappingGoldCover() throws IOException {
|
||||
final List<GoldStandardGroup> groups = List.of(
|
||||
new GoldStandardGroup(1, List.of("a", "b")),
|
||||
new GoldStandardGroup(2, List.of("a", "b", "c")));
|
||||
final Map<String, String> primary = Map.of("a", "x", "b", "y", "c", "z");
|
||||
final Map<String, List<String>> candidates = Map.of(
|
||||
"a", List.of("x", "shared"), "b", List.of("y", "shared"), "c", List.of("z"));
|
||||
final QualityResult all = CandidateAwareEvaluator.evaluate("Synthetic", "MULTI",
|
||||
ProcessingMode.ALL_WORDS, OutputPolicy.ALL_CANDIDATES, groups, adapter(primary, candidates));
|
||||
assertEquals(3, all.processedWordForms());
|
||||
assertEquals(3, all.underPossiblePairs());
|
||||
assertEquals(2, all.underErrorPairs());
|
||||
assertEquals(0, all.overPossiblePairs());
|
||||
assertEquals(0, all.overErrorPairs());
|
||||
}
|
||||
|
||||
/** Compares the optimized signature algorithm with an independent fixed-seed oracle. */
|
||||
@Test @DisplayName("Optimized candidate metrics equal a deterministic randomized brute-force oracle")
|
||||
void randomizedOracleAgreement() throws IOException {
|
||||
@@ -118,12 +136,21 @@ final class CandidateAwareEvaluatorTest {
|
||||
final List<GoldStandardGroup> groups = new ArrayList<>();
|
||||
final Map<String, String> primary = new HashMap<>();
|
||||
final Map<String, List<String>> candidates = new HashMap<>();
|
||||
final List<String> existingForms = new ArrayList<>();
|
||||
int word = 0;
|
||||
for (int group = 0; group < groupCount; group++) {
|
||||
final List<String> forms = new ArrayList<>();
|
||||
for (int member = 0; member < 1 + random.nextInt(5); member++) {
|
||||
final String form = "w" + word++;
|
||||
final boolean reuse = !existingForms.isEmpty() && random.nextInt(5) == 0;
|
||||
final String form = reuse ? existingForms.get(random.nextInt(existingForms.size())) : "w" + word++;
|
||||
if (forms.contains(form)) {
|
||||
continue;
|
||||
}
|
||||
forms.add(form);
|
||||
if (reuse) {
|
||||
continue;
|
||||
}
|
||||
existingForms.add(form);
|
||||
final String primaryStem = "s" + random.nextInt(7);
|
||||
primary.put(form, primaryStem);
|
||||
final List<String> raw = new ArrayList<>();
|
||||
@@ -204,17 +231,21 @@ final class CandidateAwareEvaluatorTest {
|
||||
/** Enumerates small word pairs independently and returns under error/possible and over error/possible counts. */
|
||||
private static long[] oracle(final List<GoldStandardGroup> groups,
|
||||
final Map<String, List<String>> candidates) {
|
||||
final List<String> forms = new ArrayList<>();
|
||||
final List<Integer> labels = new ArrayList<>();
|
||||
final Map<String, Set<Integer>> memberships = new java.util.LinkedHashMap<>();
|
||||
for (int group = 0; group < groups.size(); group++) {
|
||||
for (String form : groups.get(group).forms()) { forms.add(form); labels.add(group); }
|
||||
for (String form : groups.get(group).forms()) {
|
||||
memberships.computeIfAbsent(form, ignored -> new LinkedHashSet<>()).add(group);
|
||||
}
|
||||
}
|
||||
final List<String> forms = List.copyOf(memberships.keySet());
|
||||
long underError = 0; long underPossible = 0; long overError = 0; long overPossible = 0; long anyOverError = 0;
|
||||
for (int left = 0; left < forms.size(); left++) {
|
||||
for (int right = left + 1; right < forms.size(); right++) {
|
||||
final Set<String> intersection = new LinkedHashSet<>(candidates.get(forms.get(left)));
|
||||
intersection.retainAll(new LinkedHashSet<>(candidates.get(forms.get(right))));
|
||||
if (labels.get(left).equals(labels.get(right))) {
|
||||
final Set<Integer> sharedGroups = new LinkedHashSet<>(memberships.get(forms.get(left)));
|
||||
sharedGroups.retainAll(memberships.get(forms.get(right)));
|
||||
if (!sharedGroups.isEmpty()) {
|
||||
underPossible++; if (intersection.isEmpty()) { underError++; }
|
||||
} else {
|
||||
overPossible++; if (!intersection.isEmpty()) { overError++; }
|
||||
|
||||
@@ -56,13 +56,18 @@ final class CandidateQualityAudit {
|
||||
static Scenario evaluate(final Candidate candidate, final ProcessingMode mode,
|
||||
final List<GoldStandardGroup> groups, final QualityResult primary, final QualityResult any,
|
||||
final int limit) throws IOException {
|
||||
final List<String> forms = new ArrayList<>();
|
||||
final List<Integer> groupIndexes = new ArrayList<>();
|
||||
final List<Integer> rows = new ArrayList<>();
|
||||
for (int group = 0; group < groups.size(); group++) {
|
||||
final GoldStandardGroup item = groups.get(group);
|
||||
if (!mode.includes(item.forms())) { continue; }
|
||||
for (String form : item.forms()) { forms.add(form); groupIndexes.add(group); rows.add(item.rowNumber()); }
|
||||
final GoldStandardCover cover = GoldStandardCover.create(groups, mode);
|
||||
final List<String> forms = cover.forms();
|
||||
final List<Integer> rows = new ArrayList<>(forms.size());
|
||||
final List<Set<Integer>> memberships = new ArrayList<>(forms.size());
|
||||
for (int index = 0; index < forms.size(); index++) {
|
||||
rows.add(cover.representativeRow(index));
|
||||
memberships.add(new HashSet<>());
|
||||
}
|
||||
for (int group = 0; group < cover.groups().size(); group++) {
|
||||
for (String form : cover.groups().get(group).forms()) {
|
||||
memberships.get(cover.indexOf(form)).add(group);
|
||||
}
|
||||
}
|
||||
final BatchStemmer stemmer = candidate.createStemmer();
|
||||
final String[] primaryOutputs = stemmer.stem(forms.toArray(String[]::new));
|
||||
@@ -78,7 +83,7 @@ final class CandidateQualityAudit {
|
||||
candidateCountDistribution.merge(set.size(), 1L, Math::addExact);
|
||||
for (String value : set) { inverted.computeIfAbsent(value, ignored -> new ArrayList<>()).add(index); }
|
||||
}
|
||||
final QualityResult candidateResult = CandidateAwareEvaluator.evaluate(candidate.name(), candidate.language().name(),
|
||||
final QualityResult candidateResult = CandidateAwareEvaluator.evaluate(candidate.name(), candidate.resultLanguage(),
|
||||
mode, OutputPolicy.ALL_CANDIDATES, groups, candidate.createStemmer());
|
||||
final List<Integer> selected = new ArrayList<>();
|
||||
for (int index = 0; index < forms.size(); index++) { if (candidateSets.get(index).size() > 1) { selected.add(index); } }
|
||||
@@ -92,8 +97,10 @@ final class CandidateQualityAudit {
|
||||
long repaired = 0; long introduced = 0;
|
||||
for (int partner : partners) {
|
||||
final boolean primaryRelated = primaryOutputs[index].equals(primaryOutputs[partner]);
|
||||
if (groupIndexes.get(index).equals(groupIndexes.get(partner)) && !primaryRelated) { repaired++; }
|
||||
if (!groupIndexes.get(index).equals(groupIndexes.get(partner)) && !primaryRelated) { introduced++; }
|
||||
final Set<Integer> sharedMemberships = new HashSet<>(memberships.get(index));
|
||||
sharedMemberships.retainAll(memberships.get(partner));
|
||||
if (!sharedMemberships.isEmpty() && !primaryRelated) { repaired++; }
|
||||
if (sharedMemberships.isEmpty() && !primaryRelated) { introduced++; }
|
||||
}
|
||||
words.add(new Word(rows.get(index), forms.get(index), primaryOutputs[index], candidateSets.get(index),
|
||||
repaired, introduced));
|
||||
@@ -129,8 +136,8 @@ final class CandidateQualityAudit {
|
||||
text.append("- Row ").append(word.row()).append(", form `").append(escape(word.form()))
|
||||
.append("`, primary `").append(escape(word.primary())).append("`, candidates ")
|
||||
.append(word.candidates().stream().map(value -> "`" + escape(value) + "`").toList())
|
||||
.append(", repaired same-group relations ").append(word.repairedUnderRelations())
|
||||
.append(", introduced cross-group relations ").append(word.introducedOverRelations()).append(".\n");
|
||||
.append(", repaired gold-positive relations ").append(word.repairedUnderRelations())
|
||||
.append(", introduced gold-negative relations ").append(word.introducedOverRelations()).append(".\n");
|
||||
}
|
||||
text.append('\n');
|
||||
}
|
||||
|
||||
@@ -0,0 +1,191 @@
|
||||
/*******************************************************************************
|
||||
* Copyright (C) 2026, Leo Galambos
|
||||
* All rights reserved.
|
||||
*
|
||||
* Redistribution and use in source and binary forms, with or without
|
||||
* modification, are permitted provided that the following conditions are met:
|
||||
*
|
||||
* 1. Redistributions of source code must retain the above copyright notice,
|
||||
* this list of conditions and the following disclaimer.
|
||||
*
|
||||
* 2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
* this list of conditions and the following disclaimer in the documentation
|
||||
* and/or other materials provided with the distribution.
|
||||
*
|
||||
* 3. Neither the name of the copyright holder nor the names of its contributors
|
||||
* may be used to endorse or promote products derived from this software
|
||||
* without specific prior written permission.
|
||||
*
|
||||
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||
* POSSIBILITY OF SUCH DAMAGE.
|
||||
******************************************************************************/
|
||||
package org.egothor.stemmer.benchmark.quality;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* Immutable overlapping gold-standard cover over unique surface forms.
|
||||
*
|
||||
* <p>A form may belong to several dictionary groups. Two distinct forms are a
|
||||
* gold-positive pair when they share at least one included group, and the pair
|
||||
* is counted once even when it shares several groups.</p>
|
||||
*/
|
||||
final class GoldStandardCover {
|
||||
private final List<GoldStandardGroup> groups;
|
||||
private final List<String> forms;
|
||||
private final List<Integer> representativeRows;
|
||||
private final Map<String, Integer> formIndexes;
|
||||
private final List<DuplicateRelation> duplicateRelations;
|
||||
private final long singletonRows;
|
||||
private final long pairRows;
|
||||
private final long relatedPairs;
|
||||
|
||||
/** Builds the cover selected by one processing mode. */
|
||||
static GoldStandardCover create(final Iterable<GoldStandardGroup> source, final ProcessingMode mode) {
|
||||
final List<GoldStandardGroup> groups = new ArrayList<>();
|
||||
final Map<String, Integer> membershipCounts = new HashMap<>();
|
||||
final LinkedHashMap<String, Integer> representativeRows = new LinkedHashMap<>();
|
||||
long singletonRows = 0;
|
||||
long pairRows = 0;
|
||||
long rawRelatedPairs = 0;
|
||||
for (GoldStandardGroup group : source) {
|
||||
if (!mode.includes(group.forms())) {
|
||||
continue;
|
||||
}
|
||||
groups.add(group);
|
||||
if (group.forms().size() == 1) {
|
||||
singletonRows = add(singletonRows, 1, "singleton dictionary rows");
|
||||
} else {
|
||||
pairRows = add(pairRows, 1, "dictionary rows contributing related pairs");
|
||||
}
|
||||
rawRelatedPairs = add(rawRelatedPairs, QualityEvaluator.chooseTwo(group.forms().size()),
|
||||
"raw gold-related pairs");
|
||||
for (String form : group.forms()) {
|
||||
membershipCounts.merge(form, 1, Math::addExact);
|
||||
representativeRows.putIfAbsent(form, group.rowNumber());
|
||||
}
|
||||
}
|
||||
|
||||
final List<String> forms = List.copyOf(representativeRows.keySet());
|
||||
final Map<String, Integer> formIndexes = new HashMap<>(forms.size() * 2);
|
||||
final List<Integer> rows = new ArrayList<>(forms.size());
|
||||
for (int index = 0; index < forms.size(); index++) {
|
||||
final String form = forms.get(index);
|
||||
formIndexes.put(form, index);
|
||||
rows.add(representativeRows.get(form));
|
||||
}
|
||||
|
||||
final Set<Long> seenRelations = new HashSet<>();
|
||||
final Map<Long, Integer> extraOccurrences = new HashMap<>();
|
||||
for (GoldStandardGroup group : groups) {
|
||||
final List<Integer> repeated = new ArrayList<>();
|
||||
for (String form : group.forms()) {
|
||||
if (membershipCounts.get(form) > 1) {
|
||||
repeated.add(formIndexes.get(form));
|
||||
}
|
||||
}
|
||||
for (int left = 0; left < repeated.size(); left++) {
|
||||
for (int right = left + 1; right < repeated.size(); right++) {
|
||||
final long key = pairKey(repeated.get(left), repeated.get(right));
|
||||
if (!seenRelations.add(key)) {
|
||||
extraOccurrences.merge(key, 1, Math::addExact);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
final List<DuplicateRelation> duplicates = new ArrayList<>(extraOccurrences.size());
|
||||
long duplicateCount = 0;
|
||||
for (Map.Entry<Long, Integer> entry : extraOccurrences.entrySet()) {
|
||||
final long key = entry.getKey();
|
||||
final int extra = entry.getValue();
|
||||
duplicates.add(new DuplicateRelation((int) (key >>> 32), (int) key, extra));
|
||||
duplicateCount = add(duplicateCount, extra, "duplicate gold-relation occurrences");
|
||||
}
|
||||
duplicates.sort(null);
|
||||
return new GoldStandardCover(List.copyOf(groups), forms, List.copyOf(rows), Map.copyOf(formIndexes),
|
||||
List.copyOf(duplicates), singletonRows, pairRows,
|
||||
subtract(rawRelatedPairs, duplicateCount, "unique gold-related pairs"));
|
||||
}
|
||||
|
||||
private GoldStandardCover(final List<GoldStandardGroup> groups, final List<String> forms,
|
||||
final List<Integer> representativeRows, final Map<String, Integer> formIndexes,
|
||||
final List<DuplicateRelation> duplicateRelations, final long singletonRows,
|
||||
final long pairRows, final long relatedPairs) {
|
||||
this.groups = groups;
|
||||
this.forms = forms;
|
||||
this.representativeRows = representativeRows;
|
||||
this.formIndexes = formIndexes;
|
||||
this.duplicateRelations = duplicateRelations;
|
||||
this.singletonRows = singletonRows;
|
||||
this.pairRows = pairRows;
|
||||
this.relatedPairs = relatedPairs;
|
||||
}
|
||||
|
||||
/** Returns the included source groups. */
|
||||
List<GoldStandardGroup> groups() { return groups; }
|
||||
/** Returns every included surface form exactly once. */
|
||||
List<String> forms() { return forms; }
|
||||
/** Returns a source row suitable for diagnostics for one unique form. */
|
||||
int representativeRow(final int formIndex) { return representativeRows.get(formIndex); }
|
||||
/** Returns the unique index of a surface form. */
|
||||
int indexOf(final String form) { return formIndexes.get(form); }
|
||||
/** Returns relations repeated by more than one group. */
|
||||
List<DuplicateRelation> duplicateRelations() { return duplicateRelations; }
|
||||
/** Returns the number of included singleton rows. */
|
||||
long singletonRows() { return singletonRows; }
|
||||
/** Returns the number of included rows containing a relation. */
|
||||
long pairRows() { return pairRows; }
|
||||
/** Returns the number of unique gold-positive form pairs. */
|
||||
long relatedPairs() { return relatedPairs; }
|
||||
|
||||
/** Encodes an unordered pair of non-negative form indexes. */
|
||||
private static long pairKey(final int first, final int second) {
|
||||
final int left = Math.min(first, second);
|
||||
final int right = Math.max(first, second);
|
||||
return ((long) left << 32) | (right & 0xffffffffL);
|
||||
}
|
||||
|
||||
/** Checked addition with metric context. */
|
||||
private static long add(final long left, final long right, final String context) {
|
||||
try {
|
||||
return Math.addExact(left, right);
|
||||
} catch (ArithmeticException exception) {
|
||||
throw new IllegalStateException("Arithmetic overflow in " + context + ".", exception);
|
||||
}
|
||||
}
|
||||
|
||||
/** Checked subtraction with metric context. */
|
||||
private static long subtract(final long left, final long right, final String context) {
|
||||
try {
|
||||
return Math.subtractExact(left, right);
|
||||
} catch (ArithmeticException exception) {
|
||||
throw new IllegalStateException("Arithmetic overflow in " + context + ".", exception);
|
||||
}
|
||||
}
|
||||
|
||||
/** One relation counted by more than one source group. */
|
||||
record DuplicateRelation(int leftFormIndex, int rightFormIndex, int extraOccurrences)
|
||||
implements Comparable<DuplicateRelation> {
|
||||
/** Orders relations deterministically by their form indexes. */
|
||||
@Override
|
||||
public int compareTo(final DuplicateRelation other) {
|
||||
final int leftComparison = Integer.compare(leftFormIndex, other.leftFormIndex);
|
||||
return leftComparison != 0 ? leftComparison : Integer.compare(rightFormIndex, other.rightFormIndex);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -37,26 +37,33 @@ import java.util.OptionalDouble;
|
||||
* Undefined ratios are represented by empty optionals; no method returns NaN or infinity.
|
||||
*/
|
||||
record PairwiseMetrics(long truePositivePairs, long falsePositivePairs, long falseNegativePairs,
|
||||
long trueNegativePairs) {
|
||||
long trueNegativePairs, boolean coherentConfusionMatrix) {
|
||||
/** Creates metrics for one coherent binary relation. */
|
||||
PairwiseMetrics(final long truePositivePairs, final long falsePositivePairs,
|
||||
final long falseNegativePairs, final long trueNegativePairs) {
|
||||
this(truePositivePairs, falsePositivePairs, falseNegativePairs, trueNegativePairs, true);
|
||||
}
|
||||
|
||||
/** Creates checked confusion counts from one quality result. */
|
||||
static PairwiseMetrics from(final QualityResult result) {
|
||||
return new PairwiseMetrics(Math.subtractExact(result.underPossiblePairs(), result.underErrorPairs()),
|
||||
result.overErrorPairs(), result.underErrorPairs(),
|
||||
Math.subtractExact(result.overPossiblePairs(), result.overErrorPairs()));
|
||||
Math.subtractExact(result.overPossiblePairs(), result.overErrorPairs()),
|
||||
result.outputPolicy() != OutputPolicy.ANY_CANDIDATE);
|
||||
}
|
||||
|
||||
/** @return pairwise precision */ OptionalDouble precision() { return ratio(truePositivePairs, Math.addExact(truePositivePairs, falsePositivePairs)); }
|
||||
/** @return pairwise recall */ OptionalDouble recall() { return ratio(truePositivePairs, Math.addExact(truePositivePairs, falseNegativePairs)); }
|
||||
/** @return pairwise specificity */ OptionalDouble specificity() { return ratio(trueNegativePairs, Math.addExact(trueNegativePairs, falsePositivePairs)); }
|
||||
/** @return pairwise precision */ OptionalDouble precision() { return coherentRatio(truePositivePairs, Math.addExact(truePositivePairs, falsePositivePairs)); }
|
||||
/** @return pairwise recall */ OptionalDouble recall() { return coherentRatio(truePositivePairs, Math.addExact(truePositivePairs, falseNegativePairs)); }
|
||||
/** @return pairwise specificity */ OptionalDouble specificity() { return coherentRatio(trueNegativePairs, Math.addExact(trueNegativePairs, falsePositivePairs)); }
|
||||
/** @return pairwise accuracy, potentially dominated by true negatives */
|
||||
OptionalDouble accuracy() { return ratio(Math.addExact(truePositivePairs, trueNegativePairs), total()); }
|
||||
OptionalDouble accuracy() { return coherentRatio(Math.addExact(truePositivePairs, trueNegativePairs), total()); }
|
||||
/** @return arithmetic mean of recall and specificity */
|
||||
OptionalDouble balancedAccuracy() { return mean(recall(), specificity()); }
|
||||
/** @return pairwise F0.5 */ OptionalDouble f05() { return fBeta(0.25); }
|
||||
/** @return pairwise F1 */ OptionalDouble f1() { return fBeta(1.0); }
|
||||
/** @return pairwise F2 */ OptionalDouble f2() { return fBeta(4.0); }
|
||||
/** @return Jaccard index */
|
||||
OptionalDouble jaccard() { return ratio(truePositivePairs, Math.addExact(Math.addExact(truePositivePairs, falsePositivePairs), falseNegativePairs)); }
|
||||
OptionalDouble jaccard() { return coherentRatio(truePositivePairs, Math.addExact(Math.addExact(truePositivePairs, falsePositivePairs), falseNegativePairs)); }
|
||||
/** @return Fowlkes-Mallows index */
|
||||
OptionalDouble fowlkesMallows() {
|
||||
final OptionalDouble precisionValue = precision(); final OptionalDouble recallValue = recall();
|
||||
@@ -65,6 +72,7 @@ record PairwiseMetrics(long truePositivePairs, long falsePositivePairs, long fal
|
||||
}
|
||||
/** @return Matthews correlation coefficient using scaled double arithmetic */
|
||||
OptionalDouble matthewsCorrelationCoefficient() {
|
||||
if (!coherentConfusionMatrix) { return OptionalDouble.empty(); }
|
||||
final double a = (double) truePositivePairs + falsePositivePairs;
|
||||
final double b = (double) truePositivePairs + falseNegativePairs;
|
||||
final double c = (double) trueNegativePairs + falsePositivePairs;
|
||||
@@ -76,10 +84,11 @@ record PairwiseMetrics(long truePositivePairs, long falsePositivePairs, long fal
|
||||
return OptionalDouble.of(numerator / denominator);
|
||||
}
|
||||
/** @return pairwise error rate */
|
||||
OptionalDouble errorRate() { return ratio(Math.addExact(falsePositivePairs, falseNegativePairs), total()); }
|
||||
OptionalDouble errorRate() { return coherentRatio(Math.addExact(falsePositivePairs, falseNegativePairs), total()); }
|
||||
|
||||
/** Calculates F-beta directly from raw counts. */
|
||||
private OptionalDouble fBeta(final double betaSquared) {
|
||||
if (!coherentConfusionMatrix) { return OptionalDouble.empty(); }
|
||||
final double numerator = (1.0 + betaSquared) * truePositivePairs;
|
||||
final double denominator = numerator + betaSquared * falseNegativePairs + falsePositivePairs;
|
||||
return denominator == 0.0 ? OptionalDouble.empty() : OptionalDouble.of(numerator / denominator);
|
||||
@@ -90,6 +99,10 @@ record PairwiseMetrics(long truePositivePairs, long falsePositivePairs, long fal
|
||||
private static OptionalDouble ratio(final long numerator, final long denominator) {
|
||||
return denominator == 0 ? OptionalDouble.empty() : OptionalDouble.of((double) numerator / denominator);
|
||||
}
|
||||
/** Calculates a ratio only when the counts describe one coherent relation. */
|
||||
private OptionalDouble coherentRatio(final long numerator, final long denominator) {
|
||||
return coherentConfusionMatrix ? ratio(numerator, denominator) : OptionalDouble.empty();
|
||||
}
|
||||
/** Averages two defined ratios. */
|
||||
private static OptionalDouble mean(final OptionalDouble left, final OptionalDouble right) {
|
||||
return left.isEmpty() || right.isEmpty() ? OptionalDouble.empty()
|
||||
|
||||
@@ -70,4 +70,14 @@ final class PairwiseMetricsTest {
|
||||
assertTrue(metrics.precision().isEmpty()); assertTrue(metrics.recall().isEmpty());
|
||||
assertTrue(metrics.matthewsCorrelationCoefficient().isEmpty());
|
||||
}
|
||||
|
||||
/** Verifies oracle-assisted bounds are not misreported as one confusion matrix. */
|
||||
@Test @DisplayName("Oracle-assisted ANY policy suppresses classification aggregates")
|
||||
void oracleAssistedPolicy() {
|
||||
final PairwiseMetrics metrics = new PairwiseMetrics(10, 2, 3, 20, false);
|
||||
assertTrue(metrics.precision().isEmpty());
|
||||
assertTrue(metrics.recall().isEmpty());
|
||||
assertTrue(metrics.f1().isEmpty());
|
||||
assertTrue(metrics.matthewsCorrelationCoefficient().isEmpty());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,28 +63,30 @@ final class QualityAudit {
|
||||
static Scenario evaluate(final Candidate candidate, final ProcessingMode mode,
|
||||
final List<GoldStandardGroup> groups, final int limit) throws IOException {
|
||||
final List<GoldStandardGroup> includedGroups = groups.stream().filter(group -> mode.includes(group.forms())).toList();
|
||||
final List<String> forms = new ArrayList<>();
|
||||
for (GoldStandardGroup group : includedGroups) {
|
||||
forms.addAll(group.forms());
|
||||
}
|
||||
final GoldStandardCover cover = GoldStandardCover.create(groups, mode);
|
||||
final List<String> forms = cover.forms();
|
||||
final String[] outputs = candidate.createStemmer().stem(forms.toArray(String[]::new));
|
||||
if (outputs.length != forms.size()) {
|
||||
throw new IOException("Invalid audit output count for stemmer " + candidate.name() + ", language "
|
||||
+ candidate.language() + ", and processing mode " + mode + ".");
|
||||
+ candidate.resultLanguage() + ", and processing mode " + mode + ".");
|
||||
}
|
||||
final int[] outputIndex = {0};
|
||||
final QualityResult result = QualityEvaluator.evaluate(candidate.name(), candidate.language().name(), mode,
|
||||
groups, word -> outputs[outputIndex[0]++]);
|
||||
final Map<String, String> outputsByForm = new LinkedHashMap<>();
|
||||
for (int index = 0; index < forms.size(); index++) {
|
||||
outputsByForm.put(forms.get(index), outputs[index]);
|
||||
}
|
||||
final QualityResult result = QualityEvaluator.evaluate(candidate.name(), candidate.resultLanguage(), mode,
|
||||
groups, outputsByForm::get);
|
||||
final List<Contributor> contributors = new ArrayList<>();
|
||||
long exactMatches = 0;
|
||||
int offset = 0;
|
||||
long exactDenominator = 0;
|
||||
final List<Integer> sizes = new ArrayList<>();
|
||||
for (GoldStandardGroup group : includedGroups) {
|
||||
final Map<String, List<String>> formsByStem = new LinkedHashMap<>();
|
||||
final String expected = group.forms().get(0);
|
||||
long mergedPairs = 0;
|
||||
for (String form : group.forms()) {
|
||||
final String output = outputs[offset++];
|
||||
exactDenominator++;
|
||||
final String output = outputsByForm.get(form);
|
||||
formsByStem.computeIfAbsent(output, ignored -> new ArrayList<>()).add(form);
|
||||
if (expected.equals(output)) {
|
||||
exactMatches++;
|
||||
@@ -103,16 +105,12 @@ final class QualityAudit {
|
||||
contributors.sort(Comparator.comparingLong(Contributor::errorPairs).reversed()
|
||||
.thenComparingInt(Contributor::rowNumber));
|
||||
final long contributionSum = contributors.stream().mapToLong(Contributor::errorPairs).reduce(0L, Math::addExact);
|
||||
if (contributionSum != result.underErrorPairs()) {
|
||||
throw new IOException("The summed group contributions do not equal the optimized under-stemming total for "
|
||||
+ candidate.name() + ", " + candidate.language() + ", and " + mode + ".");
|
||||
}
|
||||
sizes.sort(Integer::compareTo);
|
||||
final double mean = sizes.stream().mapToInt(Integer::intValue).average().orElse(0.0);
|
||||
final double median = median(sizes);
|
||||
final String resource = org.egothor.stemmer.StemmerModelRegistry.fromContextClassLoader()
|
||||
.requireDefault(candidate.language()).resource();
|
||||
return new Scenario(result, resource, exactMatches, forms.size(),
|
||||
.require(candidate.dictionaryModelId()).resource();
|
||||
return new Scenario(result, resource, exactMatches, exactDenominator,
|
||||
sizes.isEmpty() ? 0 : sizes.get(0), sizes.isEmpty() ? 0 : sizes.get(sizes.size() - 1), mean, median,
|
||||
List.copyOf(contributors.subList(0, Math.min(limit, contributors.size()))), contributionSum);
|
||||
}
|
||||
@@ -136,7 +134,8 @@ final class QualityAudit {
|
||||
.append("- Exact first-field matches: ").append(scenario.exactMatches()).append(" / ").append(scenario.exactDenominator()).append("\n")
|
||||
.append("- Under-stemming pairs: ").append(result.underErrorPairs()).append(" / ").append(result.underPossiblePairs()).append("\n")
|
||||
.append("- Over-stemming pairs: ").append(result.overErrorPairs()).append(" / ").append(result.overPossiblePairs()).append("\n")
|
||||
.append("- Independently summed under-stemming contributions: ").append(scenario.contributionSum()).append("\n\n")
|
||||
.append("- Sum of row-local under-stemming contributions: ").append(scenario.contributionSum())
|
||||
.append(" (shared gold pairs can occur in more than one row)\n\n")
|
||||
.append("### Highest under-stemming contributors\n\n");
|
||||
for (Contributor contributor : scenario.contributors()) {
|
||||
text.append("#### Dictionary row ").append(contributor.rowNumber()).append("\n\n")
|
||||
|
||||
@@ -33,15 +33,14 @@ package org.egothor.stemmer.benchmark.quality;
|
||||
import java.io.IOException;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import org.egothor.stemmer.benchmark.QualityStemmerMatrix.BatchStemmer;
|
||||
|
||||
/** Evaluates pairwise partition agreement using aggregated frequencies, never explicit pairs. */
|
||||
/** Evaluates pairwise agreement with an overlapping gold-standard cover. */
|
||||
public final class QualityEvaluator {
|
||||
/** Utility class. */
|
||||
private QualityEvaluator() { throw new AssertionError("No instances."); }
|
||||
@@ -61,53 +60,52 @@ public final class QualityEvaluator {
|
||||
final Iterable<GoldStandardGroup> groups, final StemmerFunction stemmer) {
|
||||
Objects.requireNonNull(groups, "groups");
|
||||
Objects.requireNonNull(stemmer, "stemmer");
|
||||
return evaluate(stemmerName, language, mode, GoldStandardCover.create(groups, mode), stemmer);
|
||||
}
|
||||
|
||||
/** Evaluates one scenario over a prebuilt overlapping gold-standard cover. */
|
||||
private static QualityResult evaluate(final String stemmerName, final String language,
|
||||
final ProcessingMode mode, final GoldStandardCover cover, final StemmerFunction stemmer) {
|
||||
final Map<String, Long> global = new HashMap<>();
|
||||
long rows = 0;
|
||||
long words = 0;
|
||||
long singletonRows = 0;
|
||||
long pairRows = 0;
|
||||
long underPossible = 0;
|
||||
long withinSameStem = 0;
|
||||
final Set<String> stems = new HashSet<>();
|
||||
final Map<String, Long> local = new HashMap<>();
|
||||
final List<Map<String, Long>> contingency = new ArrayList<>();
|
||||
final List<Long> groupSizes = new ArrayList<>();
|
||||
for (GoldStandardGroup group : groups) {
|
||||
final List<String> forms = group.forms();
|
||||
if (!mode.includes(forms)) {
|
||||
continue;
|
||||
final String[] outputs = new String[cover.forms().size()];
|
||||
for (int index = 0; index < outputs.length; index++) {
|
||||
final String form = cover.forms().get(index);
|
||||
final String output;
|
||||
try {
|
||||
output = stemmer.stem(form);
|
||||
} catch (IOException exception) {
|
||||
throw failure(stemmerName, language, mode, cover.representativeRow(index), form,
|
||||
"the stemmer threw an exception", exception);
|
||||
}
|
||||
rows = add(rows, 1, "applied dictionary rows");
|
||||
words = add(words, forms.size(), "processed word forms");
|
||||
if (forms.size() == 1) {
|
||||
singletonRows = add(singletonRows, 1, "singleton dictionary rows");
|
||||
} else {
|
||||
pairRows = add(pairRows, 1, "dictionary rows contributing under-stemming pairs");
|
||||
if (output == null) {
|
||||
throw failure(stemmerName, language, mode, cover.representativeRow(index), form,
|
||||
"the stemmer returned null", null);
|
||||
}
|
||||
underPossible = add(underPossible, chooseTwo(forms.size()), "under-stemming possible pairs");
|
||||
outputs[index] = output;
|
||||
global.merge(output, 1L, (left, right) -> add(left, right, "global stem frequency"));
|
||||
stems.add(output);
|
||||
}
|
||||
for (GoldStandardGroup group : cover.groups()) {
|
||||
local.clear();
|
||||
for (String form : forms) {
|
||||
final String output;
|
||||
try {
|
||||
output = stemmer.stem(form);
|
||||
} catch (IOException exception) {
|
||||
throw failure(stemmerName, language, mode, group.rowNumber(), form,
|
||||
"the stemmer threw an exception", exception);
|
||||
}
|
||||
if (output == null) {
|
||||
throw failure(stemmerName, language, mode, group.rowNumber(), form,
|
||||
"the stemmer returned null", null);
|
||||
}
|
||||
for (String form : group.forms()) {
|
||||
final String output = outputs[cover.indexOf(form)];
|
||||
local.merge(output, 1L, (left, right) -> add(left, right, "group-to-stem frequency"));
|
||||
global.merge(output, 1L, (left, right) -> add(left, right, "global stem frequency"));
|
||||
stems.add(output);
|
||||
}
|
||||
for (long frequency : local.values()) {
|
||||
withinSameStem = add(withinSameStem, chooseTwo(frequency), "within-group merged pairs");
|
||||
}
|
||||
contingency.add(Map.copyOf(local));
|
||||
groupSizes.add((long) forms.size());
|
||||
}
|
||||
for (GoldStandardCover.DuplicateRelation duplicate : cover.duplicateRelations()) {
|
||||
if (outputs[duplicate.leftFormIndex()].equals(outputs[duplicate.rightFormIndex()])) {
|
||||
withinSameStem = subtract(withinSameStem, duplicate.extraOccurrences(),
|
||||
"duplicate within-group merged pairs");
|
||||
}
|
||||
}
|
||||
final long words = cover.forms().size();
|
||||
final long underPossible = cover.relatedPairs();
|
||||
long allPairs = chooseTwo(words);
|
||||
long overPossible = subtract(allPairs, underPossible, "over-stemming possible pairs");
|
||||
long allSameStem = 0;
|
||||
@@ -116,47 +114,10 @@ public final class QualityEvaluator {
|
||||
}
|
||||
final long underError = subtract(underPossible, withinSameStem, "under-stemming error pairs");
|
||||
final long overError = subtract(allSameStem, withinSameStem, "over-stemming error pairs");
|
||||
final PartitionMetrics partition = partitionMetrics(words, underPossible, allSameStem,
|
||||
withinSameStem, groupSizes, global, contingency);
|
||||
return new QualityResult(stemmerName, language, mode, OutputPolicy.PRIMARY_OUTPUT,
|
||||
rows, words, singletonRows, pairRows, words, 0, words == 0 ? 0 : 1, words, stems.size(), overError, overPossible,
|
||||
underError, underPossible, partition);
|
||||
}
|
||||
|
||||
/** Calculates strict-partition metrics from the exact contingency table. */
|
||||
private static PartitionMetrics partitionMetrics(final long words, final long rowPairs, final long columnPairs,
|
||||
final long indexPairs, final List<Long> groupSizes, final Map<String, Long> global,
|
||||
final List<Map<String, Long>> contingency) {
|
||||
if (words == 0) { return new PartitionMetrics(0.0, 0.0, 0.0, 0.0, 0.0); }
|
||||
final double totalPairs = chooseTwo(words);
|
||||
final double expected = totalPairs == 0.0 ? 0.0 : (double) rowPairs * columnPairs / totalPairs;
|
||||
final double maximum = (rowPairs + (double) columnPairs) / 2.0;
|
||||
final double adjustedRand = maximum == expected ? 1.0 : (indexPairs - expected) / (maximum - expected);
|
||||
final double goldEntropy = entropy(words, groupSizes);
|
||||
final double predictedEntropy = entropy(words, global.values());
|
||||
double mutualInformation = 0.0;
|
||||
for (int group = 0; group < contingency.size(); group++) {
|
||||
final long groupSize = groupSizes.get(group);
|
||||
for (Map.Entry<String, Long> cell : contingency.get(group).entrySet()) {
|
||||
final double frequency = cell.getValue();
|
||||
mutualInformation += frequency / words * Math.log(frequency * words
|
||||
/ (groupSize * (double) global.get(cell.getKey())));
|
||||
}
|
||||
}
|
||||
final double homogeneity = goldEntropy == 0.0 ? 1.0 : mutualInformation / goldEntropy;
|
||||
final double completeness = predictedEntropy == 0.0 ? 1.0 : mutualInformation / predictedEntropy;
|
||||
final double vMeasure = homogeneity + completeness == 0.0 ? 0.0
|
||||
: 2.0 * homogeneity * completeness / (homogeneity + completeness);
|
||||
final double nmiDenominator = (goldEntropy + predictedEntropy) / 2.0;
|
||||
final double nmi = nmiDenominator == 0.0 ? 1.0 : mutualInformation / nmiDenominator;
|
||||
return new PartitionMetrics(adjustedRand, homogeneity, completeness, vMeasure, nmi);
|
||||
}
|
||||
|
||||
/** Calculates natural-log entropy from category frequencies. */
|
||||
private static double entropy(final long total, final Iterable<Long> frequencies) {
|
||||
double weightedLogs = 0.0;
|
||||
for (long frequency : frequencies) { weightedLogs += frequency * Math.log(frequency); }
|
||||
return Math.log(total) - weightedLogs / total;
|
||||
cover.groups().size(), words, cover.singletonRows(), cover.pairRows(), words, 0,
|
||||
words == 0 ? 0 : 1, words, stems.size(), overError, overPossible,
|
||||
underError, underPossible, null);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -175,19 +136,14 @@ public final class QualityEvaluator {
|
||||
public static QualityResult evaluateBatch(final String stemmerName, final String language,
|
||||
final ProcessingMode mode, final List<GoldStandardGroup> groups, final BatchStemmer stemmer)
|
||||
throws IOException {
|
||||
final List<String> included = new ArrayList<>();
|
||||
for (GoldStandardGroup group : groups) {
|
||||
if (mode.includes(group.forms())) {
|
||||
included.addAll(group.forms());
|
||||
}
|
||||
}
|
||||
final String[] outputs = stemmer.stem(included.toArray(String[]::new));
|
||||
if (outputs == null || outputs.length != included.size()) {
|
||||
final GoldStandardCover cover = GoldStandardCover.create(groups, mode);
|
||||
final String[] outputs = stemmer.stem(cover.forms().toArray(String[]::new));
|
||||
if (outputs == null || outputs.length != cover.forms().size()) {
|
||||
throw new IOException("JMH stemmer " + stemmerName + " returned an invalid output batch for language "
|
||||
+ language + " and processing mode " + mode + ".");
|
||||
}
|
||||
final int[] index = {0};
|
||||
return evaluate(stemmerName, language, mode, groups, word -> {
|
||||
return evaluate(stemmerName, language, mode, cover, word -> {
|
||||
final String output = outputs[index[0]++];
|
||||
if (output == null) {
|
||||
throw new IOException("JMH stemmer " + stemmerName + " returned null for language " + language
|
||||
|
||||
@@ -32,6 +32,7 @@ package org.egothor.stemmer.benchmark.quality;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertNull;
|
||||
import static org.junit.jupiter.api.Assertions.assertThrows;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
@@ -58,11 +59,7 @@ final class QualityEvaluatorTest {
|
||||
assertEquals(0, result.overErrorPairs()); assertEquals(4, result.overPossiblePairs());
|
||||
assertEquals(0, result.underErrorPairs()); assertEquals(2, result.underPossiblePairs());
|
||||
assertEquals(2, result.distinctOutputStems());
|
||||
assertEquals(1.0, result.partitionMetrics().adjustedRandIndex(), 1.0e-12);
|
||||
assertEquals(1.0, result.partitionMetrics().homogeneity(), 1.0e-12);
|
||||
assertEquals(1.0, result.partitionMetrics().completeness(), 1.0e-12);
|
||||
assertEquals(1.0, result.partitionMetrics().vMeasure(), 1.0e-12);
|
||||
assertEquals(1.0, result.partitionMetrics().normalizedMutualInformation(), 1.0e-12);
|
||||
assertNull(result.partitionMetrics());
|
||||
}
|
||||
/** Verifies partial merge and pure under-stemming pair counts. */
|
||||
@Test @DisplayName("A partial within-group merge is counted by pairs")
|
||||
@@ -88,13 +85,35 @@ final class QualityEvaluatorTest {
|
||||
assertEquals(2, result.overErrorPairs()); assertEquals(6, result.overPossiblePairs());
|
||||
assertEquals(2, result.underErrorPairs()); assertEquals(4, result.underPossiblePairs());
|
||||
}
|
||||
/** Verifies duplicate scope and singleton undefined denominator. */
|
||||
@Test @DisplayName("Duplicates are removed only within a group and singleton under-stemming is undefined")
|
||||
/** Verifies unique-form identity across overlapping groups. */
|
||||
@Test @DisplayName("The same form in several groups remains one corpus item")
|
||||
void duplicateScope() {
|
||||
final QualityResult result = evaluate(List.of(group(1, "same", "same"), group(2, "same")), Map.of("same", "x"));
|
||||
assertEquals(2, result.processedWordForms()); assertEquals(1, result.overErrorPairs());
|
||||
assertEquals(1, result.processedWordForms()); assertEquals(0, result.overErrorPairs());
|
||||
assertTrue(result.underPercentage().isEmpty());
|
||||
}
|
||||
|
||||
/** Verifies a form can participate in several gold relations without duplication. */
|
||||
@Test @DisplayName("Overlapping group memberships define a deduplicated gold relation")
|
||||
void overlappingMemberships() {
|
||||
final QualityResult result = evaluate(List.of(group(1, "a", "x"), group(2, "a", "y")),
|
||||
Map.of("a", "s", "x", "s", "y", "t"));
|
||||
assertEquals(3, result.processedWordForms());
|
||||
assertEquals(2, result.underPossiblePairs());
|
||||
assertEquals(1, result.underErrorPairs());
|
||||
assertEquals(1, result.overPossiblePairs());
|
||||
assertEquals(0, result.overErrorPairs());
|
||||
}
|
||||
|
||||
/** Verifies a pair shared by several groups is counted only once. */
|
||||
@Test @DisplayName("A relation shared by several groups is counted once")
|
||||
void duplicateRelation() {
|
||||
final QualityResult result = evaluate(List.of(group(1, "a", "b"), group(2, "a", "b", "c")),
|
||||
Map.of("a", "s", "b", "s", "c", "t"));
|
||||
assertEquals(3, result.underPossiblePairs());
|
||||
assertEquals(2, result.underErrorPairs());
|
||||
assertEquals(0, result.overPossiblePairs());
|
||||
}
|
||||
/** Verifies the zero over-stemming denominator. */
|
||||
@Test @DisplayName("One gold group has an undefined over-stemming percentage")
|
||||
void zeroOverDenominator() {
|
||||
|
||||
@@ -67,7 +67,7 @@ public final class QualityReportWriter {
|
||||
if (filtered) {
|
||||
text.append("> This is a filtered analytical report and is not the complete JMH candidate matrix.\n\n");
|
||||
}
|
||||
text.append("## Methodology\n\nEach parsed multilingual dictionary row is a gold-standard equivalence class. Exact duplicates are removed only within that row. `PRIMARY_OUTPUT` is the deterministic JMH partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: within-row sets must intersect, while a cross-row error occurs only for two equal singleton sets. `ALL_CANDIDATES` activates the complete overlap relation: within-row disjoint sets are false negatives and cross-row intersections are false positives. A shared pair is counted once. Candidate policies need not define partitions.\n\nTP is a related within-row pair, FN is an unrelated within-row pair, FP is a related cross-row pair, and TN is an unrelated cross-row pair. Under-stemming is FN/(TP+FN); over-stemming is FP/(TN+FP), so their denominators differ. F0.5 emphasizes precision, F1 balances precision and recall, and F2 emphasizes recall. Undefined values are `n/a`. Percentages and scores use `Locale.ROOT`.\n\n| Stemmer | Language | Dictionary mode | Output policy | Applied dictionary rows | Processed word forms | Distinct output stems | Over-stemming | Under-stemming | Pairwise F0.5 | Pairwise F1 | Pairwise F2 |\n|---|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|\n");
|
||||
text.append("## Methodology\n\nEach distinct surface form is one evaluated item and may belong to several parsed dictionary groups. Two forms are gold-related when their membership sets intersect; a pair sharing several groups is counted once. `PRIMARY_OUTPUT` uses equality of deterministic JMH outputs. `ANY_CANDIDATE` is an optimistic oracle-assisted bound: a gold-related pair succeeds when candidate sets intersect, while a gold-negative error is unavoidable only for two equal singleton sets. `ALL_CANDIDATES` activates the complete candidate-intersection relation.\n\nUnder-stemming is the Paice Understemming Index `FN/(TP+FN)` and over-stemming is the Paice Overstemming Index `FP/(TN+FP)`, generalized here from a disjoint lemma partition to the documented overlapping gold relation. F0.5, F1, MCC, and other classification metrics require one coherent predicted relation and are therefore `n/a` for `ANY_CANDIDATE`. Standard partition metrics are not calculated because the gold memberships overlap. Undefined values are `n/a`. Percentages and scores use `Locale.ROOT`.\n\n| Stemmer | Language | Dictionary mode | Output policy | Applied dictionary rows | Processed word forms | Distinct output stems | Over-stemming | Under-stemming | Pairwise F0.5 | Pairwise F1 | Pairwise F2 |\n|---|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|\n");
|
||||
for (QualityResult row : rows) {
|
||||
text.append("| ").append(escapeMarkdown(row.stemmer())).append(TABLE_DELIMITER)
|
||||
.append(escapeMarkdown(row.language())).append(TABLE_DELIMITER).append(row.processingMode()).append(TABLE_DELIMITER)
|
||||
@@ -92,10 +92,12 @@ public final class QualityReportWriter {
|
||||
/** Writes machine-readable counts and separate percentage fields. */
|
||||
public static void writeCsv(final Path path, final Iterable<QualityResult> input) throws IOException {
|
||||
final StringBuilder text = new StringBuilder(4096);
|
||||
text.append("Stemmer,Language,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,Forms with multiple candidates,Maximum candidates for one form,Total candidate assignments,Distinct output stems,True-positive pairs,False-positive pairs,False-negative pairs,True-negative pairs,Over-stemming error pairs,Over-stemming possible pairs,Over-stemming percentage,Under-stemming error pairs,Under-stemming possible pairs,Under-stemming percentage,Pairwise precision,Pairwise recall,Pairwise specificity,Pairwise accuracy,Balanced accuracy,Pairwise F0.5,Pairwise F1,Pairwise F2,Jaccard index,Fowlkes-Mallows index,Matthews correlation coefficient,Pairwise error rate,Adjusted Rand Index,Homogeneity,Completeness,V-measure,Normalized mutual information\n");
|
||||
text.append("Stemmer,Language,Dictionary model ID,Dictionary model version,Dictionary model SHA-256,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,Forms with multiple candidates,Maximum candidates for one form,Total candidate assignments,Distinct output stems,True-positive pairs,False-positive pairs,False-negative pairs,True-negative pairs,Over-stemming error pairs,Over-stemming possible pairs,Over-stemming percentage,Under-stemming error pairs,Under-stemming possible pairs,Under-stemming percentage,Pairwise precision,Pairwise recall,Pairwise specificity,Pairwise accuracy,Balanced accuracy,Pairwise F0.5,Pairwise F1,Pairwise F2,Jaccard index,Fowlkes-Mallows index,Matthews correlation coefficient,Pairwise error rate,Adjusted Rand Index,Homogeneity,Completeness,V-measure,Normalized mutual information\n");
|
||||
for (QualityResult row : sorted(input)) {
|
||||
final PairwiseMetrics metrics = row.pairwiseMetrics();
|
||||
appendCsv(text, row.stemmer()); appendCsv(text, row.language()); appendCsv(text, row.processingMode().name());
|
||||
appendCsv(text, row.stemmer()); appendCsv(text, row.language());
|
||||
appendCsv(text, row.dictionaryModelId()); appendCsv(text, row.dictionaryModelVersion());
|
||||
appendCsv(text, row.dictionaryModelSha256()); appendCsv(text, row.processingMode().name());
|
||||
appendCsv(text, row.outputPolicy().name());
|
||||
appendCsv(text, Long.toString(row.appliedDictionaryRows())); appendCsv(text, Long.toString(row.processedWordForms()));
|
||||
appendCsv(text, Long.toString(row.singletonDictionaryRows()));
|
||||
@@ -104,8 +106,11 @@ public final class QualityReportWriter {
|
||||
appendCsv(text, Long.toString(row.maximumCandidatesForOneWord()));
|
||||
appendCsv(text, Long.toString(row.totalCandidateAssignments()));
|
||||
appendCsv(text, Long.toString(row.distinctOutputStems()));
|
||||
appendCsv(text, Long.toString(metrics.truePositivePairs())); appendCsv(text, Long.toString(metrics.falsePositivePairs()));
|
||||
appendCsv(text, Long.toString(metrics.falseNegativePairs())); appendCsv(text, Long.toString(metrics.trueNegativePairs()));
|
||||
final boolean confusionMatrix = row.outputPolicy() != OutputPolicy.ANY_CANDIDATE;
|
||||
appendCsv(text, confusionMatrix ? Long.toString(metrics.truePositivePairs()) : "");
|
||||
appendCsv(text, confusionMatrix ? Long.toString(metrics.falsePositivePairs()) : "");
|
||||
appendCsv(text, confusionMatrix ? Long.toString(metrics.falseNegativePairs()) : "");
|
||||
appendCsv(text, confusionMatrix ? Long.toString(metrics.trueNegativePairs()) : "");
|
||||
appendCsv(text, Long.toString(row.overErrorPairs()));
|
||||
appendCsv(text, Long.toString(row.overPossiblePairs())); appendCsv(text, machinePercent(row.overPercentage()));
|
||||
appendCsv(text, Long.toString(row.underErrorPairs())); appendCsv(text, Long.toString(row.underPossiblePairs()));
|
||||
@@ -170,7 +175,7 @@ public final class QualityReportWriter {
|
||||
.append("- Actual result rows: ").append(actualRows).append("\n\n")
|
||||
.append("Unsupported third-party combinations are excluded because their authoritative JMH adapter metadata declares no mapping for that language. They are not emitted as zero-valued rows. Radixor is independently registered for every reconciled dictionary language.\n");
|
||||
final java.util.Map<String, Set<String>> support = new java.util.TreeMap<>();
|
||||
for (Candidate candidate : candidates) { support.computeIfAbsent(candidate.name(), ignored -> new TreeSet<>()).add(candidate.language().name()); }
|
||||
for (Candidate candidate : candidates) { support.computeIfAbsent(candidate.name(), ignored -> new TreeSet<>()).add(candidate.resultLanguage()); }
|
||||
text.append("\n| Adapter | Supported language count | Supported languages |\n|---|---:|---|\n");
|
||||
support.forEach((name, languages) -> text.append("| ").append(escapeMarkdown(name)).append(TABLE_DELIMITER)
|
||||
.append(languages.size()).append(TABLE_DELIMITER).append(languages).append(" |\n"));
|
||||
@@ -230,7 +235,8 @@ public final class QualityReportWriter {
|
||||
if (metrics.f1().isPresent()) { macroF1 += metrics.f1().getAsDouble(); macroCount++; }
|
||||
languages.add(row.language());
|
||||
}
|
||||
final PairwiseMetrics micro = new PairwiseMetrics(tp, fp, fn, tn);
|
||||
final PairwiseMetrics micro = new PairwiseMetrics(tp, fp, fn, tn,
|
||||
first.outputPolicy() != OutputPolicy.ANY_CANDIDATE);
|
||||
text.append("| ").append(escapeMarkdown(first.stemmer())).append(TABLE_DELIMITER).append(first.processingMode())
|
||||
.append(TABLE_DELIMITER).append(first.outputPolicy()).append(TABLE_DELIMITER).append(languages.size())
|
||||
.append(TABLE_DELIMITER).append(score(micro.f05())).append(TABLE_DELIMITER).append(score(micro.f1()))
|
||||
|
||||
@@ -71,7 +71,7 @@ final class QualityReportWriterTest {
|
||||
final Path report = this.temporaryDirectory.resolve("report.csv");
|
||||
QualityReportWriter.writeCsv(report, List.of(result("Stemmer, \"quoted\"", "A", 0, 0)));
|
||||
final String text = Files.readString(report, StandardCharsets.UTF_8);
|
||||
assertTrue(text.startsWith("Stemmer,Language,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,"));
|
||||
assertTrue(text.startsWith("Stemmer,Language,Dictionary model ID,Dictionary model version,Dictionary model SHA-256,Dictionary mode,Output policy,Applied dictionary rows,Processed word forms,Singleton dictionary rows,Forms with one candidate,"));
|
||||
assertTrue(text.contains("\"Stemmer, \"\"quoted\"\"\""));
|
||||
assertTrue(text.contains("Adjusted Rand Index,Homogeneity,Completeness,V-measure,Normalized mutual information"));
|
||||
}
|
||||
|
||||
@@ -35,7 +35,9 @@ import java.util.Objects;
|
||||
import java.util.OptionalDouble;
|
||||
|
||||
/** Immutable pairwise stemming-quality result; all pair quantities are counts. */
|
||||
public record QualityResult(String stemmer, String language, ProcessingMode processingMode,
|
||||
public record QualityResult(String stemmer, String language,
|
||||
String dictionaryModelId, String dictionaryModelVersion, String dictionaryModelSha256,
|
||||
ProcessingMode processingMode,
|
||||
OutputPolicy outputPolicy,
|
||||
long appliedDictionaryRows, long processedWordForms, long singletonDictionaryRows,
|
||||
long dictionaryRowsContributingUnderPairs, long formsWithOneCandidate, long formsWithMultipleCandidates,
|
||||
@@ -51,6 +53,9 @@ public record QualityResult(String stemmer, String language, ProcessingMode proc
|
||||
public QualityResult {
|
||||
Objects.requireNonNull(stemmer, "stemmer");
|
||||
Objects.requireNonNull(language, "language");
|
||||
Objects.requireNonNull(dictionaryModelId, "dictionaryModelId");
|
||||
Objects.requireNonNull(dictionaryModelVersion, "dictionaryModelVersion");
|
||||
Objects.requireNonNull(dictionaryModelSha256, "dictionaryModelSha256");
|
||||
Objects.requireNonNull(processingMode, "processingMode");
|
||||
Objects.requireNonNull(outputPolicy, "outputPolicy");
|
||||
final long[] counts = {appliedDictionaryRows, processedWordForms, singletonDictionaryRows,
|
||||
@@ -70,6 +75,32 @@ public record QualityResult(String stemmer, String language, ProcessingMode proc
|
||||
}
|
||||
}
|
||||
|
||||
/** Creates an evaluator result before model provenance is attached by the application. */
|
||||
public QualityResult(final String stemmer, final String language, final ProcessingMode processingMode,
|
||||
final OutputPolicy outputPolicy, final long appliedDictionaryRows, final long processedWordForms,
|
||||
final long singletonDictionaryRows, final long dictionaryRowsContributingUnderPairs,
|
||||
final long formsWithOneCandidate, final long formsWithMultipleCandidates,
|
||||
final long maximumCandidatesForOneWord, final long totalCandidateAssignments,
|
||||
final long distinctOutputStems, final long overErrorPairs, final long overPossiblePairs,
|
||||
final long underErrorPairs, final long underPossiblePairs, final PartitionMetrics partitionMetrics) {
|
||||
this(stemmer, language, "", "", "", processingMode, outputPolicy, appliedDictionaryRows,
|
||||
processedWordForms, singletonDictionaryRows, dictionaryRowsContributingUnderPairs,
|
||||
formsWithOneCandidate, formsWithMultipleCandidates, maximumCandidatesForOneWord,
|
||||
totalCandidateAssignments, distinctOutputStems, overErrorPairs, overPossiblePairs,
|
||||
underErrorPairs, underPossiblePairs, partitionMetrics);
|
||||
}
|
||||
|
||||
/** Returns this result with immutable dictionary-model provenance attached. */
|
||||
public QualityResult withModelProvenance(final String modelId, final String modelVersion,
|
||||
final String modelSha256) {
|
||||
return new QualityResult(stemmer, language, modelId, modelVersion, modelSha256,
|
||||
processingMode, outputPolicy, appliedDictionaryRows, processedWordForms,
|
||||
singletonDictionaryRows, dictionaryRowsContributingUnderPairs, formsWithOneCandidate,
|
||||
formsWithMultipleCandidates, maximumCandidatesForOneWord, totalCandidateAssignments,
|
||||
distinctOutputStems, overErrorPairs, overPossiblePairs, underErrorPairs,
|
||||
underPossiblePairs, partitionMetrics);
|
||||
}
|
||||
|
||||
/** @return over-stemming percentage, or empty when its denominator is zero */
|
||||
public OptionalDouble overPercentage() { return percentage(overErrorPairs, overPossiblePairs); }
|
||||
/** @return under-stemming percentage, or empty when its denominator is zero */
|
||||
|
||||
@@ -37,8 +37,10 @@ import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.EnumSet;
|
||||
import java.util.List;
|
||||
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader.Language;
|
||||
import org.egothor.stemmer.benchmark.QualityStemmerMatrix;
|
||||
import org.egothor.stemmer.benchmark.QualityStemmerMatrix.Candidate;
|
||||
import org.junit.jupiter.api.DisplayName;
|
||||
@@ -57,10 +59,26 @@ final class QualityStemmerMatrixTest {
|
||||
@Test @DisplayName("Candidate discovery is derived from every JMH quality candidate")
|
||||
void discoversEveryCandidate() {
|
||||
final List<Candidate> candidates = QualityStemmerMatrix.candidates();
|
||||
assertEquals(92, candidates.size(), "The current adapter-language matrix size changed; report coverage must be reviewed.");
|
||||
assertEquals(98, candidates.size(), "The current adapter-language matrix size changed; report coverage must be reviewed.");
|
||||
assertTrue(candidates.stream().anyMatch(candidate -> !candidate.name().endsWith("_RADIXOR")));
|
||||
assertTrue(candidates.stream().anyMatch(candidate -> candidate.name().equals("DA_DK_RADIXOR")));
|
||||
assertTrue(candidates.stream().anyMatch(candidate -> candidate.name().equals("YI_RADIXOR")));
|
||||
assertTrue(candidates.stream().anyMatch(candidate -> candidate.name().equals("POLISH_POLIMORF_RADIXOR")
|
||||
&& candidate.resultLanguage().equals("pl-pl-polimorf")));
|
||||
assertTrue(candidates.stream().anyMatch(candidate -> candidate.name().equals("POLISH_LUCENE_STEMPEL_DIRECT")
|
||||
&& candidate.resultLanguage().equals("pl-pl-polimorf")));
|
||||
}
|
||||
|
||||
/** Verifies the publishable complete matrix uses only registered defaults. */
|
||||
@Test @DisplayName("Complete publication selection excludes optional models")
|
||||
void completePublicationSelectionUsesOnlyDefaultModels() {
|
||||
final List<Candidate> candidates = StemmingQualityApplication.selectCandidates(
|
||||
EnumSet.allOf(Language.class), "");
|
||||
assertEquals(92, candidates.size());
|
||||
assertTrue(candidates.stream().allMatch(candidate ->
|
||||
candidate.dictionaryModelId().equals(candidate.language().defaultModelId())));
|
||||
assertTrue(candidates.stream().noneMatch(candidate ->
|
||||
candidate.dictionaryModelId().equals("pl-pl-polimorf")));
|
||||
}
|
||||
|
||||
/** Verifies a complete report row exists for both modes of every discovered candidate. */
|
||||
@@ -69,7 +87,7 @@ final class QualityStemmerMatrixTest {
|
||||
final List<QualityResult> rows = new ArrayList<>();
|
||||
for (Candidate candidate : QualityStemmerMatrix.candidates()) {
|
||||
for (ProcessingMode mode : ProcessingMode.values()) {
|
||||
rows.add(new QualityResult(candidate.name(), candidate.language().name(), mode,
|
||||
rows.add(new QualityResult(candidate.name(), candidate.resultLanguage(), mode,
|
||||
OutputPolicy.PRIMARY_OUTPUT, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0,
|
||||
new PartitionMetrics(1.0, 1.0, 1.0, 1.0, 1.0)));
|
||||
}
|
||||
@@ -77,10 +95,12 @@ final class QualityStemmerMatrixTest {
|
||||
final Path report = this.temporaryDirectory.resolve("matrix.csv");
|
||||
QualityReportWriter.writeCsv(report, rows);
|
||||
final String text = Files.readString(report, StandardCharsets.UTF_8);
|
||||
assertEquals(185, text.lines().count());
|
||||
assertEquals(197, text.lines().count());
|
||||
for (Candidate candidate : QualityStemmerMatrix.candidates()) {
|
||||
assertTrue(text.contains("\"" + candidate.name() + "\",\"" + candidate.language() + "\",\"ALL_WORDS\""));
|
||||
assertTrue(text.contains("\"" + candidate.name() + "\",\"" + candidate.language() + "\",\"LOWERCASE_GROUPS_ONLY\""));
|
||||
final String prefix = "\"" + candidate.name() + "\",\"" + candidate.resultLanguage()
|
||||
+ "\",\"\",\"\",\"\",";
|
||||
assertTrue(text.contains(prefix + "\"ALL_WORDS\""));
|
||||
assertTrue(text.contains(prefix + "\"LOWERCASE_GROUPS_ONLY\""));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -33,7 +33,6 @@ package org.egothor.stemmer.benchmark.quality;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.EnumMap;
|
||||
import java.util.EnumSet;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
@@ -44,6 +43,8 @@ import java.util.Set;
|
||||
import java.util.logging.Level;
|
||||
import java.util.logging.Logger;
|
||||
|
||||
import org.egothor.stemmer.StemmerModelDescriptor;
|
||||
import org.egothor.stemmer.StemmerModelRegistry;
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader.Language;
|
||||
import org.egothor.stemmer.benchmark.QualityStemmerMatrix;
|
||||
import org.egothor.stemmer.benchmark.QualityStemmerMatrix.Candidate;
|
||||
@@ -96,7 +97,7 @@ public final class StemmingQualityApplication {
|
||||
for (ProcessingMode mode : modes) {
|
||||
for (OutputPolicy policy : policies) {
|
||||
if (policy == OutputPolicy.PRIMARY_OUTPUT || multiple) {
|
||||
expected.add(new ResultKey(candidate.name(), candidate.language().name(), mode, policy));
|
||||
expected.add(new ResultKey(candidate.name(), candidate.resultLanguage(), mode, policy));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -104,15 +105,17 @@ public final class StemmingQualityApplication {
|
||||
|
||||
LOGGER.log(Level.INFO, filtered ? "Starting a filtered stemming-quality report."
|
||||
: "Starting the complete stemming-quality report.");
|
||||
final Map<Language, List<GoldStandardGroup>> dictionaries = new EnumMap<>(Language.class);
|
||||
final Map<String, List<GoldStandardGroup>> dictionaries = new HashMap<>();
|
||||
final List<QualityResult> results = new ArrayList<>();
|
||||
final List<QualityAudit.Scenario> audits = new ArrayList<>();
|
||||
final List<CandidateQualityAudit.Scenario> candidateAudits = new ArrayList<>();
|
||||
final StemmerModelRegistry modelRegistry = StemmerModelRegistry.fromContextClassLoader();
|
||||
for (Candidate candidate : candidates) {
|
||||
List<GoldStandardGroup> groups = dictionaries.get(candidate.language());
|
||||
final StemmerModelDescriptor model = modelRegistry.require(candidate.dictionaryModelId());
|
||||
List<GoldStandardGroup> groups = dictionaries.get(candidate.dictionaryModelId());
|
||||
if (groups == null) {
|
||||
groups = BundledGoldStandardLoader.load(candidate.language());
|
||||
dictionaries.put(candidate.language(), groups);
|
||||
groups = BundledGoldStandardLoader.loadModel(candidate.dictionaryModelId());
|
||||
dictionaries.put(candidate.dictionaryModelId(), groups);
|
||||
}
|
||||
for (ProcessingMode mode : modes) {
|
||||
final QualityStemmerMatrix.BatchStemmer primaryStemmer = candidate.createStemmer();
|
||||
@@ -120,27 +123,34 @@ public final class StemmingQualityApplication {
|
||||
if (audit && policies.contains(OutputPolicy.PRIMARY_OUTPUT)) {
|
||||
final QualityAudit.Scenario scenario = QualityAudit.evaluate(candidate, mode, groups, auditLimit);
|
||||
audits.add(scenario);
|
||||
primary = scenario.result();
|
||||
primary = withModelProvenance(scenario.result(), model);
|
||||
} else {
|
||||
primary = QualityEvaluator.evaluateBatch(candidate.name(), candidate.language().name(),
|
||||
mode, groups, primaryStemmer);
|
||||
primary = withModelProvenance(
|
||||
QualityEvaluator.evaluateBatch(candidate.name(), candidate.resultLanguage(),
|
||||
mode, groups, primaryStemmer),
|
||||
model);
|
||||
}
|
||||
if (policies.contains(OutputPolicy.PRIMARY_OUTPUT)) {
|
||||
results.add(primary);
|
||||
logScenario(candidate, mode, OutputPolicy.PRIMARY_OUTPUT);
|
||||
}
|
||||
if (multiOutput.get(candidate)) {
|
||||
final QualityResult anyCandidate = CandidateAwareEvaluator.evaluate(candidate.name(),
|
||||
candidate.language().name(), mode, OutputPolicy.ANY_CANDIDATE, groups, candidate.createStemmer());
|
||||
final QualityResult anyCandidate = withModelProvenance(
|
||||
CandidateAwareEvaluator.evaluate(candidate.name(),
|
||||
candidate.resultLanguage(), mode, OutputPolicy.ANY_CANDIDATE, groups,
|
||||
candidate.createStemmer()),
|
||||
model);
|
||||
final QualityResult allCandidates;
|
||||
if (audit) {
|
||||
final CandidateQualityAudit.Scenario scenario = CandidateQualityAudit.evaluate(
|
||||
candidate, mode, groups, primary, anyCandidate, auditLimit);
|
||||
candidateAudits.add(scenario);
|
||||
allCandidates = scenario.candidate();
|
||||
allCandidates = withModelProvenance(scenario.candidate(), model);
|
||||
} else {
|
||||
allCandidates = CandidateAwareEvaluator.evaluate(candidate.name(), candidate.language().name(),
|
||||
mode, OutputPolicy.ALL_CANDIDATES, groups, candidate.createStemmer());
|
||||
allCandidates = withModelProvenance(
|
||||
CandidateAwareEvaluator.evaluate(candidate.name(), candidate.resultLanguage(),
|
||||
mode, OutputPolicy.ALL_CANDIDATES, groups, candidate.createStemmer()),
|
||||
model);
|
||||
}
|
||||
verifyPolicyInvariants(primary, anyCandidate, allCandidates);
|
||||
if (policies.contains(OutputPolicy.ANY_CANDIDATE)) {
|
||||
@@ -175,15 +185,43 @@ public final class StemmingQualityApplication {
|
||||
LOGGER.log(Level.INFO, "Completed the stemming-quality report with {0} evaluated scenarios.", results.size());
|
||||
}
|
||||
|
||||
/** Selects candidates directly from the authoritative JMH matrix. */
|
||||
private static List<Candidate> selectCandidates(final Set<Language> languages, final String filter) {
|
||||
/** Attaches the exact independently versioned model used by one scenario. */
|
||||
private static QualityResult withModelProvenance(final QualityResult result,
|
||||
final StemmerModelDescriptor model) {
|
||||
return result.withModelProvenance(model.id(), model.version(), model.sha256());
|
||||
}
|
||||
|
||||
/**
|
||||
* Selects candidates directly from the authoritative JMH matrix.
|
||||
*
|
||||
* <p>
|
||||
* An unfiltered publication run evaluates only each language's registered
|
||||
* default model. Optional model variants remain available only through an
|
||||
* explicit stemmer or model-ID filter and therefore cannot enter the complete
|
||||
* documentation snapshot accidentally.
|
||||
* </p>
|
||||
*/
|
||||
static List<Candidate> selectCandidates(final Set<Language> languages, final String filter) {
|
||||
return QualityStemmerMatrix.candidates().stream()
|
||||
.filter(candidate -> languages.contains(candidate.language()))
|
||||
.filter(candidate -> filter.isBlank() || candidate.name().equalsIgnoreCase(filter)
|
||||
|| candidate.name().toUpperCase(Locale.ROOT).endsWith("_" + filter.toUpperCase(Locale.ROOT)))
|
||||
.filter(candidate -> !filter.isBlank()
|
||||
|| candidate.dictionaryModelId().equals(candidate.language().defaultModelId()))
|
||||
.filter(candidate -> matchesFilter(candidate, filter))
|
||||
.toList();
|
||||
}
|
||||
|
||||
/** Tests one candidate against the exact, suffix, and model-ID filters. */
|
||||
private static boolean matchesFilter(final Candidate candidate, final String filter) {
|
||||
if (filter.isBlank()) {
|
||||
return true;
|
||||
}
|
||||
final String normalizedFilter = filter.toUpperCase(Locale.ROOT);
|
||||
final String name = candidate.name().toUpperCase(Locale.ROOT);
|
||||
final String model = candidate.dictionaryModelId().toUpperCase(Locale.ROOT);
|
||||
return name.equals(normalizedFilter) || name.endsWith("_" + normalizedFilter)
|
||||
|| model.equals(normalizedFilter) || model.endsWith("-" + normalizedFilter);
|
||||
}
|
||||
|
||||
/** Parses a comma-separated language filter or selects every language. */
|
||||
private static Set<Language> parseLanguages(final String filter) {
|
||||
if (filter.isBlank()) {
|
||||
@@ -268,7 +306,7 @@ public final class StemmingQualityApplication {
|
||||
private static void logScenario(final Candidate candidate, final ProcessingMode mode, final OutputPolicy policy) {
|
||||
if (LOGGER.isLoggable(Level.INFO)) {
|
||||
LOGGER.log(Level.INFO, "Completed stemming-quality evaluation for stemmer {0}, language {1}, dictionary mode {2}, and output policy {3}.",
|
||||
new Object[] {candidate.name(), candidate.language(), mode, policy});
|
||||
new Object[] {candidate.name(), candidate.resultLanguage(), mode, policy});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -49,6 +49,8 @@ import java.util.Set;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.egothor.stemmer.StemmerPatchTrieLoader.Language;
|
||||
|
||||
/**
|
||||
* Publishes validated stemming-quality CSV results into marked sections of the
|
||||
* existing language benchmark pages. This test-source utility never modifies
|
||||
@@ -182,7 +184,9 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
throw new IllegalStateException("The stemming-quality CSV is empty.");
|
||||
}
|
||||
final List<String> header = parseCsv(lines.getFirst());
|
||||
final List<String> required = List.of("Stemmer", "Language", "Dictionary mode", "Output policy", "Applied dictionary rows",
|
||||
final List<String> required = List.of("Stemmer", "Language", "Dictionary model ID",
|
||||
"Dictionary model version", "Dictionary model SHA-256",
|
||||
"Dictionary mode", "Output policy", "Applied dictionary rows",
|
||||
"Processed word forms", "Forms with multiple candidates", "Maximum candidates for one form", "Total candidate assignments",
|
||||
"True-positive pairs", "False-positive pairs", "False-negative pairs", "True-negative pairs",
|
||||
"Over-stemming error pairs", "Over-stemming possible pairs", "Over-stemming percentage", "Under-stemming error pairs",
|
||||
@@ -244,6 +248,14 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
if (!keys.add(row.key())) {
|
||||
throw new IllegalStateException("Duplicate stemming-quality result key: " + row.key());
|
||||
}
|
||||
final String expectedModelId = Language.valueOf(row.language()).defaultModelId();
|
||||
if (!row.modelId().equals(expectedModelId)) {
|
||||
throw new IllegalStateException("Stemming-quality row " + row.key()
|
||||
+ " uses model " + row.modelId() + " instead of default model " + expectedModelId + ".");
|
||||
}
|
||||
if (row.modelVersion().isBlank() || !row.modelSha256().matches("[0-9a-f]{64}")) {
|
||||
throw new IllegalStateException("Incomplete dictionary-model provenance for " + row.key() + ".");
|
||||
}
|
||||
row.validate();
|
||||
}
|
||||
final Set<String> resultLanguages = new HashSet<>();
|
||||
@@ -275,8 +287,9 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
}
|
||||
validatePolicies(languageRows);
|
||||
}
|
||||
if (!documentedLanguages.contains("DA_DK") || !documentedLanguages.contains("YI")) {
|
||||
throw new IllegalStateException("The documentation mapping must contain DA_DK and YI.");
|
||||
if (!documentedLanguages.contains("DA_DK") || !documentedLanguages.contains("HE_IL")
|
||||
|| !documentedLanguages.contains("YI")) {
|
||||
throw new IllegalStateException("The documentation mapping must contain DA_DK, HE_IL, and YI.");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -304,13 +317,15 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
|
||||
/** Renders one complete generated section for a language page. */
|
||||
private static String render(final Page page, final List<ResultRow> rows, final String checksum) {
|
||||
final String modelId = Language.valueOf(page.language()).defaultModelId();
|
||||
final StringBuilder output = new StringBuilder(32768);
|
||||
output.append(START).append("\n\n## Stemming Quality\n\n")
|
||||
.append("Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `")
|
||||
.append(page.language()).append("` using the complete validated stemming-quality result matrix. Every usable dictionary row is one gold-standard group of forms expected to share a morphological family or lemma. Exact equality with a predetermined lemma is not required. Same-row pairs are positive pairs; pairs from different rows are negative pairs.\n\n")
|
||||
.append(page.language()).append("` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required.\n\n")
|
||||
.append("`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv).\n\n")
|
||||
.append("### Evaluation Scope and Key Findings\n\n")
|
||||
.append("The dictionary resource is `src/main/resources/").append(page.language().toLowerCase(Locale.ROOT)).append("/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.\n\n");
|
||||
.append("The default model is `").append(modelId).append("`, loaded from classpath resource `org/egothor/stemmer/models/")
|
||||
.append(modelId).append("/stemmer.gz`. The following findings compare only deterministic `PRIMARY_OUTPUT` rows over identical included groups; candidate policies are reported separately as capability analyses.\n\n");
|
||||
for (String mode : MODES) {
|
||||
appendFinding(output, rows, mode);
|
||||
}
|
||||
@@ -320,13 +335,17 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
final long policies = selected.stream().map(ResultRow::policy).distinct().count();
|
||||
output.append("### `").append(mode).append("`\n\n")
|
||||
.append("This mode contains **").append(selected.size()).append(" result rows**, **").append(stemmers)
|
||||
.append(" evaluated stemmers**, and **").append(policies).append(" output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Rankings are separated by output policy and ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. Balanced accuracy is a navigation metric, not a universally authoritative quality score.\n\n");
|
||||
.append(" evaluated stemmers**, and **").append(policies).append(" output policies**. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. `PRIMARY_OUTPUT` and `ALL_CANDIDATES` rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. `ANY_CANDIDATE` has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.\n\n");
|
||||
for (String policy : List.of("PRIMARY_OUTPUT", "ANY_CANDIDATE", "ALL_CANDIDATES")) {
|
||||
final List<ResultRow> policyRows = selected.stream().filter(row -> row.policy().equals(policy)).toList();
|
||||
if (!policyRows.isEmpty()) {
|
||||
output.append("#### `").append(policy).append("` ranking\n\n");
|
||||
renderPrimaryTable(output, policyRows);
|
||||
renderDetailedTables(output, policyRows);
|
||||
if (policy.equals("ANY_CANDIDATE")) {
|
||||
renderAnyCandidatePolicy(output, policyRows);
|
||||
} else {
|
||||
output.append("#### `").append(policy).append("` ranking\n\n");
|
||||
renderPrimaryTable(output, policyRows);
|
||||
renderDetailedTables(output, policyRows);
|
||||
}
|
||||
}
|
||||
}
|
||||
renderCandidateAnalysis(output, selected);
|
||||
@@ -335,11 +354,12 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
output.append("### Provenance\n\n")
|
||||
.append("- Authoritative source: `docs/benchmarks/data/stemming-quality.csv`\n")
|
||||
.append("- Source SHA-256: `").append(checksum).append("`\n")
|
||||
.append("- Evaluation command: `./gradlew stemmingQuality`\n")
|
||||
.append("- Evaluation command: `./gradlew stemmingQuality --no-daemon`\n")
|
||||
.append("- Dictionary language: `").append(page.language()).append("`\n")
|
||||
.append("- Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY`\n")
|
||||
.append("- Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and `gradle.lockfile`\n")
|
||||
.append("- Radixor version, Git revision, generation date, JDK version, operating system, and dictionary revision: not recorded in the authoritative CSV\n\n")
|
||||
.append("- Model ID, version, and SHA-256: recorded in every CSV row\n")
|
||||
.append("- Run date, core source state, JDK, operating system, and hardware: recorded on the [benchmark environment page](../reference/environment.md)\n\n")
|
||||
.append(END).append('\n');
|
||||
return output.toString();
|
||||
}
|
||||
@@ -366,24 +386,22 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
output.append(". This rank does not imply leadership in throughput or every secondary metric.\n");
|
||||
}
|
||||
|
||||
/** Renders the compact primary ranking table in an accessible scroll region. */
|
||||
/** Renders the compact primary ranking without duplicating metrics available in the details. */
|
||||
private static void renderPrimaryTable(final StringBuilder output, final List<ResultRow> rows) {
|
||||
output.append("<div class=\"quality-table quality-table--compact\" role=\"region\" aria-label=\"Compact stemming-quality ranking; scroll horizontally for additional columns\" tabindex=\"0\" markdown=\"1\">\n\n")
|
||||
.append("| Rank | Stemmer | Output policy | Balanced accuracy | Over-stemming | Under-stemming | F0.5 | F1 | MCC |\n")
|
||||
.append("|---:|---|---|---:|---:|---:|---:|---:|---:|\n");
|
||||
output.append("<div class=\"quality-summary\" markdown=\"1\">\n\n")
|
||||
.append("| Rank | Stemmer | Balanced accuracy | Over-stemming (OI) | Under-stemming (UI) |\n")
|
||||
.append("|---:|---|---:|---:|---:|\n");
|
||||
for (int index = 0; index < rows.size(); index++) {
|
||||
final ResultRow row = rows.get(index);
|
||||
output.append('|').append(index + 1).append('|').append(displayStemmer(row.stemmer())).append('|').append(row.policy()).append('|')
|
||||
output.append('|').append(index + 1).append('|').append(displayStemmer(row.stemmer())).append('|')
|
||||
.append(metric(row, "Balanced accuracy")).append('|')
|
||||
.append(pair(row, "Over-stemming error pairs", "Over-stemming possible pairs", "Over-stemming percentage")).append('|')
|
||||
.append(pair(row, "Under-stemming error pairs", "Under-stemming possible pairs", "Under-stemming percentage")).append('|')
|
||||
.append(metric(row, "Pairwise F0.5")).append('|').append(metric(row, "Pairwise F1")).append('|')
|
||||
.append(metric(row, "Matthews correlation coefficient")).append("|\n");
|
||||
.append(rate(row, "Over-stemming error pairs", "Over-stemming percentage")).append('|')
|
||||
.append(rate(row, "Under-stemming error pairs", "Under-stemming percentage")).append("|\n");
|
||||
}
|
||||
output.append("\n</div>\n\n");
|
||||
}
|
||||
|
||||
/** Renders classification, relation, partition, and raw-count tables with repeated identities. */
|
||||
/** Renders classification, relation, and raw-count tables with repeated identities. */
|
||||
private static void renderDetailedTables(final StringBuilder output, final List<ResultRow> rows) {
|
||||
output.append("<details class=\"quality-details\" markdown=\"1\"><summary>Classification metrics</summary>\n\n")
|
||||
.append("| Rank | Stemmer | Output policy | Precision | Recall | Specificity | Balanced accuracy | Pairwise accuracy | Error rate |\n")
|
||||
@@ -403,22 +421,40 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
.append(metric(row, "Pairwise F2")).append('|').append(metric(row, "Jaccard index")).append('|')
|
||||
.append(metric(row, "Fowlkes-Mallows index")).append('|').append(metric(row, "Matthews correlation coefficient")).append("|\n");
|
||||
}
|
||||
output.append("\n</details>\n\n<details class=\"quality-details\" markdown=\"1\"><summary>Partition metrics (PRIMARY_OUTPUT only)</summary>\n\n")
|
||||
.append("| Rank | Stemmer | Output policy | Adjusted Rand Index | Homogeneity | Completeness | V-measure | Normalized mutual information |\n")
|
||||
.append("|---:|---|---|---:|---:|---:|---:|---:|\n");
|
||||
for (int index = 0; index < rows.size(); index++) {
|
||||
final ResultRow row = rows.get(index);
|
||||
output.append(identity(index, row)).append(metric(row, "Adjusted Rand Index")).append('|').append(metric(row, "Homogeneity")).append('|')
|
||||
.append(metric(row, "Completeness")).append('|').append(metric(row, "V-measure")).append('|')
|
||||
.append(metric(row, "Normalized mutual information")).append("|\n");
|
||||
}
|
||||
output.append("\n</details>\n\n<details class=\"quality-details\" markdown=\"1\"><summary>Raw pair counts</summary>\n\n")
|
||||
.append("| Rank | Stemmer | Output policy | TP | FP | FN | TN | Over error / possible | Under error / possible |\n")
|
||||
.append("|---:|---|---|---:|---:|---:|---:|---:|---:|\n");
|
||||
for (int index = 0; index < rows.size(); index++) {
|
||||
final ResultRow row = rows.get(index);
|
||||
output.append(identity(index, row)).append(row.value("True-positive pairs")).append('|').append(row.value("False-positive pairs"))
|
||||
.append('|').append(row.value("False-negative pairs")).append('|').append(row.value("True-negative pairs")).append('|')
|
||||
output.append(identity(index, row)).append(rawCount(row, "True-positive pairs")).append('|')
|
||||
.append(rawCount(row, "False-positive pairs")).append('|')
|
||||
.append(rawCount(row, "False-negative pairs")).append('|')
|
||||
.append(rawCount(row, "True-negative pairs")).append('|')
|
||||
.append(row.value("Over-stemming error pairs")).append(" / ").append(row.value("Over-stemming possible pairs")).append('|')
|
||||
.append(row.value("Under-stemming error pairs")).append(" / ").append(row.value("Under-stemming possible pairs")).append("|\n");
|
||||
}
|
||||
output.append("\n</details>\n\n");
|
||||
}
|
||||
|
||||
/** Renders the two defined per-pair oracle bounds without implying one confusion matrix. */
|
||||
private static void renderAnyCandidatePolicy(final StringBuilder output, final List<ResultRow> rows) {
|
||||
final List<ResultRow> alphabetical = rows.stream().sorted(Comparator.comparing(ResultRow::stemmer)).toList();
|
||||
output.append("#### `ANY_CANDIDATE` oracle bounds\n\n")
|
||||
.append("These results are measured, not missing. `ANY_CANDIDATE` answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically **not applicable**, rather than unknown.\n\n")
|
||||
.append("<div class=\"quality-summary quality-summary--oracle\" markdown=\"1\">\n\n")
|
||||
.append("| Stemmer | Optimistic over-stemming (OI) | Optimistic under-stemming (UI) |\n")
|
||||
.append("|---|---:|---:|\n");
|
||||
for (ResultRow row : alphabetical) {
|
||||
output.append('|').append(displayStemmer(row.stemmer())).append('|')
|
||||
.append(rate(row, "Over-stemming error pairs", "Over-stemming percentage")).append('|')
|
||||
.append(rate(row, "Under-stemming error pairs", "Under-stemming percentage")).append("|\n");
|
||||
}
|
||||
output.append("\n</div>\n\n")
|
||||
.append("<details class=\"quality-details\" markdown=\"1\"><summary>Oracle-bound pair counts</summary>\n\n")
|
||||
.append("| Stemmer | Unavoidable over errors / gold-negative pairs | Unrepairable under errors / gold-related pairs |\n")
|
||||
.append("|---|---:|---:|\n");
|
||||
for (ResultRow row : alphabetical) {
|
||||
output.append('|').append(displayStemmer(row.stemmer())).append('|')
|
||||
.append(row.value("Over-stemming error pairs")).append(" / ").append(row.value("Over-stemming possible pairs")).append('|')
|
||||
.append(row.value("Under-stemming error pairs")).append(" / ").append(row.value("Under-stemming possible pairs")).append("|\n");
|
||||
}
|
||||
@@ -464,13 +500,13 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
/** Appends the self-contained policy, confusion-matrix, and metric definitions. */
|
||||
private static void appendMethodology(final StringBuilder output) {
|
||||
output.append("### Output Policies and Metric Definitions\n\n")
|
||||
.append("`PRIMARY_OUTPUT` uses one deterministic stem per form and therefore defines a strict partition. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a same-group pair succeeds when candidates intersect, while a different-group pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and need not represent one globally consistent assignment. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect. Alternatives can reduce under-stemming but can introduce cross-group collisions, and the resulting relation can overlap and need not be a partition.\n\n")
|
||||
.append("For each row, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. TP and FN concern same-group pairs; FP and TN concern different-group pairs. Consequently, under-stemming and over-stemming use different denominators. Undefined values are rendered as `n/a`.\n\n")
|
||||
.append("- Under-stemming rate: `FN / (TP + FN)`, the false-negative rate over same-group pairs.\n")
|
||||
.append("- Over-stemming rate: `FP / (TN + FP)`, the false-positive rate over different-group pairs.\n")
|
||||
.append("Each distinct surface form is one item and may belong to several gold groups. Two forms are gold-related when their membership sets intersect; a relation shared by several groups is counted once. `PRIMARY_OUTPUT` uses one deterministic stem per form. `ANY_CANDIDATE` is an optimistic oracle-assisted pairwise upper bound: a gold-related pair succeeds when candidates intersect, while a gold-negative pair succeeds when a non-colliding selection exists. Candidate choices may differ between pairs, so this is not deterministic runtime behaviour and does not define one confusion matrix. `ALL_CANDIDATES` activates every returned candidate; forms are related when candidate sets intersect.\n\n")
|
||||
.append("For `PRIMARY_OUTPUT` and `ALL_CANDIDATES`, `TP = underPossiblePairs - underErrorPairs`, `FN = underErrorPairs`, `FP = overErrorPairs`, and `TN = overPossiblePairs - overErrorPairs`. `ANY_CANDIDATE` publishes only its separate oracle-assisted under/over bounds; confusion-derived metrics are mathematically inapplicable and are not presented in its language-page section. Their machine-readable CSV fields remain empty. Undefined metric denominators in otherwise applicable policies are rendered as `n/a`.\n\n")
|
||||
.append("- Under-stemming rate (Paice UI): `FN / (TP + FN)`, the false-negative rate over gold-related pairs.\n")
|
||||
.append("- Over-stemming rate (Paice OI): `FP / (TN + FP)`, the false-positive rate over gold-negative pairs.\n")
|
||||
.append("- Pairwise precision: `TP / (TP + FP)`, the fraction of predicted conflations that are gold-standard positive pairs.\n")
|
||||
.append("- Pairwise recall: `TP / (TP + FN)`, the fraction of gold-standard positive pairs successfully connected.\n")
|
||||
.append("- Pairwise specificity: `TN / (TN + FP)`, the fraction of different-group pairs correctly separated.\n")
|
||||
.append("- Pairwise specificity: `TN / (TN + FP)`, the fraction of gold-negative pairs correctly separated.\n")
|
||||
.append("- Balanced accuracy: `(recall + specificity) / 2`. It gives equal weight to positive and negative pair classes and is less dominated by the large true-negative class than ordinary accuracy. It does not replace the raw errors or other metrics.\n")
|
||||
.append("- Pairwise F-beta: `((1 + betaSquared) * TP) / (((1 + betaSquared) * TP) + (betaSquared * FN) + FP)`. F0.5 emphasizes precision and penalizes over-stemming more; F1 weights precision and recall equally; F2 emphasizes recall and penalizes under-stemming more.\n")
|
||||
.append("- MCC: `(TP * TN - FP * FN) / sqrt((TP + FP) * (TP + FN) * (TN + FP) * (TN + FN))`. It uses all confusion counts and remains useful under class imbalance, except when its denominator is degenerate.\n")
|
||||
@@ -478,14 +514,15 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
.append("- Fowlkes–Mallows index: `sqrt(precision * recall)`.\n")
|
||||
.append("- Pairwise accuracy: `(TP + TN) / (TP + TN + FP + FN)`. It can be dominated by true-negative cross-group pairs.\n")
|
||||
.append("- Pairwise error rate: `(FP + FN) / (TP + TN + FP + FN)`.\n\n")
|
||||
.append("Adjusted Rand Index uses the gold/predicted contingency table and chance correction. Homogeneity is `1 - H(gold | predicted) / H(gold)`; completeness is `1 - H(predicted | gold) / H(predicted)`; V-measure is their harmonic mean; normalized mutual information uses the arithmetic-mean entropy normalization `MI / ((H(gold) + H(predicted)) / 2)`. These partition-only metrics apply to `PRIMARY_OUTPUT`; candidate-relation rows show `n/a`.\n\n");
|
||||
.append("Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: their usual contingency-table definitions require an exclusive gold partition, while this gold standard is an overlapping cover.\n\n");
|
||||
}
|
||||
|
||||
/** Renders the generated executive findings, winner matrix, and Radixor aggregates. */
|
||||
private static String renderOverview(final Map<String, Page> pages, final List<ResultRow> rows, final String checksum) {
|
||||
final StringBuilder output = new StringBuilder(16384);
|
||||
output.append(OVERVIEW_START).append("\n\n## Pairwise Quality Findings\n\n")
|
||||
.append("The validated snapshot is a broad multilingual comparison covering the complete 20-language Radixor dictionary universe; 19 languages have existing benchmark pages. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim.\n\n");
|
||||
.append("The validated snapshot is a broad multilingual comparison covering the complete ")
|
||||
.append(pages.size()).append("-language Radixor default-model universe, with one benchmark page per language. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim.\n\n");
|
||||
int radixorWins = 0;
|
||||
int comparisons = 0;
|
||||
for (String mode : MODES) {
|
||||
@@ -524,7 +561,8 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
for (String mode : MODES) {
|
||||
renderPlacementSummary(output, pages, rows, mode);
|
||||
}
|
||||
output.append("\n### Radixor full-coverage aggregates\n\nThese aggregates cover all 19 documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking.\n\n")
|
||||
output.append("\n### Radixor full-coverage aggregates\n\nThese aggregates cover all ")
|
||||
.append(pages.size()).append(" documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking.\n\n")
|
||||
.append("| Dictionary mode | Languages | Macro balanced accuracy | Micro balanced accuracy | Micro precision | Micro recall | Micro F1 |\n")
|
||||
.append("|---|---:|---:|---:|---:|---:|---:|\n");
|
||||
for (String mode : MODES) {
|
||||
@@ -669,12 +707,28 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
return value.isEmpty() ? "n/a" : String.format(Locale.ROOT, "%.6f", Double.parseDouble(value));
|
||||
}
|
||||
|
||||
/** Formats an over- or under-stemming rate as a percentage. */
|
||||
private static String rate(final ResultRow row, final String errorName, final String percentageName) {
|
||||
final String value = row.value(percentageName);
|
||||
if (value.isEmpty()) {
|
||||
return "n/a";
|
||||
}
|
||||
final double rate = Double.parseDouble(value);
|
||||
return rate < 0.000001 && row.longValue(errorName) > 0 ? "<0.000001%" : String.format(Locale.ROOT, "%.6f%%", rate);
|
||||
}
|
||||
|
||||
/** Formats one raw error numerator, denominator, and percentage. */
|
||||
private static String pair(final ResultRow row, final String error, final String possible, final String percentage) {
|
||||
final String rate = row.value(percentage);
|
||||
return row.value(error) + " / " + row.value(possible) + " (" + (rate.isEmpty() ? "n/a" : String.format(Locale.ROOT, "%.6f%%", Double.parseDouble(rate))) + ")";
|
||||
}
|
||||
|
||||
/** Formats an inapplicable confusion count explicitly. */
|
||||
private static String rawCount(final ResultRow row, final String name) {
|
||||
final String value = row.value(name);
|
||||
return value.isEmpty() ? "n/a" : value;
|
||||
}
|
||||
|
||||
/** Replaces an existing marked section or appends the first generated section. */
|
||||
private static String replaceSection(final String original, final String section) {
|
||||
return replaceMarkedSection(original, section, START, END);
|
||||
@@ -725,6 +779,12 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
private String stemmer() { return value("Stemmer"); }
|
||||
/** Returns the language identifier. */
|
||||
private String language() { return value("Language"); }
|
||||
/** Returns the dictionary model identifier. */
|
||||
private String modelId() { return value("Dictionary model ID"); }
|
||||
/** Returns the dictionary model version. */
|
||||
private String modelVersion() { return value("Dictionary model version"); }
|
||||
/** Returns the dictionary model SHA-256. */
|
||||
private String modelSha256() { return value("Dictionary model SHA-256"); }
|
||||
/** Returns the dictionary-processing mode. */
|
||||
private String mode() { return value("Dictionary mode"); }
|
||||
/** Returns the output policy. */
|
||||
@@ -736,21 +796,34 @@ public final class StemmingQualityDocumentationPublisher {
|
||||
/** Parses a numeric field, placing undefined values last during sorting. */
|
||||
private double number(final String name) { return value(name).isEmpty() ? Double.NEGATIVE_INFINITY : Double.parseDouble(value(name)); }
|
||||
/** Returns false-negative pairs. */
|
||||
private long fn() { return longValue("False-negative pairs"); }
|
||||
private long fn() { return longValue("Under-stemming error pairs"); }
|
||||
/** Returns false-positive pairs. */
|
||||
private long fp() { return longValue("False-positive pairs"); }
|
||||
private long fp() { return longValue("Over-stemming error pairs"); }
|
||||
|
||||
/** Validates raw confusion counts and the published balanced accuracy. */
|
||||
private void validate() {
|
||||
final long tp = longValue("True-positive pairs");
|
||||
final long fp = fp();
|
||||
final long fn = fn();
|
||||
final long tn = longValue("True-negative pairs");
|
||||
if (fn != longValue("Under-stemming error pairs") || fp != longValue("Over-stemming error pairs")
|
||||
|| Math.addExact(tp, fn) != longValue("Under-stemming possible pairs")
|
||||
|| Math.addExact(tn, fp) != longValue("Over-stemming possible pairs")) {
|
||||
final long underPossible = longValue("Under-stemming possible pairs");
|
||||
final long overPossible = longValue("Over-stemming possible pairs");
|
||||
if (fn < 0 || fp < 0 || fn > underPossible || fp > overPossible) {
|
||||
throw new IllegalStateException("Raw pair-count invariants fail for " + key());
|
||||
}
|
||||
if (policy().equals("ANY_CANDIDATE")) {
|
||||
if (!value("True-positive pairs").isEmpty() || !value("False-positive pairs").isEmpty()
|
||||
|| !value("False-negative pairs").isEmpty() || !value("True-negative pairs").isEmpty()
|
||||
|| !value("Balanced accuracy").isEmpty() || !value("Pairwise F1").isEmpty()
|
||||
|| !value("Matthews correlation coefficient").isEmpty()) {
|
||||
throw new IllegalStateException("Oracle-assisted ANY_CANDIDATE row contains incoherent classification metrics: "
|
||||
+ key());
|
||||
}
|
||||
return;
|
||||
}
|
||||
final long tp = longValue("True-positive pairs");
|
||||
final long tn = longValue("True-negative pairs");
|
||||
if (Math.addExact(tp, fn) != underPossible || Math.addExact(tn, fp) != overPossible) {
|
||||
throw new IllegalStateException("Raw confusion-count invariants fail for " + key());
|
||||
}
|
||||
final double recall = ratio(tp, Math.addExact(tp, fn));
|
||||
final double specificity = ratio(tn, Math.addExact(tn, fp));
|
||||
final double expected = (recall + specificity) / 2.0;
|
||||
|
||||
143
tools/run-published-speed-benchmarks.sh
Executable file
143
tools/run-published-speed-benchmarks.sh
Executable file
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
report_date="${1:-$(date +%F)}"
|
||||
project_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "${project_root}"
|
||||
|
||||
classpath_file="build/reports/jmh/jmh-runtime-classpath.txt"
|
||||
if [[ ! -s "${classpath_file}" ]]; then
|
||||
printf 'Missing %s; run ./gradlew writeJmhRuntimeClasspath --no-daemon first.\n' "${classpath_file}" >&2
|
||||
exit 1
|
||||
fi
|
||||
IFS= read -r jmh_classpath < "${classpath_file}"
|
||||
|
||||
tmp_dir="${project_root}/build/tmp/jmh"
|
||||
report_dir="${project_root}/build/reports/jmh"
|
||||
mkdir -p "${tmp_dir}" "${report_dir}"
|
||||
|
||||
for governor_file in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
|
||||
governor="$(<"${governor_file}")"
|
||||
if [[ "${governor}" != "performance" ]]; then
|
||||
printf 'CPU governor is %s in %s; expected performance.\n' "${governor}" "${governor_file}" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
comparison_include='^(org\.egothor\.stemmer\.benchmark\.(EnglishStemmerComparisonBenchmark\.|MultiLanguageStemmerComparisonBenchmark\.|SnowballLanguageStemmerComparisonBenchmark\.).*|org\.egothor\.stemmer\.benchmark\.HunspellStemmerComparisonBenchmark\.luceneHunspellStemFilter)$'
|
||||
coverage_include='^org\.egothor\.stemmer\.benchmark\.EnglishRadixorDictionaryCoverageBenchmark\.changedTokenStemmingSpeed$'
|
||||
selection_file="${report_dir}/published-speed-benchmarks-${report_date}.txt"
|
||||
|
||||
java -Djava.io.tmpdir="${tmp_dir}" -cp "${jmh_classpath}" org.openjdk.jmh.Main \
|
||||
"${comparison_include}" -l > "${selection_file}"
|
||||
if grep -Eq 'PolishPolimorf|BenchmarkQuality|GermanGoldstandard' "${selection_file}"; then
|
||||
printf 'The selected speed benchmark list contains an excluded benchmark.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q 'MultiLanguageStemmerComparisonBenchmark.hebrewRadixor' "${selection_file}"; then
|
||||
printf 'The selected speed benchmark list omits Hebrew Radixor.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
environment_file="${report_dir}/performance-environment-${report_date}.txt"
|
||||
source_patch="${report_dir}/measured-source-${report_date}.patch"
|
||||
untracked_checksums="${report_dir}/measured-untracked-${report_date}.sha256"
|
||||
git diff --binary > "${source_patch}"
|
||||
git ls-files --others --exclude-standard -z -- src tools docs build.gradle mkdocs.yml \
|
||||
| sort -z \
|
||||
| xargs -0 --no-run-if-empty sha256sum > "${untracked_checksums}"
|
||||
jmh_jar="${jmh_classpath%%:*}"
|
||||
{
|
||||
printf 'Benchmark start: '
|
||||
date --iso-8601=seconds
|
||||
printf 'Project root: %s\n' "${project_root}"
|
||||
printf 'Core base commit: '
|
||||
git rev-parse HEAD
|
||||
printf 'Git describe: '
|
||||
git describe --always --dirty
|
||||
printf 'JMH runtime classpath SHA-256: '
|
||||
sha256sum "${classpath_file}" | cut -d' ' -f1
|
||||
printf 'JMH executable JAR SHA-256: '
|
||||
sha256sum "${jmh_jar}" | cut -d' ' -f1
|
||||
printf 'Measured source patch SHA-256: '
|
||||
sha256sum "${source_patch}" | cut -d' ' -f1
|
||||
printf 'Untracked source checksum manifest SHA-256: '
|
||||
sha256sum "${untracked_checksums}" | cut -d' ' -f1
|
||||
printf 'Corpus report SHA-256: '
|
||||
sha256sum build/reports/jmh/benchmark-corpora.csv | cut -d' ' -f1
|
||||
printf 'Stemming-quality report SHA-256: '
|
||||
sha256sum build/reports/stemming-quality/stemming-quality.csv | cut -d' ' -f1
|
||||
printf '\nJava:\n'
|
||||
java -version 2>&1
|
||||
printf '\nKernel:\n'
|
||||
uname -a
|
||||
printf '\nCPU:\n'
|
||||
lscpu
|
||||
printf '\nCPU governors:\n'
|
||||
for governor_file in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
|
||||
printf '%s=' "${governor_file}"
|
||||
cat "${governor_file}"
|
||||
done
|
||||
printf 'Energy performance preference: '
|
||||
cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference
|
||||
printf '\nMemory:\n'
|
||||
free -h
|
||||
printf '\nInitial load:\n'
|
||||
cat /proc/loadavg
|
||||
if command -v sensors >/dev/null 2>&1; then
|
||||
printf '\nInitial sensors:\n'
|
||||
sensors
|
||||
fi
|
||||
printf '\nSelected comparison benchmarks:\n'
|
||||
cat "${selection_file}"
|
||||
printf '\nWorking tree:\n'
|
||||
git status --short
|
||||
} > "${environment_file}"
|
||||
|
||||
sleep 120
|
||||
{
|
||||
printf '\nPre-run load after 120 s idle interval:\n'
|
||||
cat /proc/loadavg
|
||||
if command -v sensors >/dev/null 2>&1; then
|
||||
printf '\nPre-run sensors after 120 s idle interval:\n'
|
||||
sensors
|
||||
fi
|
||||
} >> "${environment_file}"
|
||||
|
||||
common_arguments=(
|
||||
-f 3
|
||||
-wi 5
|
||||
-i 10
|
||||
-w 1s
|
||||
-r 1s
|
||||
-t 1
|
||||
-bm avgt
|
||||
-tu ns
|
||||
-jvmArgsAppend "-Djava.io.tmpdir=${tmp_dir} -Xms6g -Xmx6g"
|
||||
-rf csv
|
||||
)
|
||||
|
||||
java -Djava.io.tmpdir="${tmp_dir}" -Xms512m -Xmx1g \
|
||||
-cp "${jmh_classpath}" org.openjdk.jmh.Main \
|
||||
"${comparison_include}" "${common_arguments[@]}" \
|
||||
-rff "${report_dir}/stemmer-speed-${report_date}.csv" \
|
||||
-o "${report_dir}/stemmer-speed-${report_date}.txt"
|
||||
|
||||
sleep 60
|
||||
|
||||
java -Djava.io.tmpdir="${tmp_dir}" -Xms512m -Xmx1g \
|
||||
-cp "${jmh_classpath}" org.openjdk.jmh.Main \
|
||||
"${coverage_include}" "${common_arguments[@]}" \
|
||||
-rff "${report_dir}/english-coverage-speed-${report_date}.csv" \
|
||||
-o "${report_dir}/english-coverage-speed-${report_date}.txt"
|
||||
|
||||
{
|
||||
printf '\nFinal load:\n'
|
||||
cat /proc/loadavg
|
||||
if command -v sensors >/dev/null 2>&1; then
|
||||
printf '\nFinal sensors:\n'
|
||||
sensors
|
||||
fi
|
||||
printf '\nBenchmark end: '
|
||||
date --iso-8601=seconds
|
||||
} >> "${environment_file}"
|
||||
581
tools/update-benchmark-documentation.py
Normal file
581
tools/update-benchmark-documentation.py
Normal file
@@ -0,0 +1,581 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Update published benchmark tables from deterministic corpus and JMH CSV reports."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import math
|
||||
import re
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
LANGUAGES = {
|
||||
"czech.md": "CS_CZ",
|
||||
"danish.md": "DA_DK",
|
||||
"dutch.md": "NL_NL",
|
||||
"english.md": "US_UK",
|
||||
"finnish.md": "FI_FI",
|
||||
"french.md": "FR_FR",
|
||||
"german.md": "DE_DE",
|
||||
"hebrew.md": "HE_IL",
|
||||
"hungarian.md": "HU_HU",
|
||||
"italian.md": "IT_IT",
|
||||
"norwegian-bokmal.md": "NB_NO",
|
||||
"norwegian-nynorsk.md": "NN_NO",
|
||||
"persian.md": "FA_IR",
|
||||
"polish.md": "PL_PL",
|
||||
"portuguese.md": "PT_PT",
|
||||
"russian.md": "RU_RU",
|
||||
"spanish.md": "ES_ES",
|
||||
"swedish.md": "SV_SE",
|
||||
"ukrainian.md": "UK_UA",
|
||||
"yiddish.md": "YI",
|
||||
}
|
||||
|
||||
LANGUAGE_IDENTITY_WORDS = {
|
||||
"CS_CZ": {"CZECH"},
|
||||
"DA_DK": {"DANISH"},
|
||||
"NL_NL": {"DUTCH"},
|
||||
"US_UK": {"ENGLISH"},
|
||||
"FI_FI": {"FINNISH"},
|
||||
"FR_FR": {"FRENCH"},
|
||||
"DE_DE": {"GERMAN"},
|
||||
"HE_IL": {"HEBREW"},
|
||||
"HU_HU": {"HUNGARIAN"},
|
||||
"IT_IT": {"ITALIAN"},
|
||||
"NB_NO": {"NORWEGIAN", "BOKMAL"},
|
||||
"NN_NO": {"NORWEGIAN", "NYNORSK"},
|
||||
"FA_IR": {"PERSIAN"},
|
||||
"PL_PL": {"POLISH"},
|
||||
"PT_PT": {"PORTUGUESE"},
|
||||
"RU_RU": {"RUSSIAN"},
|
||||
"ES_ES": {"SPANISH"},
|
||||
"SV_SE": {"SWEDISH"},
|
||||
"UK_UA": {"UKRAINIAN"},
|
||||
"YI": {"YIDDISH"},
|
||||
}
|
||||
|
||||
COMMAND_MEANINGS = {
|
||||
"AppendCharacterCommand": "Appends one character to the end of the word form.",
|
||||
"BackwardCompoundCommand": "Applies a multi-step backward patch made from skip, delete, insert, and replace operations.",
|
||||
"DeletePrefixCommand": "Deletes one or more leading characters from the word form in forward traversal.",
|
||||
"DeleteSuffixCommand": "Deletes one or more trailing characters from the word form.",
|
||||
"ForwardCompoundCommand": "Applies a multi-step forward patch made from skip, delete, insert, and replace operations.",
|
||||
"PrependCharacterCommand": "Prepends one character to the beginning of the word form.",
|
||||
"PreserveCommand": "Returns the word form unchanged because it already matches the preferred root.",
|
||||
"ReplaceFirstCharacterCommand": "Replaces the first character of the word form in forward traversal.",
|
||||
"ReplaceLastCharacterCommand": "Replaces the final character of the word form.",
|
||||
}
|
||||
|
||||
AUXILIARY_NAMES = {
|
||||
"changedCorrectMatches",
|
||||
"changedEvaluatedTokens",
|
||||
"correctMatches",
|
||||
"evaluatedTokens",
|
||||
"rootEvaluatedTokens",
|
||||
"rootPreservedMatches",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Key:
|
||||
benchmark: str
|
||||
parameters: tuple[tuple[str, str], ...]
|
||||
|
||||
@property
|
||||
def method(self) -> str:
|
||||
return self.benchmark.rsplit(".", 1)[-1]
|
||||
|
||||
def parameter(self, name: str) -> str:
|
||||
return dict(self.parameters).get(name, "")
|
||||
|
||||
|
||||
@dataclass
|
||||
class JmhData:
|
||||
primary: dict[Key, dict[str, str]]
|
||||
auxiliary: dict[Key, dict[str, float]]
|
||||
|
||||
|
||||
def parse_arguments() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--docs-root", type=Path, default=Path("docs"))
|
||||
parser.add_argument("--readme", type=Path, default=Path("README.md"))
|
||||
parser.add_argument("--corpus", type=Path, required=True)
|
||||
parser.add_argument("--old-comparison", type=Path, required=True)
|
||||
parser.add_argument("--accuracy", type=Path, required=True)
|
||||
parser.add_argument("--speed", type=Path, required=True)
|
||||
parser.add_argument("--coverage-accuracy", type=Path, required=True)
|
||||
parser.add_argument("--coverage-speed", type=Path, required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def read_jmh(path: Path) -> JmhData:
|
||||
primary: dict[Key, dict[str, str]] = {}
|
||||
auxiliary: dict[Key, dict[str, float]] = defaultdict(dict)
|
||||
with path.open(newline="", encoding="utf-8") as source:
|
||||
for row in csv.DictReader(source):
|
||||
benchmark_with_metric = row["Benchmark"]
|
||||
benchmark, separator, metric = benchmark_with_metric.partition(":")
|
||||
parameters = tuple(
|
||||
(name.removeprefix("Param: "), value)
|
||||
for name, value in row.items()
|
||||
if name.startswith("Param: ") and value
|
||||
)
|
||||
key = Key(benchmark, parameters)
|
||||
if separator:
|
||||
auxiliary[key][metric] = float(row["Score"])
|
||||
else:
|
||||
primary[key] = row
|
||||
return JmhData(primary, dict(auxiliary))
|
||||
|
||||
|
||||
def accuracy(data: JmhData, key: Key) -> tuple[float, float, float]:
|
||||
counters = data.auxiliary[key]
|
||||
return (
|
||||
100.0 * counters["correctMatches"] / counters["evaluatedTokens"],
|
||||
100.0 * counters["changedCorrectMatches"] / counters["changedEvaluatedTokens"],
|
||||
100.0 * counters["rootPreservedMatches"] / counters["rootEvaluatedTokens"],
|
||||
)
|
||||
|
||||
|
||||
def read_corpora(path: Path) -> dict[str, dict[str, object]]:
|
||||
corpora: dict[str, dict[str, object]] = {}
|
||||
with path.open(newline="", encoding="utf-8") as source:
|
||||
for row in csv.DictReader(source):
|
||||
language = row["Language"]
|
||||
entry = corpora.setdefault(
|
||||
language,
|
||||
{
|
||||
"model": row["Model ID"],
|
||||
"version": row["Model version"],
|
||||
"sha256": row["Model SHA-256"],
|
||||
"rows": int(row["Dictionary rows"]),
|
||||
"total": int(row["Total tokens"]),
|
||||
"roots": int(row["Already-root tokens"]),
|
||||
"changed": int(row["Changed tokens"]),
|
||||
"timing": int(row["Speed timing tokens"]),
|
||||
"all_exact": int(row["All exact matches"]),
|
||||
"changed_exact": int(row["Changed exact matches"]),
|
||||
"root_exact": int(row["Root preserved matches"]),
|
||||
"commands": [],
|
||||
},
|
||||
)
|
||||
entry["commands"].append((row["Command class"], int(row["Command count"])))
|
||||
if set(corpora) != set(LANGUAGES.values()):
|
||||
raise ValueError(f"Corpus report languages differ from documentation languages: {sorted(corpora)}")
|
||||
if any(entry["model"] == "pl-pl-polimorf" for entry in corpora.values()):
|
||||
raise ValueError("The default-model corpus report must not contain pl-pl-polimorf.")
|
||||
return corpora
|
||||
|
||||
|
||||
def format_integer(value: int) -> str:
|
||||
return f"{value:,}"
|
||||
|
||||
|
||||
def render_corpus_sections(language: str, entry: dict[str, object]) -> str:
|
||||
total = int(entry["total"])
|
||||
lines = [
|
||||
"## Dictionary Corpus",
|
||||
"",
|
||||
"| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed speed tokens |",
|
||||
"| --- | --- | --- | ---: | ---: | ---: | ---: |",
|
||||
f"| `{entry['model']}` | `{entry['version']}` | `{language}` | {format_integer(int(entry['rows']))} | "
|
||||
f"{format_integer(total)} | {format_integer(int(entry['roots']))} | "
|
||||
f"{format_integer(int(entry['changed']))} |",
|
||||
"",
|
||||
"## Radixor Patch Command Distribution",
|
||||
"",
|
||||
"Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. "
|
||||
"This distribution shows which runtime command class is selected by the trained trie for the complete default-model "
|
||||
f"dictionary. The total number of preferred patch commands analyzed for this language is **{format_integer(total)}**.",
|
||||
"",
|
||||
"| Command class | Meaning | Word forms | Share |",
|
||||
"| --- | --- | ---: | ---: |",
|
||||
]
|
||||
command_total = 0
|
||||
for command, count in entry["commands"]:
|
||||
if command not in COMMAND_MEANINGS:
|
||||
raise ValueError(f"Undocumented patch command class: {command}")
|
||||
command_total += count
|
||||
lines.append(
|
||||
f"| `{command}` | {COMMAND_MEANINGS[command]} | {format_integer(count)} | "
|
||||
f"{100.0 * count / total:.3f}% |"
|
||||
)
|
||||
if command_total != total:
|
||||
raise ValueError(f"Patch command count {command_total} differs from corpus total {total} for {language}.")
|
||||
return "\n".join(lines) + "\n\n"
|
||||
|
||||
|
||||
def rounded_accuracy(values: tuple[float, float, float]) -> tuple[str, str, str]:
|
||||
return tuple(f"{value:.3f}" for value in values)
|
||||
|
||||
|
||||
def words(value: str) -> set[str]:
|
||||
value = value.replace("OpenNLP", "OPENNLP")
|
||||
value = re.sub(r"(?<=[a-z0-9])(?=[A-Z])", " ", value)
|
||||
return {
|
||||
{"COPIED": "COPY"}.get(word, word)
|
||||
for word in re.sub(r"[^A-Za-z0-9]+", " ", value).upper().split()
|
||||
if len(word) > 2
|
||||
and word
|
||||
not in {
|
||||
"ACCURACY",
|
||||
"AGREEMENT",
|
||||
"BENCHMARK",
|
||||
"CANDIDATE",
|
||||
"CASE",
|
||||
"COMPARISON",
|
||||
"EGOTHOR",
|
||||
"EXACT",
|
||||
"LANGUAGE",
|
||||
"NAME",
|
||||
"ORG",
|
||||
"QUALITY",
|
||||
"ROOT",
|
||||
"STEM",
|
||||
"STEMMER",
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def select_accuracy_key(
|
||||
label: str,
|
||||
language: str,
|
||||
data: JmhData,
|
||||
) -> Key:
|
||||
language_words = LANGUAGE_IDENTITY_WORDS[language]
|
||||
matches = [
|
||||
key
|
||||
for key, counters in data.auxiliary.items()
|
||||
if AUXILIARY_NAMES.issubset(counters)
|
||||
and language_words.issubset(
|
||||
words(key.benchmark + " " + " ".join(f"{name} {value}" for name, value in key.parameters))
|
||||
)
|
||||
]
|
||||
if not matches:
|
||||
raise ValueError(f"No current JMH accuracy row matches language {language} and label {label}.")
|
||||
|
||||
label_words = words(label) - language_words
|
||||
|
||||
def score(key: Key) -> tuple[int, int, int, int, int]:
|
||||
identity_words = (
|
||||
words(key.benchmark + " " + " ".join(f"{name} {value}" for name, value in key.parameters))
|
||||
- language_words
|
||||
)
|
||||
return (
|
||||
len(label_words & identity_words),
|
||||
-len(label_words - identity_words),
|
||||
-len(identity_words - label_words),
|
||||
int(key.method != "exactRootAgreement"),
|
||||
int(language_words.issubset(words(key.benchmark))),
|
||||
)
|
||||
|
||||
ranked = sorted(((score(key), key) for key in matches), reverse=True, key=lambda item: item[0])
|
||||
if ranked[0][0][0] == 0:
|
||||
raise ValueError(f"No implementation identity words match accuracy label {label} for {language}.")
|
||||
if len(ranked) > 1 and ranked[0][0] == ranked[1][0]:
|
||||
raise ValueError(
|
||||
f"Ambiguous current JMH accuracy identity for {label} in {language}: "
|
||||
f"{ranked[0][1]} and {ranked[1][1]}"
|
||||
)
|
||||
return ranked[0][1]
|
||||
|
||||
|
||||
def corpus_accuracy(entry: dict[str, object]) -> tuple[float, float, float]:
|
||||
return (
|
||||
100.0 * int(entry["all_exact"]) / int(entry["total"]),
|
||||
100.0 * int(entry["changed_exact"]) / int(entry["changed"]),
|
||||
100.0 * int(entry["root_exact"]) / int(entry["roots"]),
|
||||
)
|
||||
|
||||
|
||||
def update_accuracy_table(
|
||||
text: str,
|
||||
new_data: JmhData,
|
||||
language: str,
|
||||
corpus: dict[str, object],
|
||||
) -> str:
|
||||
start = text.index("## Accuracy")
|
||||
end = text.index("## Speed", start)
|
||||
section = text[start:end]
|
||||
output: list[str] = []
|
||||
for line in section.splitlines():
|
||||
cells = [cell.strip() for cell in line.split("|")[1:-1]]
|
||||
if len(cells) == 5 and all(re.fullmatch(r"\d+\.\d{3}%", cell) for cell in cells[1:4]):
|
||||
if cells[0] == "Radixor":
|
||||
values = rounded_accuracy(corpus_accuracy(corpus))
|
||||
else:
|
||||
key = select_accuracy_key(cells[0], language, new_data)
|
||||
values = rounded_accuracy(accuracy(new_data, key))
|
||||
cells[1:4] = [f"{value}%" for value in values]
|
||||
line = "| " + " | ".join(cells) + " |"
|
||||
elif language == "HE_IL" and len(cells) == 5 and cells[0] == "Radixor" and cells[1] == "pending":
|
||||
values = rounded_accuracy(corpus_accuracy(corpus))
|
||||
cells[1:4] = [f"{value}%" for value in values]
|
||||
line = "| " + " | ".join(cells) + " |"
|
||||
output.append(line)
|
||||
replacement = "\n".join(output) + "\n\n"
|
||||
return text[:start] + replacement + text[end:]
|
||||
|
||||
|
||||
def method_and_parameter(display: str) -> tuple[str, str]:
|
||||
match = re.fullmatch(r"([A-Za-z0-9]+)(?:\[([A-Z_]+)])?", display)
|
||||
if not match:
|
||||
raise ValueError(f"Unsupported benchmark method display: {display}")
|
||||
return match.group(1), match.group(2) or ""
|
||||
|
||||
|
||||
def speed_matches(display: str, data: JmhData) -> list[Key]:
|
||||
method, language_case = method_and_parameter(display)
|
||||
return [
|
||||
key
|
||||
for key, row in data.primary.items()
|
||||
if key.method == method
|
||||
and (not language_case or key.parameter("languageCaseName") == language_case)
|
||||
and key not in data.auxiliary
|
||||
and row["Unit"] == "ns/op"
|
||||
]
|
||||
|
||||
|
||||
def closest_speed_key(display: str, score_ms: float, data: JmhData) -> tuple[Key, float]:
|
||||
matches = speed_matches(display, data)
|
||||
if not matches:
|
||||
raise ValueError(f"No JMH speed row matches {display}")
|
||||
selected = min(matches, key=lambda key: abs(float(data.primary[key]["Score"]) / 1_000_000.0 - score_ms))
|
||||
difference = abs(float(data.primary[selected]["Score"]) / 1_000_000.0 - score_ms)
|
||||
return selected, difference
|
||||
|
||||
|
||||
def select_speed_key(display: str, published_score_ms: float, old_data: JmhData, new_data: JmhData) -> Key:
|
||||
current, current_difference = closest_speed_key(display, published_score_ms, new_data)
|
||||
if current_difference < 0.001:
|
||||
return current
|
||||
selected, difference = closest_speed_key(display, published_score_ms, old_data)
|
||||
if difference >= 0.001:
|
||||
raise ValueError(f"Old speed row for {display} differs by {difference:.6f} ms from documentation.")
|
||||
return selected
|
||||
|
||||
|
||||
def update_speed_table(
|
||||
text: str,
|
||||
old_data: JmhData,
|
||||
new_data: JmhData,
|
||||
changed_tokens: int,
|
||||
language: str,
|
||||
) -> str:
|
||||
start = text.index("## Speed")
|
||||
end = text.index("## Interpretation Notes", start)
|
||||
section = text[start:end]
|
||||
parsed: list[tuple[str, list[str] | None, Key | None]] = []
|
||||
radixor_score = math.nan
|
||||
for line in section.splitlines():
|
||||
cells = [cell.strip() for cell in line.split("|")[1:-1]]
|
||||
if len(cells) == 7 and cells[1].startswith("`") and cells[1].endswith("`"):
|
||||
display = cells[1].strip("`")
|
||||
if cells[2] == "pending" and language == "HE_IL":
|
||||
matches = [
|
||||
key
|
||||
for key, row in new_data.primary.items()
|
||||
if key.method == "hebrewRadixor" and key not in new_data.auxiliary and row["Unit"] == "ns/op"
|
||||
]
|
||||
if len(matches) != 1:
|
||||
raise ValueError(f"Expected one Hebrew speed row, found {len(matches)}")
|
||||
key = matches[0]
|
||||
elif re.fullmatch(r"\d+\.\d{3}", cells[2]):
|
||||
key = select_speed_key(display, float(cells[2]), old_data, new_data)
|
||||
else:
|
||||
parsed.append((line, None, None))
|
||||
continue
|
||||
if key not in new_data.primary:
|
||||
raise ValueError(f"New JMH report omits speed key {key}")
|
||||
score = float(new_data.primary[key]["Score"])
|
||||
if cells[0] == "Radixor":
|
||||
radixor_score = score
|
||||
parsed.append((line, cells, key))
|
||||
else:
|
||||
parsed.append((line, None, None))
|
||||
if math.isnan(radixor_score):
|
||||
raise ValueError(f"No Radixor speed baseline found for {language}")
|
||||
|
||||
output: list[str] = []
|
||||
for line, cells, key in parsed:
|
||||
if cells is not None and key is not None:
|
||||
row = new_data.primary[key]
|
||||
score = float(row["Score"])
|
||||
error = float(row["Score Error (99.9%)"])
|
||||
cells[2] = f"{score / 1_000_000.0:.3f}"
|
||||
cells[3] = f"{error / 1_000_000.0:.3f}"
|
||||
cells[4] = f"{score / changed_tokens:.1f}"
|
||||
cells[5] = f"{score / radixor_score:.3f}"
|
||||
line = "| " + " | ".join(cells) + " |"
|
||||
output.append(line)
|
||||
replacement = "\n".join(output) + "\n\n"
|
||||
return text[:start] + replacement + text[end:]
|
||||
|
||||
|
||||
def update_language_pages(
|
||||
docs_root: Path,
|
||||
corpora: dict[str, dict[str, object]],
|
||||
old_data: JmhData,
|
||||
accuracy_data: JmhData,
|
||||
speed_data: JmhData,
|
||||
) -> None:
|
||||
directory = docs_root / "benchmarks" / "languages"
|
||||
for file_name, language in LANGUAGES.items():
|
||||
path = directory / file_name
|
||||
text = path.read_text(encoding="utf-8")
|
||||
corpus_start = text.index("## Dictionary Corpus")
|
||||
accuracy_start = text.index("## Accuracy", corpus_start)
|
||||
text = text[:corpus_start] + render_corpus_sections(language, corpora[language]) + text[accuracy_start:]
|
||||
text = re.sub(
|
||||
r"Speed uses JMH average time, \d+ warmup iterations, \d+ measurement iterations, "
|
||||
r"\d+ forks?, and 1 thread\.",
|
||||
"Speed uses JMH average time, 5 warmup iterations, 10 measurement iterations, "
|
||||
"3 independent forks, and 1 thread.",
|
||||
text,
|
||||
count=1,
|
||||
)
|
||||
text = update_accuracy_table(text, accuracy_data, language, corpora[language])
|
||||
text = update_speed_table(text, old_data, speed_data, int(corpora[language]["changed"]), language)
|
||||
path.write_text(text, encoding="utf-8")
|
||||
|
||||
|
||||
def update_corpora_reference(docs_root: Path, corpora: dict[str, dict[str, object]]) -> None:
|
||||
path = docs_root / "benchmarks" / "reference" / "corpora.md"
|
||||
text = path.read_text(encoding="utf-8")
|
||||
original_header = "| Language resource |"
|
||||
current_header = "| Default model ID |"
|
||||
if original_header in text:
|
||||
table_start = text.index(original_header)
|
||||
elif current_header in text:
|
||||
table_start = text.index(current_header)
|
||||
else:
|
||||
raise ValueError("The corpora reference contains no recognized corpus-table header.")
|
||||
table_end = text.index("\n\n", table_start)
|
||||
lines = [
|
||||
"| Default model ID | Version | SHA-256 | Language | Dictionary rows | Total tokens | Already-root tokens | Changed tokens | Speed timing tokens |",
|
||||
"| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: |",
|
||||
]
|
||||
for language in LANGUAGES.values():
|
||||
entry = corpora[language]
|
||||
lines.append(
|
||||
f"| `{entry['model']}` | `{entry['version']}` | `{entry['sha256']}` | `{language}` | "
|
||||
f"{format_integer(int(entry['rows']))} | "
|
||||
f"{format_integer(int(entry['total']))} | {format_integer(int(entry['roots']))} | "
|
||||
f"{format_integer(int(entry['changed']))} | {format_integer(int(entry['timing']))} |"
|
||||
)
|
||||
replacement = "\n".join(lines)
|
||||
path.write_text(text[:table_start] + replacement + text[table_end:], encoding="utf-8")
|
||||
|
||||
|
||||
def coverage_rows(accuracy_data: JmhData, speed_data: JmhData) -> list[str]:
|
||||
lines = [
|
||||
"| Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token |",
|
||||
"| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |",
|
||||
]
|
||||
for percent in range(100, 0, -10):
|
||||
parameter = str(percent)
|
||||
accuracy_keys = [
|
||||
key
|
||||
for key, counters in accuracy_data.auxiliary.items()
|
||||
if key.method == "exactRootAgreement"
|
||||
and key.parameter("coveragePercent") == parameter
|
||||
and AUXILIARY_NAMES.issubset(counters)
|
||||
]
|
||||
speed_keys = [
|
||||
key
|
||||
for key, row in speed_data.primary.items()
|
||||
if key.method == "changedTokenStemmingSpeed"
|
||||
and key.parameter("coveragePercent") == parameter
|
||||
and row["Unit"] == "ns/op"
|
||||
]
|
||||
if len(accuracy_keys) != 1 or len(speed_keys) != 1:
|
||||
raise ValueError(f"Incomplete English coverage results for {percent}%.")
|
||||
accuracy_key = accuracy_keys[0]
|
||||
speed_key = speed_keys[0]
|
||||
counters = accuracy_data.auxiliary[accuracy_key]
|
||||
actual = 100.0 * counters["selectedRows"] / counters["totalRows"]
|
||||
values = accuracy(accuracy_data, accuracy_key)
|
||||
speed = float(speed_data.primary[speed_key]["Score"])
|
||||
error = float(speed_data.primary[speed_key]["Score Error (99.9%)"])
|
||||
lines.append(
|
||||
f"| {percent}% | {actual:.3f}% | {values[0]:.3f}% | {values[1]:.3f}% | {values[2]:.3f}% | "
|
||||
f"{speed / 1_000_000.0:.3f} | {error / 1_000_000.0:.3f} | {speed / 210_500:.1f} |"
|
||||
)
|
||||
return lines
|
||||
|
||||
|
||||
def replace_coverage_table(text: str, lines: list[str]) -> str:
|
||||
start = text.index("| Used rows |")
|
||||
end = text.index("\n\n", start)
|
||||
return text[:start] + "\n".join(lines) + text[end:]
|
||||
|
||||
|
||||
def update_coverage(
|
||||
docs_root: Path,
|
||||
readme: Path,
|
||||
accuracy_data: JmhData,
|
||||
speed_data: JmhData,
|
||||
) -> None:
|
||||
lines = coverage_rows(accuracy_data, speed_data)
|
||||
full = [cell.strip() for cell in lines[2].split("|")[1:-1]]
|
||||
reduced = [cell.strip() for cell in lines[-1].split("|")[1:-1]]
|
||||
reference = docs_root / "benchmarks" / "reference" / "english-coverage.md"
|
||||
reference.write_text(
|
||||
replace_coverage_table(reference.read_text(encoding="utf-8"), lines),
|
||||
encoding="utf-8",
|
||||
)
|
||||
readme_text = replace_coverage_table(readme.read_text(encoding="utf-8"), lines)
|
||||
readme_text = re.sub(
|
||||
r"The contracted trie result is materially stronger than the older uncontracted profile: "
|
||||
r"full English coverage reaches .*?"
|
||||
r"This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge\.",
|
||||
"The contracted trie result is materially stronger than the older uncontracted profile: "
|
||||
f"full English coverage reaches {full[2]} all-token exactness and {full[3]} changed-token exactness "
|
||||
f"at {full[7]} ns/token, while even a 10% deterministic dictionary slice remains at {reduced[2]} "
|
||||
f"all-token exactness and {reduced[3]} changed-token exactness at {reduced[7]} ns/token. "
|
||||
"This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge.",
|
||||
readme_text,
|
||||
count=1,
|
||||
flags=re.DOTALL,
|
||||
)
|
||||
readme.write_text(readme_text, encoding="utf-8")
|
||||
|
||||
index = docs_root / "benchmarks" / "index.md"
|
||||
index_text = index.read_text(encoding="utf-8")
|
||||
key_start = index_text.index("## Key Published Result")
|
||||
key_end = index_text.index("## Quality versus performance", key_start)
|
||||
key_section = (
|
||||
"## Key Published Result\n\n"
|
||||
"The English dictionary coverage benchmark shows the current contracted-trie operating curve. With\n"
|
||||
f"the full English dictionary, Radixor reaches `{full[2]}` all-token exactness and `{full[3]}`\n"
|
||||
f"changed-token exactness at `{full[7]} ns/token`. Even with a deterministic 10% dictionary slice, it\n"
|
||||
f"keeps `{reduced[2]}` all-token exactness and `{reduced[3]}` changed-token exactness at `{reduced[7]} ns/token`.\n\n"
|
||||
"Those figures should not be reduced to a single speed badge. The professional interpretation is a\n"
|
||||
"quality/speed envelope: the amount and quality of dictionary knowledge affect stemming precision,\n"
|
||||
"while contracted tries reduce lookup cost in uniform regions of the compiled graph.\n\n"
|
||||
)
|
||||
index.write_text(index_text[:key_start] + key_section + index_text[key_end:], encoding="utf-8")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
arguments = parse_arguments()
|
||||
corpora = read_corpora(arguments.corpus)
|
||||
old_data = read_jmh(arguments.old_comparison)
|
||||
accuracy_data = read_jmh(arguments.accuracy)
|
||||
speed_data = read_jmh(arguments.speed)
|
||||
coverage_accuracy_data = read_jmh(arguments.coverage_accuracy)
|
||||
coverage_speed_data = read_jmh(arguments.coverage_speed)
|
||||
measured_keys = set(accuracy_data.primary) | set(speed_data.primary)
|
||||
if any("PolishPolimorf" in key.benchmark for key in measured_keys):
|
||||
raise ValueError("A published report contains the excluded PolishPolimorf benchmark.")
|
||||
update_language_pages(arguments.docs_root, corpora, old_data, accuracy_data, speed_data)
|
||||
update_corpora_reference(arguments.docs_root, corpora)
|
||||
update_coverage(arguments.docs_root, arguments.readme, coverage_accuracy_data, coverage_speed_data)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user