Add CISTEM and Hunspell benchmarks and refresh results
This commit is contained in:
@@ -519,6 +519,7 @@ apply from: 'gradle/snowball-benchmarks.gradle'
|
|||||||
apply from: 'gradle/paicehusk-benchmarks.gradle'
|
apply from: 'gradle/paicehusk-benchmarks.gradle'
|
||||||
apply from: 'gradle/opennlp-benchmarks.gradle'
|
apply from: 'gradle/opennlp-benchmarks.gradle'
|
||||||
apply from: 'gradle/hunspell-benchmarks.gradle'
|
apply from: 'gradle/hunspell-benchmarks.gradle'
|
||||||
|
apply from: 'gradle/cistem-benchmarks.gradle'
|
||||||
|
|
||||||
gradle.taskGraph.whenReady { taskGraph ->
|
gradle.taskGraph.whenReady { taskGraph ->
|
||||||
def banner = """
|
def banner = """
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ This page is the entry point for benchmark interpretation. Detailed tables and l
|
|||||||
- Radixor is the quality-oriented baseline in same-language comparisons. Its exact-root accuracy is often close to 100%, while many faster competitors are light, minimal, possessive, or aggressive rule-based stemmers with much lower root agreement.
|
- Radixor is the quality-oriented baseline in same-language comparisons. Its exact-root accuracy is often close to 100%, while many faster competitors are light, minimal, possessive, or aggressive rule-based stemmers with much lower root agreement.
|
||||||
- The measured Radixor cost buys dictionary-trained stemming precision. That precision improves search quality by mapping inflected forms to intended dictionary roots instead of approximate or over-reduced stems.
|
- The measured Radixor cost buys dictionary-trained stemming precision. That precision improves search quality by mapping inflected forms to intended dictionary roots instead of approximate or over-reduced stems.
|
||||||
- Speed benchmarks process changed dictionary tokens where the surface form differs from the expected root. Accuracy benchmarks process the complete dictionary.
|
- Speed benchmarks process changed dictionary tokens where the surface form differs from the expected root. Accuracy benchmarks process the complete dictionary.
|
||||||
- Accuracy-only benchmarks intentionally use one deterministic JMH measurement iteration without warmup because repeated precision passes would duplicate the same counters.
|
- Accuracy tables use deterministic auxiliary counters from the current JMH reports. Repeated measurement samples duplicate the same exact-root accounting and are not interpreted as timing results.
|
||||||
- The historical Porter performance badge is retired. Benchmark reporting now uses speed and quality tables rather than a single Porter ratio.
|
- The historical Porter performance badge is retired. Benchmark reporting now uses speed and quality tables rather than a single Porter ratio.
|
||||||
|
|
||||||
## Benchmark Documentation Map
|
## Benchmark Documentation Map
|
||||||
|
|||||||
@@ -47,8 +47,8 @@ Open [Language Benchmark Pages](languages/index.md) for the complete language li
|
|||||||
|
|
||||||
The English dictionary coverage benchmark shows the current contracted-trie operating curve. With
|
The English dictionary coverage benchmark shows the current contracted-trie operating curve. With
|
||||||
the full English dictionary, Radixor reaches `97.478%` all-token exactness and `97.197%`
|
the full English dictionary, Radixor reaches `97.478%` all-token exactness and `97.197%`
|
||||||
changed-token exactness at `109.8 ns/token`. Even with a deterministic 10% dictionary slice, it
|
changed-token exactness at `135.8 ns/token`. Even with a deterministic 10% dictionary slice, it
|
||||||
keeps `92.868%` all-token exactness and `76.516%` changed-token exactness at `90.9 ns/token`.
|
keeps `92.868%` all-token exactness and `76.516%` changed-token exactness at `86.0 ns/token`.
|
||||||
|
|
||||||
Those figures should not be reduced to a single speed badge. The professional interpretation is a
|
Those figures should not be reduced to a single speed badge. The professional interpretation is a
|
||||||
quality/speed envelope: the amount and quality of dictionary knowledge affect stemming precision,
|
quality/speed envelope: the amount and quality of dictionary knowledge affect stemming precision,
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 99.465% | 99.439% | 99.582% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 99.465% | 99.439% | 99.582% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 84.850% | 82.269% | 96.806% | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene CzechStemFilter | 16.784% | 15.538% | 22.559% | Lucene Czech suffix stemmer implemented as a TokenFilter. |
|
| Lucene CzechStemFilter | 16.784% | 15.538% | 22.559% | Lucene Czech suffix stemmer implemented as a TokenFilter. |
|
||||||
|
|
||||||
## Speed
|
## Speed
|
||||||
@@ -39,8 +40,9 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `czechRadixor` | 3.117 | 0.454 | 66.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `czechRadixor` | 3.332 | 0.240 | 71.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 2.921 | 0.202 | 62.7 | 0.937 | Czech suffix stemmer implemented as a Lucene TokenFilter. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 346.819 | 3.622 | 7448.4 | 104.091 | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
|
| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 3.163 | 0.253 | 67.9 | 0.949 | Czech suffix stemmer implemented as a Lucene TokenFilter. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,11 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 99.371% | 99.527% | 98.923% | Radixor baseline in the Snowball-language comparison family. |
|
| Radixor | 99.371% | 99.527% | 98.923% | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene SnowballFilter | 55.509% | 54.159% | 59.371% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene SnowballFilter | 55.509% | 54.159% | 59.371% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
| Official Snowball direct | 55.509% | 54.159% | 59.371% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Official Snowball direct | 55.509% | 54.159% | 59.371% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
|
|
||||||
@@ -40,9 +40,9 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `radixor[DANISH]` | 1.065 | 0.019 | 44.6 | 1.000 | Radixor baseline for the Snowball-language comparison family. |
|
| Radixor | `radixor[DANISH]` | 1.143 | 0.017 | 47.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[DANISH]` | 2.028 | 0.011 | 84.9 | 1.904 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[DANISH]` | 2.168 | 0.058 | 90.7 | 1.896 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 2.692 | 0.028 | 112.6 | 2.527 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 2.975 | 0.143 | 124.5 | 2.602 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 99.120% | 98.711% | 100.000% | Radixor baseline in the Snowball-language comparison family. |
|
| Radixor | 99.120% | 98.711% | 100.000% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 46.590% | 22.718% | 97.976% | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Official Snowball direct | 15.954% | 8.992% | 30.939% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Official Snowball direct | 15.954% | 8.992% | 30.939% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
| Lucene SnowballFilter | 12.620% | 5.441% | 28.073% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene SnowballFilter | 12.620% | 5.441% | 28.073% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
|
|
||||||
@@ -40,9 +41,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `radixor[DUTCH]` | 1.262 | 0.039 | 58.7 | 1.000 | Radixor baseline for the Snowball-language comparison family. |
|
| Radixor | `radixor[DUTCH]` | 1.331 | 0.114 | 61.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[DUTCH]` | 3.968 | 0.258 | 184.7 | 3.145 | Official Snowball generated Java stemmer; direct API. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 22.760 | 1.387 | 1059.3 | 17.105 | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 6.866 | 0.337 | 319.6 | 5.441 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Official Snowball direct | `snowballDirect[DUTCH]` | 4.146 | 0.291 | 193.0 | 3.116 | Official Snowball generated Java stemmer; direct API. |
|
||||||
|
| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 7.375 | 0.595 | 343.3 | 5.543 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,13 +26,14 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 97.478% | 97.197% | 97.552% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 97.478% | 97.197% | 97.552% | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene EnglishMinimalStemFilter | 90.981% | 65.189% | 97.820% | Minimal English plural reduction, not a full stemmer. |
|
| Lucene EnglishMinimalStemFilter | 90.981% | 65.189% | 97.820% | Minimal English plural reduction, not a full stemmer. |
|
||||||
| Lucene KStemFilter | 80.076% | 76.608% | 80.996% | Krovetz-style English stemming TokenFilter; broader than minimal suffix reducers. |
|
| Lucene KStemFilter | 80.076% | 76.608% | 80.996% | Krovetz-style English stemming TokenFilter; broader than minimal suffix reducers. |
|
||||||
|
| Lucene HunspellStemFilter | 80.243% | 12.750% | 98.139% | Benchmark-only English Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene EnglishPossessiveFilter | 79.032% | 0.003% | 99.987% | Possessive-ending remover only, not a full stemmer. |
|
| Lucene EnglishPossessiveFilter | 79.032% | 0.003% | 99.987% | Possessive-ending remover only, not a full stemmer. |
|
||||||
| Snowball English / Porter2 | 40.342% | 46.296% | 38.763% | Porter2 rule-based suffix stemmer, distinct from original Porter. |
|
| Snowball English / Porter2 | 40.342% | 46.296% | 38.763% | Porter2 rule-based suffix stemmer, distinct from original Porter. |
|
||||||
| Lucene PorterStemFilter | 39.538% | 46.201% | 37.772% | Lucene TokenFilter path for Porter suffix rules; not dictionary-root equivalent. |
|
| Lucene PorterStemFilter | 39.538% | 46.201% | 37.772% | Lucene TokenFilter path for Porter suffix rules; not dictionary-root equivalent. |
|
||||||
@@ -47,16 +48,17 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `radixorUsUkProfiPreferredStem` | 16.621 | 8.532 | 79.0 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. |
|
| Radixor | `radixorUsUkProfiPreferredStem` | 21.987 | 8.707 | 104.5 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. |
|
||||||
| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 23.845 | 0.833 | 113.3 | 1.435 | Possessive-ending remover only; not a full stemmer. |
|
| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 24.539 | 1.515 | 116.6 | 1.116 | Possessive-ending remover only; not a full stemmer. |
|
||||||
| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 17.091 | 0.198 | 81.2 | 1.028 | Narrow plural reduction filter; not a full stemmer. |
|
| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 22.702 | 1.195 | 107.8 | 1.032 | Narrow plural reduction filter; not a full stemmer. |
|
||||||
| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 18.598 | 10.954 | 88.4 | 1.119 | Benchmark-only generated copy of Lucene package-private Porter implementation. |
|
| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 24.696 | 13.235 | 117.3 | 1.123 | Benchmark-only generated copy of Lucene package-private Porter implementation. |
|
||||||
| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 18.213 | 10.674 | 86.5 | 1.096 | Apache OpenNLP Porter implementation. |
|
| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 23.121 | 12.528 | 109.8 | 1.052 | Apache OpenNLP Porter implementation. |
|
||||||
| Snowball original Porter | `snowballOriginalPorter` | 32.921 | 11.520 | 156.4 | 1.981 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. |
|
| Snowball original Porter | `snowballOriginalPorter` | 38.904 | 10.353 | 184.8 | 1.769 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. |
|
||||||
| Lucene PorterStemFilter | `lucenePorterStemFilter` | 42.874 | 1.321 | 203.7 | 2.579 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. |
|
| Lucene PorterStemFilter | `lucenePorterStemFilter` | 37.021 | 1.196 | 175.9 | 1.684 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. |
|
||||||
| Lucene KStemFilter | `luceneKStemFilter` | 50.483 | 3.624 | 239.8 | 3.037 | Krovetz-style English TokenFilter; broader than minimal suffix filters. |
|
| Lucene KStemFilter | `luceneKStemFilter` | 51.640 | 2.591 | 245.3 | 2.349 | Krovetz-style English TokenFilter; broader than minimal suffix filters. |
|
||||||
| Snowball English / Porter2 | `snowballEnglishPorter2` | 47.844 | 1.887 | 227.3 | 2.878 | Porter2 suffix-rule stemmer, distinct from original Porter. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 79.785 | 1.347 | 379.0 | 3.629 | Benchmark-only English Hunspell comparison using the benchmark Hunspell corpus. |
|
||||||
| Paice/Husk Lancaster | `paiceHuskLancaster` | 135.050 | 11.088 | 641.6 | 8.125 | Aggressive rule-based English stemmer. |
|
| Snowball English / Porter2 | `snowballEnglishPorter2` | 52.437 | 0.773 | 249.1 | 2.385 | Porter2 suffix-rule stemmer, distinct from original Porter. |
|
||||||
|
| Paice/Husk Lancaster | `paiceHuskLancaster` | 141.556 | 12.324 | 672.5 | 6.438 | Aggressive rule-based English stemmer. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -41,10 +41,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `finnishRadixor` | 228.248 | 9.245 | 130.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `finnishRadixor` | 308.076 | 15.529 | 175.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 175.923 | 87.177 | 100.3 | 0.771 | Light Finnish suffix stemmer. |
|
| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 175.250 | 46.995 | 99.9 | 0.569 | Light Finnish suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[FINNISH]` | 265.579 | 95.687 | 151.4 | 1.164 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[FINNISH]` | 264.652 | 63.054 | 150.8 | 0.859 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 338.099 | 175.101 | 192.7 | 1.481 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 374.883 | 238.157 | 213.6 | 1.217 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 94.831% | 94.859% | 94.734% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 94.831% | 94.859% | 94.734% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 68.923% | 63.617% | 86.876% | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene FrenchMinimalStemFilter | 11.472% | 6.236% | 29.192% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
| Lucene FrenchMinimalStemFilter | 11.472% | 6.236% | 29.192% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
||||||
| Lucene SnowballFilter | 8.551% | 5.183% | 19.952% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene SnowballFilter | 8.551% | 5.183% | 19.952% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
| Official Snowball direct | 8.462% | 5.067% | 19.952% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Official Snowball direct | 8.462% | 5.067% | 19.952% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
@@ -42,11 +43,12 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `frenchRadixor` | 38.598 | 5.425 | 105.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `frenchRadixor` | 47.033 | 4.146 | 128.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 17.657 | 1.956 | 48.2 | 0.457 | Minimal French suffix reducer; narrow baseline. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1664.935 | 65.928 | 4549.4 | 35.399 | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 28.742 | 2.391 | 78.5 | 0.745 | Light French suffix stemmer. |
|
| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 19.234 | 2.098 | 52.6 | 0.409 | Minimal French suffix reducer; narrow baseline. |
|
||||||
| Official Snowball direct | `snowballDirect[FRENCH]` | 104.983 | 10.116 | 286.9 | 2.720 | Official Snowball generated Java stemmer; direct API. |
|
| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 30.560 | 3.680 | 83.5 | 0.650 | Light French suffix stemmer. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 117.938 | 4.007 | 322.3 | 3.056 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Official Snowball direct | `snowballDirect[FRENCH]` | 111.057 | 8.172 | 303.5 | 2.361 | Official Snowball generated Java stemmer; direct API. |
|
||||||
|
| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 123.648 | 3.500 | 337.9 | 2.629 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,16 +26,18 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 97.455% | 97.973% | 96.476% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 92.725% | 92.847% | 92.396% | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene GermanLightStemFilter | 38.583% | 35.800% | 43.849% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
| Lucene HunspellStemFilter | 47.064% | 29.661% | 93.678% | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene GermanMinimalStemFilter | 37.492% | 38.538% | 35.513% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
| CISTEM (German) | 24.675% | 23.724% | 27.222% | Benchmark-only CISTEM implementation. |
|
||||||
| Lucene SnowballFilter | 33.380% | 30.939% | 37.999% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene GermanLightStemFilter | 37.434% | 35.465% | 42.707% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||||
| Official Snowball direct | 32.863% | 31.225% | 35.963% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Lucene GermanMinimalStemFilter | 27.640% | 24.951% | 34.844% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
||||||
| Lucene GermanStemFilter | 26.168% | 24.979% | 28.416% | German Lucene stemming TokenFilter; broader than minimal/light variants. |
|
| Lucene SnowballFilter | 30.956% | 28.853% | 36.589% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
|
| Official Snowball direct | 30.481% | 29.027% | 34.376% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
|
| Lucene GermanStemFilter | 21.559% | 19.312% | 27.576% | German Lucene stemming TokenFilter; broader than minimal/light variants. |
|
||||||
|
|
||||||
## Speed
|
## Speed
|
||||||
|
|
||||||
@@ -43,12 +45,14 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `germanRadixor` | 9.518 | 0.338 | 68.2 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `germanRadixor` | 41.166 | 2.396 | 294.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 12.020 | 0.460 | 86.1 | 1.263 | Minimal German suffix reduction; narrow baseline. |
|
| CISTEM | `germanCistem` | 248.392 | 12.294 | 1778.8 | 6.034 | Benchmark-only CISTEM implementation. |
|
||||||
| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 12.413 | 1.059 | 88.9 | 1.304 | Light German suffix stemmer; narrower than a dictionary stemmer. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 281.322 | 3.411 | 2014.6 | 6.834 | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 36.644 | 5.667 | 262.4 | 3.850 | Older German stemming TokenFilter with normalization requirements. |
|
| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 23.562 | 0.969 | 168.7 | 0.572 | Minimal German suffix reduction; narrow baseline. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 54.846 | 8.710 | 392.8 | 5.762 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 24.410 | 1.034 | 174.8 | 0.593 | Light German suffix stemmer; narrower than a dictionary stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[GERMAN]` | 52.974 | 7.989 | 379.4 | 5.566 | Official Snowball generated Java stemmer; direct API. |
|
| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 71.039 | 4.443 | 508.7 | 1.726 | Older German stemming TokenFilter with normalization requirements. |
|
||||||
|
| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 105.771 | 9.617 | 757.4 | 2.569 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
| Official Snowball direct | `snowballDirect[GERMAN]` | 100.688 | 9.018 | 721.0 | 2.446 | Official Snowball generated Java stemmer; direct API. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -41,10 +41,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `hungarianRadixor` | 53.844 | 5.619 | 60.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `hungarianRadixor` | 62.232 | 6.412 | 69.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 85.802 | 4.554 | 95.7 | 1.594 | Light Hungarian suffix stemmer. |
|
| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 92.813 | 6.929 | 103.5 | 1.491 | Light Hungarian suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 161.996 | 61.038 | 180.6 | 3.009 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 157.765 | 13.202 | 175.9 | 2.535 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 185.097 | 44.447 | 206.4 | 3.438 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 188.863 | 15.880 | 210.6 | 3.035 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -40,10 +40,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `italianRadixor` | 23.776 | 9.977 | 74.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `italianRadixor` | 24.491 | 3.128 | 77.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 14.940 | 1.682 | 47.1 | 0.628 | Light Italian suffix stemmer. |
|
| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 15.977 | 1.041 | 50.3 | 0.652 | Light Italian suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[ITALIAN]` | 99.401 | 9.433 | 313.0 | 4.181 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[ITALIAN]` | 109.526 | 12.572 | 344.9 | 4.472 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 108.462 | 8.014 | 341.6 | 4.562 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 116.260 | 7.459 | 366.1 | 4.747 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -42,11 +42,11 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `norwegianBokmalRadixor` | 3.235 | 0.147 | 56.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `norwegianBokmalRadixor` | 3.631 | 1.377 | 63.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.720 | 0.190 | 47.4 | 0.841 | Minimal Norwegian suffix reducer. |
|
| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.910 | 0.177 | 50.7 | 0.801 | Minimal Norwegian suffix reducer. |
|
||||||
| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.189 | 0.261 | 55.6 | 0.986 | Light Norwegian suffix stemmer. |
|
| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.335 | 0.116 | 58.1 | 0.919 | Light Norwegian suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 3.978 | 0.031 | 69.3 | 1.230 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 4.277 | 0.082 | 74.5 | 1.178 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 5.526 | 0.409 | 96.3 | 1.708 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 6.077 | 0.208 | 105.9 | 1.674 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,11 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 93.089% | 91.395% | 96.863% | Radixor baseline in the Snowball-language comparison family. |
|
| Radixor | 93.089% | 91.395% | 96.863% | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Official Snowball direct | 60.974% | 60.212% | 62.670% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Official Snowball direct | 60.974% | 60.212% | 62.670% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
| Lucene SnowballFilter | 60.918% | 60.146% | 62.638% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene SnowballFilter | 60.918% | 60.146% | 62.638% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
|
|
||||||
@@ -40,9 +40,9 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.541 | 0.021 | 39.9 | 1.000 | Radixor baseline for the Snowball-language comparison family. |
|
| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.571 | 0.012 | 42.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 0.855 | 0.008 | 63.0 | 1.580 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 0.919 | 0.038 | 67.7 | 1.609 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.221 | 0.025 | 90.0 | 2.258 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.309 | 0.015 | 96.5 | 2.292 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -37,8 +37,8 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `persianRadixor` | 0.231 | 0.021 | 63.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `persianRadixor` | 0.245 | 0.025 | 49.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.443 | 0.012 | 122.0 | 1.918 | Persian suffix stemmer with Lucene normalization in the measured path. |
|
| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.466 | 0.015 | 93.3 | 1.902 | Persian suffix stemmer with Lucene normalization in the measured path. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 98.837% | 98.744% | 99.359% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 98.837% | 98.744% | 99.359% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 89.545% | 88.272% | 96.713% | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene MorfologikFilter | 87.729% | 86.606% | 94.047% | Dictionary-based path; Morfologik can emit multiple terms. |
|
| Lucene MorfologikFilter | 87.729% | 86.606% | 94.047% | Dictionary-based path; Morfologik can emit multiple terms. |
|
||||||
| Lucene StempelFilter | 70.009% | 69.262% | 74.220% | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
| Lucene StempelFilter | 70.009% | 69.262% | 74.220% | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
||||||
| Lucene StempelStemmer direct | 70.009% | 69.262% | 74.220% | Direct table-driven Polish Stempel stemmer API. |
|
| Lucene StempelStemmer direct | 70.009% | 69.262% | 74.220% | Direct table-driven Polish Stempel stemmer API. |
|
||||||
@@ -41,10 +42,11 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `polishRadixor` | 7.760 | 0.240 | 69.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `polishRadixor` | 9.049 | 0.485 | 80.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 34.295 | 0.418 | 305.3 | 4.420 | Direct table-driven Polish Stempel stemmer API. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 483.316 | 11.455 | 4301.8 | 53.408 | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene StempelFilter | `polishLuceneStempelFilter` | 39.116 | 1.717 | 348.2 | 5.041 | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 41.932 | 1.916 | 373.2 | 4.634 | Direct table-driven Polish Stempel stemmer API. |
|
||||||
| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 128.516 | 12.557 | 1143.9 | 16.562 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
| Lucene StempelFilter | `polishLuceneStempelFilter` | 45.277 | 13.693 | 403.0 | 5.003 | Lucene TokenFilter integration path for table-driven Polish Stempel. |
|
||||||
|
| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 135.763 | 31.634 | 1208.4 | 15.002 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -43,12 +43,12 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `portugueseRadixor` | 10.598 | 0.273 | 51.1 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `portugueseRadixor` | 12.109 | 0.698 | 58.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 10.101 | 0.389 | 48.7 | 0.953 | Light Portuguese suffix stemmer. |
|
| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 11.172 | 1.870 | 53.8 | 0.923 | Light Portuguese suffix stemmer. |
|
||||||
| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 14.493 | 0.760 | 69.8 | 1.367 | Minimal Portuguese suffix reducer. |
|
| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 16.038 | 1.752 | 77.3 | 1.325 | Minimal Portuguese suffix reducer. |
|
||||||
| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 52.508 | 6.338 | 253.1 | 4.954 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 53.725 | 5.356 | 258.9 | 4.437 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 54.048 | 1.173 | 260.5 | 5.100 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 57.457 | 1.182 | 276.9 | 4.745 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 141.208 | 12.785 | 680.6 | 13.324 | Portuguese RSLP-style Lucene TokenFilter. |
|
| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 165.447 | 40.334 | 797.4 | 13.663 | Portuguese RSLP-style Lucene TokenFilter. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -41,10 +41,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `russianRadixor` | 72.970 | 15.642 | 99.8 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `russianRadixor` | 89.671 | 3.886 | 122.6 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 57.900 | 4.404 | 79.2 | 0.793 | Light Russian suffix stemmer. |
|
| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 60.522 | 5.310 | 82.7 | 0.675 | Light Russian suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[RUSSIAN]` | 99.019 | 14.688 | 135.4 | 1.357 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[RUSSIAN]` | 106.031 | 9.287 | 145.0 | 1.182 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 128.272 | 7.201 | 175.4 | 1.758 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 137.512 | 10.801 | 188.0 | 1.534 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 97.459% | 97.544% | 96.891% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 97.459% | 97.544% | 96.891% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 49.074% | 42.656% | 92.154% | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene SpanishMinimalStemFilter | 17.284% | 5.347% | 97.403% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
| Lucene SpanishMinimalStemFilter | 17.284% | 5.347% | 97.403% | Minimal suffix reducer; narrow baseline, not a full stemmer. |
|
||||||
| Lucene SpanishPluralStemFilter | 15.140% | 5.802% | 77.820% | Plural-focused suffix reducer; narrow baseline. |
|
| Lucene SpanishPluralStemFilter | 15.140% | 5.802% | 77.820% | Plural-focused suffix reducer; narrow baseline. |
|
||||||
| Lucene SpanishLightStemFilter | 9.577% | 7.088% | 26.279% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
| Lucene SpanishLightStemFilter | 9.577% | 7.088% | 26.279% | Light suffix stemmer; intentionally narrower than a dictionary-derived stemmer. |
|
||||||
@@ -43,12 +44,13 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `spanishRadixor` | 64.539 | 3.448 | 80.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `spanishRadixor` | 78.919 | 7.253 | 97.9 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 38.288 | 2.382 | 47.5 | 0.593 | Minimal Spanish suffix reducer; narrow baseline. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 2079.041 | 193.548 | 2578.6 | 26.344 | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 40.855 | 2.407 | 50.7 | 0.633 | Light Spanish suffix stemmer. |
|
| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 45.596 | 4.639 | 56.6 | 0.578 | Minimal Spanish suffix reducer; narrow baseline. |
|
||||||
| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 91.054 | 8.822 | 112.9 | 1.411 | Plural-oriented Spanish suffix reducer. |
|
| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 42.003 | 1.683 | 52.1 | 0.532 | Light Spanish suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[SPANISH]` | 168.813 | 32.860 | 209.4 | 2.616 | Official Snowball generated Java stemmer; direct API. |
|
| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 93.734 | 6.247 | 116.3 | 1.188 | Plural-oriented Spanish suffix reducer. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 185.626 | 50.297 | 230.2 | 2.876 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Official Snowball direct | `snowballDirect[SPANISH]` | 171.995 | 11.035 | 213.3 | 2.179 | Official Snowball generated Java stemmer; direct API. |
|
||||||
|
| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 211.138 | 17.940 | 261.9 | 2.675 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
@@ -42,11 +42,11 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `swedishRadixor` | 4.916 | 0.525 | 57.3 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `swedishRadixor` | 5.489 | 0.355 | 64.0 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.453 | 0.530 | 51.9 | 0.906 | Minimal Swedish suffix reducer. |
|
| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.630 | 0.130 | 54.0 | 0.843 | Minimal Swedish suffix reducer. |
|
||||||
| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.523 | 0.151 | 52.8 | 0.920 | Light Swedish suffix stemmer. |
|
| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.876 | 0.328 | 56.9 | 0.888 | Light Swedish suffix stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[SWEDISH]` | 7.075 | 0.541 | 82.5 | 1.439 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[SWEDISH]` | 7.517 | 0.072 | 87.7 | 1.370 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 9.379 | 0.056 | 109.4 | 1.908 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 9.793 | 0.338 | 114.2 | 1.784 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -26,11 +26,12 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 99.307% | 99.365% | 99.062% | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | 99.307% | 99.365% | 99.062% | Full Radixor dictionary patch-command stemmer. |
|
||||||
|
| Lucene HunspellStemFilter | 86.815% | 83.759% | 99.866% | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene MorfologikFilter | 92.362% | 90.637% | 99.732% | Dictionary-based path; Morfologik can emit multiple terms. |
|
| Lucene MorfologikFilter | 92.362% | 90.637% | 99.732% | Dictionary-based path; Morfologik can emit multiple terms. |
|
||||||
| Morfologik direct | 92.362% | 90.637% | 99.732% | Direct dictionary lookup; first returned stem is used for quality when no ranking weight is exposed. |
|
| Morfologik direct | 92.362% | 90.637% | 99.732% | Direct dictionary lookup; first returned stem is used for quality when no ranking weight is exposed. |
|
||||||
|
|
||||||
@@ -40,9 +41,10 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `ukrainianRadixor` | 0.605 | 0.056 | 47.4 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
| Radixor | `ukrainianRadixor` | 0.682 | 0.057 | 53.5 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Morfologik direct | `ukrainianMorfologikDirect` | 8.106 | 0.040 | 635.7 | 13.408 | Direct Morfologik dictionary lookup; first returned stem is used for quality. |
|
| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 43.527 | 1.207 | 3413.3 | 63.799 | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. |
|
||||||
| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 14.684 | 5.214 | 1151.5 | 24.287 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
| Morfologik direct | `ukrainianMorfologikDirect` | 8.680 | 0.073 | 680.7 | 12.723 | Direct Morfologik dictionary lookup; first returned stem is used for quality. |
|
||||||
|
| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 14.575 | 0.248 | 1143.0 | 21.364 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -25,11 +25,11 @@ Radixor stores the preferred transformation for each normalized dictionary word
|
|||||||
|
|
||||||
## Accuracy
|
## Accuracy
|
||||||
|
|
||||||
Accuracy is computed from one deterministic JMH measurement iteration without warmup. The benchmark may execute the full dictionary pass more than once inside that single timed iteration; percentages divide matching counters by evaluated counters from the same iteration.
|
Accuracy is computed from JMH auxiliary counters in the current report. The counters are deterministic for a fixed corpus and stemmer; percentages divide matching counters by evaluated counters from the same report and are not timing metrics.
|
||||||
|
|
||||||
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
| Stemmer | All exact | Changed exact | Root preserved | Note |
|
||||||
| --- | ---: | ---: | ---: | --- |
|
| --- | ---: | ---: | ---: | --- |
|
||||||
| Radixor | 98.930% | 98.343% | 100.000% | Radixor baseline in the Snowball-language comparison family. |
|
| Radixor | 98.930% | 98.343% | 100.000% | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Lucene SnowballFilter | 2.837% | 2.558% | 3.346% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
| Lucene SnowballFilter | 2.837% | 2.558% | 3.346% | Lucene TokenFilter integration path around the Snowball algorithm. |
|
||||||
| Official Snowball direct | 2.837% | 2.558% | 3.346% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
| Official Snowball direct | 2.837% | 2.558% | 3.346% | Official Snowball generated Java stemmer; rule-based suffix algorithm. |
|
||||||
|
|
||||||
@@ -39,9 +39,9 @@ Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 1 fo
|
|||||||
|
|
||||||
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note |
|
||||||
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
| --- | --- | ---: | ---: | ---: | ---: | --- |
|
||||||
| Radixor | `radixor[YIDDISH]` | 0.236 | 0.004 | 85.1 | 1.000 | Radixor baseline for the Snowball-language comparison family. |
|
| Radixor | `radixor[YIDDISH]` | 0.254 | 0.004 | 50.7 | 1.000 | Full Radixor dictionary patch-command stemmer. |
|
||||||
| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.432 | 0.193 | 515.7 | 6.058 | Official Snowball generated Java stemmer; direct API. |
|
| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.537 | 0.220 | 307.3 | 6.058 | Official Snowball generated Java stemmer; direct API. |
|
||||||
| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.595 | 0.068 | 574.6 | 6.749 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.714 | 0.120 | 342.8 | 6.756 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. |
|
||||||
|
|
||||||
## Interpretation Notes
|
## Interpretation Notes
|
||||||
|
|
||||||
|
|||||||
@@ -4,20 +4,28 @@ Implemented benchmark methods are documented on the per-language pages under [La
|
|||||||
|
|
||||||
## Included Candidate Families
|
## Included Candidate Families
|
||||||
|
|
||||||
The current benchmark pages include Radixor baselines, Lucene language filters where the language matches a bundled Radixor resource, Lucene Stempel and Morfologik paths where applicable, official Snowball Java stemmers where same-language comparison is available, and selected English-specific non-Lucene baselines such as OpenNLP Porter and Paice/Husk Lancaster.
|
The current benchmark pages include Radixor baselines, Lucene language filters where the language matches a bundled Radixor resource, Lucene Stempel and Morfologik paths where applicable, official Snowball Java stemmers where same-language comparison is available, benchmark-only CISTEM German stemmer evaluation, benchmark-only Hunspell comparisons, and selected English-specific non-Lucene baselines such as OpenNLP Porter and Paice/Husk Lancaster.
|
||||||
|
|
||||||
|
Benchmark-only Hunspell comparisons use bundled benchmark dictionaries and the Lucene HunspellStemFilter adapter over the selected language token streams.
|
||||||
|
The CISTEM candidate is implemented in `src/jmh/java/org/egothor/stemmer/benchmark/Cistem.java` and follows the original MIT-licensed upstream implementation from Leonie Weissweiler's CISTEM project.
|
||||||
|
CISTEM German gold-standard files are not vendored in this repository. The Gradle JMH resource preparation tasks download `goldstandard1.txt` and `goldstandard2.txt` from the upstream CISTEM repository into generated build resources.
|
||||||
|
|
||||||
Direct stemmer APIs and Lucene TokenFilter paths are documented separately on language pages. TokenFilter rows include TokenStream, attribute, and required normalization overhead. Direct rows measure exposed direct APIs.
|
Direct stemmer APIs and Lucene TokenFilter paths are documented separately on language pages. TokenFilter rows include TokenStream, attribute, and required normalization overhead. Direct rows measure exposed direct APIs.
|
||||||
|
|
||||||
|
For the benchmark refresh used in this documentation build:
|
||||||
|
|
||||||
|
- Hunspell families are included in `HunspellStemmerComparisonBenchmark` (speed) and `HunspellStemmerComparisonBenchmarkQuality` (quality for all benchmark languages in this corpus). The legacy
|
||||||
|
`EnglishHunspellStemmerComparisonBenchmarkQuality` result is retained for continuity.
|
||||||
|
- CISTEM quality is present in the published per-language results under `GERMAN_CISTEM`. CISTEM speed is present as `germanCistem` in `MultiLanguageStemmerComparisonBenchmark`.
|
||||||
|
|
||||||
## Evaluated But Skipped Candidates
|
## Evaluated But Skipped Candidates
|
||||||
|
|
||||||
| Candidate | Language | Link/source | Reason skipped |
|
| Candidate | Language | Link/source | Reason skipped |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| Lucene Arabic, Bulgarian, Bengali, Sorani, Greek, Galician, Hindi, Indonesian, Latvian, Telugu filters | Various | `lucene-analysis-common` | No bundled same-language Radixor resource in this repository snapshot. |
|
| Lucene Arabic, Bulgarian, Bengali, Sorani, Greek, Galician, Hindi, Indonesian, Latvian, Telugu filters | Various | `lucene-analysis-common` | No bundled same-language Radixor resource in this repository snapshot. |
|
||||||
| Lucene analyzer-only paths | Multiple | Lucene analyzers | Full analyzers mix tokenization, stop-word handling, and other behavior; direct filters are used where available. |
|
| Lucene analyzer-only paths | Multiple | Lucene analyzers | Full analyzers mix tokenization, stop-word handling, and other behavior; direct filters are used where available. |
|
||||||
| Lucene HunspellStemFilter | Multiple | `lucene-analysis-common` | Requires external Hunspell dictionaries not resolved as benchmark-only resources here. |
|
|
||||||
| Lucene StemmerOverrideFilter | Multiple | `lucene-analysis-common` | Override map facility, not a stemmer algorithm. |
|
| Lucene StemmerOverrideFilter | Multiple | `lucene-analysis-common` | Override map facility, not a stemmer algorithm. |
|
||||||
| Additional Snowball Lovins | English | Official Snowball Java distribution | No Lovins Java stemmer was present in the selected Snowball Java distribution. |
|
| Additional Snowball Lovins | English | Official Snowball Java distribution | No Lovins Java stemmer was present in the selected Snowball Java distribution. |
|
||||||
| Lemur Project Krovetz Stemmer | English | Lemur Project | Lucene KStem represents the Krovetz-style path without adding separate dependency and license risk. |
|
| Lemur Project Krovetz Stemmer | English | Lemur Project | Lucene KStem represents the Krovetz-style path without adding separate dependency and license risk. |
|
||||||
| Smile Lancaster / Paice-Husk | English | Smile NLP | Smile is large for one stemmer; Paice/Husk is included through a smaller benchmark-only generated path. |
|
| Smile Lancaster / Paice-Husk | English | Smile NLP | Smile is large for one stemmer; Paice/Husk is included through a smaller benchmark-only generated path. |
|
||||||
| CISTEM German stemmer | German | `https://github.com/LeonieWeissweiler/CISTEM` | Clean benchmark-only Java integration was not completed in this phase. |
|
|
||||||
| `stemmerEval` reference repository | Multiple | `https://github.com/endredy/stemmerEval` | Used only as a candidate reference; no code or data copied. |
|
| `stemmerEval` reference repository | Multiple | `https://github.com/endredy/stemmerEval` | Used only as a candidate reference; no code or data copied. |
|
||||||
|
|||||||
@@ -6,16 +6,16 @@ This benchmark is the clearest demonstration of the Radixor quality/speed envelo
|
|||||||
|
|
||||||
| Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token |
|
| Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token |
|
||||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||||
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 23.113 | 7.065 | 109.8 |
|
| 100% | 100.000% | 97.478% | 97.197% | 97.552% | 28.578 | 7.571 | 135.8 |
|
||||||
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 21.270 | 9.914 | 101.0 |
|
| 90% | 90.000% | 97.047% | 94.913% | 97.613% | 26.612 | 9.227 | 126.4 |
|
||||||
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 19.170 | 6.609 | 91.1 |
|
| 80% | 80.000% | 96.635% | 92.768% | 97.661% | 23.331 | 8.106 | 110.8 |
|
||||||
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 20.857 | 6.734 | 99.1 |
|
| 70% | 70.000% | 96.209% | 90.565% | 97.705% | 22.362 | 1.957 | 106.2 |
|
||||||
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 14.975 | 1.215 | 71.1 |
|
| 60% | 60.000% | 95.750% | 88.384% | 97.703% | 16.497 | 2.026 | 78.4 |
|
||||||
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 15.249 | 1.078 | 72.4 |
|
| 50% | 50.000% | 95.262% | 86.107% | 97.690% | 16.035 | 0.986 | 76.2 |
|
||||||
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 15.323 | 2.340 | 72.8 |
|
| 40% | 40.000% | 94.753% | 83.855% | 97.643% | 16.459 | 0.664 | 78.2 |
|
||||||
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 16.778 | 2.643 | 79.7 |
|
| 30% | 30.000% | 94.208% | 81.651% | 97.537% | 19.566 | 0.758 | 92.9 |
|
||||||
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 18.929 | 3.241 | 89.9 |
|
| 20% | 20.000% | 93.633% | 79.366% | 97.416% | 14.616 | 0.487 | 69.4 |
|
||||||
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 19.124 | 1.883 | 90.9 |
|
| 10% | 10.000% | 92.868% | 76.516% | 97.204% | 18.093 | 3.147 | 86.0 |
|
||||||
|
|
||||||
## Column Meanings
|
## Column Meanings
|
||||||
|
|
||||||
|
|||||||
@@ -4,11 +4,11 @@ The values below are environment-specific and must not be read as universal perf
|
|||||||
|
|
||||||
| Item | Value |
|
| Item | Value |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Benchmark date | 2026-07-03 |
|
| Benchmark date | 2026-07-06 (Europe/Prague) |
|
||||||
| Focused comparison command family | JMH jar runs limited to `EnglishStemmerComparisonBenchmark`, `MultiLanguageStemmerComparisonBenchmark`, and `SnowballLanguageStemmerComparisonBenchmark`; Radixor exact-root metrics were recomputed deterministically against the same contracted loaders |
|
| Focused comparison command family | `./gradlew jmh -Pjmh.includes='.*StemmerComparisonBenchmark.*' --no-daemon` |
|
||||||
| English coverage command | `./gradlew jmh -Pjmh.includes='.*EnglishRadixorDictionaryCoverageBenchmark.*' --no-daemon` |
|
| English coverage command | `./gradlew jmh -Pjmh.includes='.*EnglishRadixorDictionaryCoverageBenchmark.*' --no-daemon` |
|
||||||
| Speed result reports | `build/reports/jmh/contracted/english-comparison.csv`, `multilanguage-speed.csv`, `snowball-language-speed.csv` |
|
| Speed result reports | `build/reports/jmh/stemmer-comparison-2026-07-06.csv`, `build/reports/jmh/stemmer-comparison-2026-07-06.txt`, `build/reports/jmh/english-coverage-2026-07-06.csv`, and `build/reports/jmh/english-coverage-2026-07-06.txt` |
|
||||||
| Accuracy result reports | Deterministic Radixor exact-root pass over bundled dictionaries; non-Radixor quality rows retained from the existing published quality suite |
|
| Accuracy result reports | `build/reports/jmh/stemmer-comparison-2026-07-06.csv`, `build/reports/jmh/english-coverage-2026-07-06.csv`, and deterministic Radixor exact-root accounting over the same bundled language corpora |
|
||||||
| Final comparison JMH scope | Stemmer comparison benchmarks only; internal `FrequencyTrie*` microbenchmarks were not run |
|
| Final comparison JMH scope | Stemmer comparison benchmarks only; internal `FrequencyTrie*` microbenchmarks were not run |
|
||||||
| Coverage JMH scope | English Radixor dictionary coverage benchmark only |
|
| Coverage JMH scope | English Radixor dictionary coverage benchmark only |
|
||||||
| JMH version | 1.37 |
|
| JMH version | 1.37 |
|
||||||
@@ -16,15 +16,19 @@ The values below are environment-specific and must not be read as universal perf
|
|||||||
| Score unit | `ns/op` |
|
| Score unit | `ns/op` |
|
||||||
| Speed warmup | 3 iterations, 1 s each |
|
| Speed warmup | 3 iterations, 1 s each |
|
||||||
| Speed measurement | 5 iterations, 1 s each |
|
| Speed measurement | 5 iterations, 1 s each |
|
||||||
| Accuracy warmup | none for deterministic exact-root accounting |
|
| Accuracy warmup | 3 JMH warmup iterations were applied by the Gradle invocation; timing scores from quality methods are not interpreted |
|
||||||
| Accuracy measurement | 1 deterministic measurement iteration; counters only, not speed interpretation |
|
| Accuracy measurement | 5 JMH measurement samples; documentation uses deterministic auxiliary counter ratios from the same report |
|
||||||
| Fork count in generated report files | 1 |
|
| Fork count in generated report files | 1 |
|
||||||
| Default fork policy for accuracy-only benchmark classes | `@Fork(0)` for future default runs because accuracy counters are deterministic and not interpreted as speed |
|
| Default fork policy for accuracy-only benchmark classes | `@Fork(0)` for future default runs because accuracy counters are deterministic and not interpreted as speed |
|
||||||
| Thread count | 1 |
|
| Thread count | 1 |
|
||||||
| JVM reported by JMH | OpenJDK 64-Bit Server VM, 25.0.3+9 |
|
| JVM reported by JMH | JDK 25.0.3, OpenJDK 64-Bit Server VM, 25.0.3+9 |
|
||||||
|
| Java runtime | OpenJDK Runtime Environment, Red Hat build 25.0.3+9 |
|
||||||
| JVM invoker | `/usr/lib/jvm/java-25-openjdk/bin/java` |
|
| JVM invoker | `/usr/lib/jvm/java-25-openjdk/bin/java` |
|
||||||
| Operating system | Linux 7.0.13-200.fc44.x86_64 |
|
| Operating system | Fedora Linux 44 (MATE-Compiz) |
|
||||||
| CPU | AMD Ryzen 5 7600 6-Core Processor |
|
| Kernel | Linux 7.0.12-201.fc44.x86_64 |
|
||||||
|
| Architecture | x86_64 |
|
||||||
|
| CPU | AMD Ryzen 5 8600G w/ Radeon 760M Graphics |
|
||||||
|
| Physical cores | 6 |
|
||||||
| Logical CPUs | 12 |
|
| Logical CPUs | 12 |
|
||||||
|
|
||||||
## Contracted Trie Baseline
|
## Contracted Trie Baseline
|
||||||
@@ -35,12 +39,10 @@ All Radixor rows in the refreshed benchmark tables use contracted compiled patch
|
|||||||
|
|
||||||
Generated local report files for this benchmark update:
|
Generated local report files for this benchmark update:
|
||||||
|
|
||||||
- `build/reports/jmh/contracted/english-comparison.csv`
|
- `build/reports/jmh/stemmer-comparison-2026-07-06.csv`
|
||||||
- `build/reports/jmh/contracted/english-comparison.txt`
|
- `build/reports/jmh/stemmer-comparison-2026-07-06.txt`
|
||||||
- `build/reports/jmh/contracted/multilanguage-speed.csv`
|
- `build/reports/jmh/english-coverage-2026-07-06.csv`
|
||||||
- `build/reports/jmh/contracted/multilanguage-speed.txt`
|
- `build/reports/jmh/english-coverage-2026-07-06.txt`
|
||||||
- `build/reports/jmh/contracted/snowball-language-speed.csv`
|
|
||||||
- `build/reports/jmh/contracted/snowball-language-speed.txt`
|
|
||||||
|
|
||||||
JMH TXT and CSV reports are still published as benchmark artifacts. They are not converted into a Porter speed badge.
|
JMH TXT and CSV reports are still published as benchmark artifacts. They are not converted into a Porter speed badge.
|
||||||
|
|
||||||
|
|||||||
@@ -56,4 +56,4 @@ rootPreservedPercent = rootPreservedMatches / rootEvaluatedTokens * 100
|
|||||||
|
|
||||||
Morfologik can emit multiple terms for one input token. The quality benchmark uses the first emitted term for exact-root accounting when no ranking weight is exposed. Throughput benchmarks for Morfologik TokenFilter paths consume all emitted terms.
|
Morfologik can emit multiple terms for one input token. The quality benchmark uses the first emitted term for exact-root accounting when no ranking weight is exposed. Throughput benchmarks for Morfologik TokenFilter paths consume all emitted terms.
|
||||||
|
|
||||||
Quality reports intentionally use one deterministic measurement iteration without warmup, because exact-root agreement is not a timing metric and repeated precision passes would only duplicate the same counters.
|
Quality reports use JMH auxiliary counter rows. Exact-root accounting is deterministic for a fixed corpus and stemmer, so repeated measurement samples duplicate the same counters; documentation uses the counter ratios and does not interpret quality benchmark timing scores.
|
||||||
|
|||||||
69
gradle/cistem-benchmarks.gradle
Normal file
69
gradle/cistem-benchmarks.gradle
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
def cistemGoldStandardBaseUrl = 'https://raw.githubusercontent.com/LeonieWeissweiler/CISTEM/refs/heads/master/gold_standards'
|
||||||
|
def cistemGoldStandardFiles = [
|
||||||
|
'goldstandard1.txt',
|
||||||
|
'goldstandard2.txt'
|
||||||
|
]
|
||||||
|
def cistemGoldStandardDownloadDirectory = layout.buildDirectory.dir('third-party/cistem-gold-standards')
|
||||||
|
def cistemGoldStandardGeneratedResourcesDirectory = layout.buildDirectory.dir('generated/resources/cistem-gold-standards')
|
||||||
|
|
||||||
|
def cistemGoldStandardDownloadedFiles = cistemGoldStandardFiles.collect { String fileName ->
|
||||||
|
cistemGoldStandardDownloadDirectory.map { it.file(fileName) }
|
||||||
|
}
|
||||||
|
|
||||||
|
tasks.register('downloadCistemGoldStandards') {
|
||||||
|
group = 'build setup'
|
||||||
|
description = 'Downloads benchmark-only CISTEM German gold standards.'
|
||||||
|
|
||||||
|
outputs.files(cistemGoldStandardDownloadedFiles)
|
||||||
|
|
||||||
|
doLast {
|
||||||
|
cistemGoldStandardFiles.each { String fileName ->
|
||||||
|
final File targetFile = cistemGoldStandardDownloadDirectory.get().file(fileName).asFile
|
||||||
|
targetFile.parentFile.mkdirs()
|
||||||
|
|
||||||
|
if (!targetFile.exists()) {
|
||||||
|
final URL sourceUrl = new URL("${cistemGoldStandardBaseUrl}/${fileName}")
|
||||||
|
try {
|
||||||
|
sourceUrl.withInputStream { inputStream ->
|
||||||
|
targetFile.withOutputStream { outputStream ->
|
||||||
|
outputStream << inputStream
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (FileNotFoundException exception) {
|
||||||
|
throw new GradleException(
|
||||||
|
"Unable to download CISTEM gold standard ${fileName} from ${sourceUrl}.",
|
||||||
|
exception)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (targetFile.length() <= 0L) {
|
||||||
|
throw new GradleException("Downloaded CISTEM gold standard ${fileName} was empty.")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
tasks.register('prepareCistemGoldStandardResources', Copy) {
|
||||||
|
group = 'build setup'
|
||||||
|
description = 'Copies benchmark-only CISTEM German gold standards into the JMH resource output.'
|
||||||
|
|
||||||
|
dependsOn(tasks.named('downloadCistemGoldStandards'))
|
||||||
|
|
||||||
|
from(cistemGoldStandardDownloadDirectory) {
|
||||||
|
include 'goldstandard1.txt'
|
||||||
|
include 'goldstandard2.txt'
|
||||||
|
}
|
||||||
|
into(cistemGoldStandardGeneratedResourcesDirectory)
|
||||||
|
}
|
||||||
|
|
||||||
|
sourceSets {
|
||||||
|
jmh {
|
||||||
|
resources {
|
||||||
|
srcDir(cistemGoldStandardGeneratedResourcesDirectory)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
tasks.named('processJmhResources') {
|
||||||
|
dependsOn(tasks.named('prepareCistemGoldStandardResources'))
|
||||||
|
}
|
||||||
178
src/jmh/java/org/egothor/stemmer/benchmark/Cistem.java
Normal file
178
src/jmh/java/org/egothor/stemmer/benchmark/Cistem.java
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
/*******************************************************************************
|
||||||
|
* MIT License
|
||||||
|
*
|
||||||
|
* Copyright (c) 2017 Leonie Weißweiler
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
* of this software and associated documentation files (the "Software"), to deal
|
||||||
|
* in the Software without restriction, including without limitation the rights
|
||||||
|
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
* copies of the Software, and to permit persons to whom the Software is
|
||||||
|
* furnished to do so, subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
* SOFTWARE.
|
||||||
|
*
|
||||||
|
* Source: CISTEM German stemmer
|
||||||
|
* Authors: Leonie Weissweiler, Alexander Fraser
|
||||||
|
* https://github.com/LeonieWeissweiler/CISTEM
|
||||||
|
* https://www.cis.lmu.de/~weissweiler/cistem/
|
||||||
|
******************************************************************************/
|
||||||
|
package org.egothor.stemmer.benchmark;
|
||||||
|
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
public final class Cistem {
|
||||||
|
|
||||||
|
private static final Pattern GE_PATTERN = Pattern.compile("^ge(.{4,})");
|
||||||
|
private static final Pattern DOLLAR1_PATTERN = Pattern.compile("(.)\\1");
|
||||||
|
private static final Pattern ND_PATTERN = Pattern.compile("nd$");
|
||||||
|
private static final Pattern EMR_PATTERN = Pattern.compile("e[mr]$");
|
||||||
|
private static final Pattern T_PATTERN = Pattern.compile("t$");
|
||||||
|
private static final Pattern ESN_PATTERN = Pattern.compile("[esn]$");
|
||||||
|
private static final Pattern STAR_PATTERN = Pattern.compile("(.)\\*");
|
||||||
|
|
||||||
|
private Cistem() {
|
||||||
|
}
|
||||||
|
|
||||||
|
public static String stem(final String word) {
|
||||||
|
return stem(word, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
public static String stem(final String word, final boolean caseInsensitive) {
|
||||||
|
if (word.isEmpty()) {
|
||||||
|
return word;
|
||||||
|
}
|
||||||
|
|
||||||
|
String normalized = word;
|
||||||
|
normalized = normalized.replace("Ü", "U");
|
||||||
|
normalized = normalized.replace("Ö", "O");
|
||||||
|
normalized = normalized.replace("Ä", "A");
|
||||||
|
normalized = normalized.replace("ü", "u");
|
||||||
|
normalized = normalized.replace("ö", "o");
|
||||||
|
normalized = normalized.replace("ä", "a");
|
||||||
|
|
||||||
|
final boolean uppercase = Character.isUpperCase(normalized.charAt(0));
|
||||||
|
|
||||||
|
normalized = normalized.toLowerCase();
|
||||||
|
normalized = normalized.replace("ß", "ss");
|
||||||
|
normalized = GE_PATTERN.matcher(normalized).replaceAll("$1");
|
||||||
|
normalized = normalized.replace("sch", "$");
|
||||||
|
normalized = normalized.replace("ei", "%");
|
||||||
|
normalized = normalized.replace("ie", "&");
|
||||||
|
normalized = DOLLAR1_PATTERN.matcher(normalized).replaceAll("$1*");
|
||||||
|
|
||||||
|
while (normalized.length() > 3) {
|
||||||
|
if (normalized.length() > 5) {
|
||||||
|
String newWord = EMR_PATTERN.matcher(normalized).replaceAll("");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
newWord = ND_PATTERN.matcher(normalized).replaceAll("");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!uppercase || caseInsensitive) {
|
||||||
|
final String newWord = T_PATTERN.matcher(normalized).replaceAll("");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
final String newWord = ESN_PATTERN.matcher(normalized).replaceAll("");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
normalized = newWord;
|
||||||
|
} else {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
normalized = STAR_PATTERN.matcher(normalized).replaceAll("$1$1");
|
||||||
|
normalized = normalized.replace("&", "ie");
|
||||||
|
normalized = normalized.replace("%", "ei");
|
||||||
|
normalized = normalized.replace("$", "sch");
|
||||||
|
|
||||||
|
return normalized;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static String[] segment(final String word) {
|
||||||
|
return segment(word, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
public static String[] segment(final String word, final boolean caseInsensitive) {
|
||||||
|
if (word.isEmpty()) {
|
||||||
|
return new String[] {"", ""};
|
||||||
|
}
|
||||||
|
|
||||||
|
int restLength = 0;
|
||||||
|
final boolean uppercase = Character.isUpperCase(word.charAt(0));
|
||||||
|
String normalized = word.toLowerCase();
|
||||||
|
final String original = new String(normalized);
|
||||||
|
|
||||||
|
normalized = normalized.replace("sch", "$");
|
||||||
|
normalized = normalized.replace("ei", "%");
|
||||||
|
normalized = normalized.replace("ie", "&");
|
||||||
|
normalized = DOLLAR1_PATTERN.matcher(normalized).replaceAll("$1*");
|
||||||
|
|
||||||
|
while (normalized.length() > 3) {
|
||||||
|
if (normalized.length() > 5) {
|
||||||
|
String newWord = normalized.replaceAll("e[mr]$", "");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
restLength += 2;
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
newWord = normalized.replaceAll("nd$", "");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
restLength += 2;
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!uppercase || caseInsensitive) {
|
||||||
|
final String newWord = normalized.replaceAll("t$", "");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
restLength += 1;
|
||||||
|
normalized = newWord;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
final String newWord = normalized.replaceAll("[esn]$", "");
|
||||||
|
if (!normalized.equals(newWord)) {
|
||||||
|
restLength += 1;
|
||||||
|
normalized = newWord;
|
||||||
|
} else {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
normalized = normalized.replaceAll("(.)\\*", "$1$1");
|
||||||
|
normalized = normalized.replace("&", "ie");
|
||||||
|
normalized = normalized.replace("%", "ei");
|
||||||
|
normalized = normalized.replace("$", "sch");
|
||||||
|
|
||||||
|
String rest = "";
|
||||||
|
if (restLength != 0) {
|
||||||
|
rest = original.substring(original.length() - restLength);
|
||||||
|
}
|
||||||
|
|
||||||
|
return new String[] {normalized, rest};
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,579 @@
|
|||||||
|
/*******************************************************************************
|
||||||
|
* Copyright (C) 2026, Leo Galambos
|
||||||
|
* All rights reserved.
|
||||||
|
*
|
||||||
|
* Redistribution and use in source and binary forms, with or without
|
||||||
|
* modification, are permitted provided that the following conditions are met:
|
||||||
|
*
|
||||||
|
* 1. Redistributions of source code must retain the above copyright notice,
|
||||||
|
* this list of conditions and the following disclaimer.
|
||||||
|
*
|
||||||
|
* 2. Redistributions in binary form must reproduce the above copyright notice,
|
||||||
|
* this list of conditions and the following disclaimer in the documentation
|
||||||
|
* and/or other materials provided with the distribution.
|
||||||
|
*
|
||||||
|
* 3. Neither the name of the copyright holder nor the names of its contributors
|
||||||
|
* may be used to endorse or promote products derived from this software
|
||||||
|
* without specific prior written permission.
|
||||||
|
*
|
||||||
|
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||||
|
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||||
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||||
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||||
|
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||||
|
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||||
|
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||||
|
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||||
|
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||||
|
* POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
******************************************************************************/
|
||||||
|
package org.egothor.stemmer.benchmark;
|
||||||
|
|
||||||
|
import java.io.BufferedReader;
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.InputStream;
|
||||||
|
import java.io.InputStreamReader;
|
||||||
|
import java.nio.charset.StandardCharsets;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.LinkedHashMap;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Objects;
|
||||||
|
import java.util.function.Function;
|
||||||
|
|
||||||
|
import org.apache.lucene.analysis.LowerCaseFilter;
|
||||||
|
import org.apache.lucene.analysis.TokenStream;
|
||||||
|
import org.apache.lucene.analysis.de.GermanLightStemFilter;
|
||||||
|
import org.apache.lucene.analysis.de.GermanMinimalStemFilter;
|
||||||
|
import org.apache.lucene.analysis.de.GermanNormalizationFilter;
|
||||||
|
import org.apache.lucene.analysis.de.GermanStemFilter;
|
||||||
|
import org.apache.lucene.analysis.snowball.SnowballFilter;
|
||||||
|
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
|
||||||
|
import org.apache.lucene.analysis.tokenattributes.PositionIncrementAttribute;
|
||||||
|
import org.openjdk.jmh.annotations.AuxCounters;
|
||||||
|
import org.openjdk.jmh.annotations.Benchmark;
|
||||||
|
import org.openjdk.jmh.annotations.BenchmarkMode;
|
||||||
|
import org.openjdk.jmh.annotations.Fork;
|
||||||
|
import org.openjdk.jmh.annotations.Level;
|
||||||
|
import org.openjdk.jmh.annotations.Measurement;
|
||||||
|
import org.openjdk.jmh.annotations.Mode;
|
||||||
|
import org.openjdk.jmh.annotations.OutputTimeUnit;
|
||||||
|
import org.openjdk.jmh.annotations.Param;
|
||||||
|
import org.openjdk.jmh.annotations.Scope;
|
||||||
|
import org.openjdk.jmh.annotations.Setup;
|
||||||
|
import org.openjdk.jmh.annotations.State;
|
||||||
|
import org.openjdk.jmh.annotations.Warmup;
|
||||||
|
import org.openjdk.jmh.infra.Blackhole;
|
||||||
|
|
||||||
|
import org.egothor.stemmer.FrequencyTrie;
|
||||||
|
import org.egothor.stemmer.ReductionMode;
|
||||||
|
import org.egothor.stemmer.StemmerPatchTrieLoader;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* German-only stemmer comparison on CISTEM gold standards.
|
||||||
|
*
|
||||||
|
* <p>
|
||||||
|
* Each benchmark operation is fed by one cluster file. The same candidate set is
|
||||||
|
* evaluated twice, once per file, to produce one precision/recall/f-measure
|
||||||
|
* table for each gold standard.
|
||||||
|
* </p>
|
||||||
|
*/
|
||||||
|
@BenchmarkMode(Mode.AverageTime)
|
||||||
|
@OutputTimeUnit(java.util.concurrent.TimeUnit.NANOSECONDS)
|
||||||
|
@Warmup(iterations = 3, time = 1, timeUnit = java.util.concurrent.TimeUnit.SECONDS)
|
||||||
|
@Measurement(iterations = 5, time = 1, timeUnit = java.util.concurrent.TimeUnit.SECONDS)
|
||||||
|
@Fork(1)
|
||||||
|
public class GermanGoldstandardStemmerComparisonBenchmark {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shared German benchmark state for one dataset and one candidate.
|
||||||
|
*/
|
||||||
|
@State(Scope.Benchmark)
|
||||||
|
public static class SharedState {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Gold standard dataset.
|
||||||
|
*/
|
||||||
|
@Param({"goldstandard1.txt", "goldstandard2.txt"})
|
||||||
|
public String goldStandardFileName;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Candidate stemmer.
|
||||||
|
*/
|
||||||
|
@Param({
|
||||||
|
"GERMAN_RADIXOR",
|
||||||
|
"GERMAN_LUCENE_GERMAN_STEM_FILTER",
|
||||||
|
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER",
|
||||||
|
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER",
|
||||||
|
"GERMAN_CISTEM",
|
||||||
|
"SNOWBALL_GERMAN_DIRECT",
|
||||||
|
"SNOWBALL_GERMAN_LUCENE_FILTER"
|
||||||
|
})
|
||||||
|
public String candidateName;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parsed gold standard corpus.
|
||||||
|
*/
|
||||||
|
private GermanGoldstandardCorpus corpus;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Gold standard words flattened by cluster order.
|
||||||
|
*/
|
||||||
|
private String[] allTokens;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Candidate evaluator.
|
||||||
|
*/
|
||||||
|
private GoldstandardStemmer stemmer;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initializes one candidate on one gold standard corpus.
|
||||||
|
*
|
||||||
|
* @throws IOException when the corpus cannot be loaded
|
||||||
|
*/
|
||||||
|
@Setup(Level.Trial)
|
||||||
|
public void setUp() throws IOException {
|
||||||
|
this.corpus = loadCorpus(this.goldStandardFileName);
|
||||||
|
this.allTokens = flattenCorpusTokens(this.corpus);
|
||||||
|
this.stemmer = GermanCandidate.valueOf(this.candidateName).createEvaluator();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* JMH auxiliary counters for CISTEM-style cluster accounting.
|
||||||
|
*/
|
||||||
|
@State(Scope.Thread)
|
||||||
|
@AuxCounters(AuxCounters.Type.EVENTS)
|
||||||
|
public static class GoldstandardQualityCounters {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* True positives across clusters.
|
||||||
|
*/
|
||||||
|
public long truePositives;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* False positives across clusters.
|
||||||
|
*/
|
||||||
|
public long falsePositives;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* False negatives across clusters.
|
||||||
|
*/
|
||||||
|
public long falseNegatives;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluated clusters.
|
||||||
|
*/
|
||||||
|
public long evaluatedClusters;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluated tokens.
|
||||||
|
*/
|
||||||
|
public long evaluatedTokens;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resets counters before each measured iteration.
|
||||||
|
*/
|
||||||
|
@Setup(Level.Iteration)
|
||||||
|
public void reset() {
|
||||||
|
this.truePositives = 0L;
|
||||||
|
this.falsePositives = 0L;
|
||||||
|
this.falseNegatives = 0L;
|
||||||
|
this.evaluatedClusters = 0L;
|
||||||
|
this.evaluatedTokens = 0L;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluates CISTEM-style precision, recall, and F1-relevant counts.
|
||||||
|
*
|
||||||
|
* @param state shared benchmark state
|
||||||
|
* @param counters quality counters
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return evaluated token count for this operation
|
||||||
|
* @throws IOException if token filtering cannot run
|
||||||
|
*/
|
||||||
|
@Benchmark
|
||||||
|
@Warmup(iterations = 0)
|
||||||
|
@Measurement(iterations = 1, time = 1, timeUnit = java.util.concurrent.TimeUnit.MILLISECONDS)
|
||||||
|
@Fork(0)
|
||||||
|
public long cistemStyleQuality(final SharedState state, final GoldstandardQualityCounters counters,
|
||||||
|
final Blackhole blackhole) throws IOException {
|
||||||
|
final GoldstandardResult result = evaluateCistemStyle(state.corpus, state.allTokens, state.stemmer, blackhole);
|
||||||
|
counters.truePositives += result.truePositives();
|
||||||
|
counters.falsePositives += result.falsePositives();
|
||||||
|
counters.falseNegatives += result.falseNegatives();
|
||||||
|
counters.evaluatedClusters += result.evaluatedClusters();
|
||||||
|
counters.evaluatedTokens += result.evaluatedTokens();
|
||||||
|
return result.evaluatedTokens();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Benchmarks candidate throughput over the selected gold standard.
|
||||||
|
*
|
||||||
|
* @param state shared benchmark state
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @throws IOException if token filtering cannot run
|
||||||
|
*/
|
||||||
|
@Benchmark
|
||||||
|
public void cistemStyleSpeed(final SharedState state, final Blackhole blackhole) throws IOException {
|
||||||
|
state.stemmer.stem(state.allTokens, blackhole);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Named German candidates used for the CISTEM gold-standard comparison.
|
||||||
|
*/
|
||||||
|
private enum GermanCandidate {
|
||||||
|
GERMAN_RADIXOR,
|
||||||
|
GERMAN_LUCENE_GERMAN_STEM_FILTER,
|
||||||
|
GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER,
|
||||||
|
GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER,
|
||||||
|
GERMAN_CISTEM,
|
||||||
|
SNOWBALL_GERMAN_DIRECT,
|
||||||
|
SNOWBALL_GERMAN_LUCENE_FILTER;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a candidate evaluator.
|
||||||
|
*
|
||||||
|
* @return stemmer evaluator
|
||||||
|
* @throws IOException if trie resources cannot be loaded
|
||||||
|
*/
|
||||||
|
GoldstandardStemmer createEvaluator() throws IOException {
|
||||||
|
return switch (this) {
|
||||||
|
case GERMAN_RADIXOR -> direct(createGermanRadixorStemmer());
|
||||||
|
case GERMAN_LUCENE_GERMAN_STEM_FILTER ->
|
||||||
|
tokenFilter(input -> new GermanStemFilter(lowercase(input)));
|
||||||
|
case GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER ->
|
||||||
|
tokenFilter(input -> new GermanLightStemFilter(germanNormalize(input)));
|
||||||
|
case GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER ->
|
||||||
|
tokenFilter(input -> new GermanMinimalStemFilter(germanNormalize(input)));
|
||||||
|
case GERMAN_CISTEM -> direct(Cistem::stem);
|
||||||
|
case SNOWBALL_GERMAN_DIRECT -> direct(SnowballLanguageCase.GERMAN.createDirectStemmer()::stem);
|
||||||
|
case SNOWBALL_GERMAN_LUCENE_FILTER ->
|
||||||
|
tokenFilter(input -> new SnowballFilter(new LowerCaseFilter(input),
|
||||||
|
SnowballLanguageCase.GERMAN.luceneSnowballName()));
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluates one full corpus through CISTEM-style cluster scoring.
|
||||||
|
*
|
||||||
|
* <p>
|
||||||
|
* For each cluster, the most frequent predicted stem is considered the
|
||||||
|
* cluster main stem. TP are cluster words mapped to this stem, FN are
|
||||||
|
* words mapped elsewhere inside the same cluster, and FP are words from
|
||||||
|
* other clusters mapped to the same main stem.
|
||||||
|
* </p>
|
||||||
|
*
|
||||||
|
* @param corpus parsed gold standard corpus
|
||||||
|
* @param allTokens flattened token sequence
|
||||||
|
* @param stemmer candidate stemmer
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return aggregated TP/FP/FN counters and token metrics
|
||||||
|
* @throws IOException when token filtering cannot run
|
||||||
|
*/
|
||||||
|
private static GoldstandardResult evaluateCistemStyle(final GermanGoldstandardCorpus corpus,
|
||||||
|
final String[] allTokens, final GoldstandardStemmer stemmer, final Blackhole blackhole) throws IOException {
|
||||||
|
final String[] predicted = stemmer.stem(allTokens, blackhole);
|
||||||
|
final Map<String, Integer> globalPredictions = new LinkedHashMap<>();
|
||||||
|
for (int index = 0; index < allTokens.length; index++) {
|
||||||
|
final String prediction = normalizePrediction(predicted[index], allTokens[index]);
|
||||||
|
globalPredictions.put(prediction, globalPredictions.getOrDefault(prediction, 0) + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
long truePositives = 0L;
|
||||||
|
long falsePositives = 0L;
|
||||||
|
long falseNegatives = 0L;
|
||||||
|
int tokenOffset = 0;
|
||||||
|
for (final String[] cluster : corpus.clusters()) {
|
||||||
|
if (cluster.length == 0) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
final Map<String, Integer> localPredictions = new LinkedHashMap<>();
|
||||||
|
for (int index = 0; index < cluster.length; index++) {
|
||||||
|
final int tokenIndex = tokenOffset + index;
|
||||||
|
final String word = allTokens[tokenIndex];
|
||||||
|
final String prediction = normalizePrediction(predicted[tokenIndex], word);
|
||||||
|
localPredictions.put(prediction, localPredictions.getOrDefault(prediction, 0) + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
final String mainStem = mostFrequent(localPredictions);
|
||||||
|
final int predictedAsMain = localPredictions.get(mainStem);
|
||||||
|
final int clusterSize = cluster.length;
|
||||||
|
final int falseNegative = clusterSize - predictedAsMain;
|
||||||
|
final int falsePositive = globalPredictions.get(mainStem) - predictedAsMain;
|
||||||
|
|
||||||
|
truePositives += predictedAsMain;
|
||||||
|
falseNegatives += falseNegative;
|
||||||
|
falsePositives += falsePositive;
|
||||||
|
tokenOffset += clusterSize;
|
||||||
|
}
|
||||||
|
|
||||||
|
return new GoldstandardResult(truePositives, falsePositives, falseNegatives, corpus.clusters().length,
|
||||||
|
allTokens.length);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a direct evaluator.
|
||||||
|
*
|
||||||
|
* @param stemmer direct word stemmer
|
||||||
|
* @return evaluator
|
||||||
|
*/
|
||||||
|
private static GoldstandardStemmer direct(final Stemmer stemmer) {
|
||||||
|
Objects.requireNonNull(stemmer, "stemmer");
|
||||||
|
return (tokens, blackhole) -> {
|
||||||
|
final String[] outputs = new String[tokens.length];
|
||||||
|
for (int index = 0; index < tokens.length; index++) {
|
||||||
|
final String output = stemmer.stem(tokens[index]);
|
||||||
|
outputs[index] = output;
|
||||||
|
blackhole.consume(output);
|
||||||
|
}
|
||||||
|
return outputs;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a TokenFilter evaluator.
|
||||||
|
*
|
||||||
|
* @param factory filter stream factory
|
||||||
|
* @return evaluator
|
||||||
|
*/
|
||||||
|
private static GoldstandardStemmer tokenFilter(final Function<TokenStream, TokenStream> factory) {
|
||||||
|
Objects.requireNonNull(factory, "factory");
|
||||||
|
return (tokens, blackhole) -> firstTokenFilterOutputs(tokens, factory, blackhole);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Loads and parses one gold standard file from generated JMH resources.
|
||||||
|
*
|
||||||
|
* @param resourceName gold standard file name
|
||||||
|
* @return parsed corpus
|
||||||
|
* @throws IOException if reading fails
|
||||||
|
*/
|
||||||
|
private static GermanGoldstandardCorpus loadCorpus(final String resourceName) throws IOException {
|
||||||
|
final ClassLoader classLoader = GermanGoldstandardStemmerComparisonBenchmark.class.getClassLoader();
|
||||||
|
final InputStream resourceStream = classLoader.getResourceAsStream(resourceName);
|
||||||
|
if (resourceStream == null) {
|
||||||
|
throw new IllegalStateException("Missing generated CISTEM gold standard resource: " + resourceName
|
||||||
|
+ ". Run the Gradle JMH resource preparation task to download benchmark-only inputs.");
|
||||||
|
}
|
||||||
|
try (InputStream input = resourceStream) {
|
||||||
|
return parseCorpus(new BufferedReader(new InputStreamReader(input, StandardCharsets.UTF_8)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses CISTEM gold standard format into clustered candidates.
|
||||||
|
*
|
||||||
|
* @param reader UTF-8 reader
|
||||||
|
* @return parsed corpus
|
||||||
|
* @throws IOException if input cannot be read
|
||||||
|
*/
|
||||||
|
private static GermanGoldstandardCorpus parseCorpus(final BufferedReader reader) throws IOException {
|
||||||
|
final List<String[]> clusters = new ArrayList<>();
|
||||||
|
String line = reader.readLine();
|
||||||
|
while (line != null) {
|
||||||
|
final String trimmed = line.trim();
|
||||||
|
if (!trimmed.isEmpty()) {
|
||||||
|
final String[] words = trimmed.split("\\s+");
|
||||||
|
if (words.length > 0) {
|
||||||
|
clusters.add(words);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
line = reader.readLine();
|
||||||
|
}
|
||||||
|
return new GermanGoldstandardCorpus(clusters.toArray(String[][]::new));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Flattens the corpus in deterministic cluster order.
|
||||||
|
*
|
||||||
|
* @param corpus corpus to flatten
|
||||||
|
* @return flattened token array
|
||||||
|
*/
|
||||||
|
private static String[] flattenCorpusTokens(final GermanGoldstandardCorpus corpus) {
|
||||||
|
int total = 0;
|
||||||
|
for (final String[] cluster : corpus.clusters()) {
|
||||||
|
total += cluster.length;
|
||||||
|
}
|
||||||
|
final String[] tokens = new String[total];
|
||||||
|
int index = 0;
|
||||||
|
for (final String[] cluster : corpus.clusters()) {
|
||||||
|
System.arraycopy(cluster, 0, tokens, index, cluster.length);
|
||||||
|
index += cluster.length;
|
||||||
|
}
|
||||||
|
return tokens;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Returns the most frequent key; insertion order is preserved on ties.
|
||||||
|
*
|
||||||
|
* @param frequencies predicted stem frequencies
|
||||||
|
* @return most frequent stem
|
||||||
|
*/
|
||||||
|
private static String mostFrequent(final Map<String, Integer> frequencies) {
|
||||||
|
String best = null;
|
||||||
|
int bestCount = -1;
|
||||||
|
for (final Map.Entry<String, Integer> entry : frequencies.entrySet()) {
|
||||||
|
if (entry.getValue() > bestCount) {
|
||||||
|
best = entry.getKey();
|
||||||
|
bestCount = entry.getValue();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalizes a null/empty prediction using the input token as fallback.
|
||||||
|
*
|
||||||
|
* @param prediction stemmed token
|
||||||
|
* @param fallback fallback token
|
||||||
|
* @return safe prediction
|
||||||
|
*/
|
||||||
|
private static String normalizePrediction(final String prediction, final String fallback) {
|
||||||
|
if (prediction == null || prediction.isEmpty()) {
|
||||||
|
return fallback;
|
||||||
|
}
|
||||||
|
return prediction;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Applies one TokenFilter to all input tokens and returns the first emitted term
|
||||||
|
* for each input token.
|
||||||
|
*
|
||||||
|
* @param tokens input token corpus
|
||||||
|
* @param factory TokenFilter factory
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return first emitted term per input token
|
||||||
|
* @throws IOException if token streaming fails
|
||||||
|
*/
|
||||||
|
private static String[] firstTokenFilterOutputs(final String[] tokens, final Function<TokenStream, TokenStream> factory,
|
||||||
|
final Blackhole blackhole) throws IOException {
|
||||||
|
final String[] outputs = new String[tokens.length];
|
||||||
|
final BenchmarkTokenStream input = new BenchmarkTokenStream(tokens);
|
||||||
|
final TokenStream output = factory.apply(input);
|
||||||
|
final CharTermAttribute termAttribute = output.addAttribute(CharTermAttribute.class);
|
||||||
|
final PositionIncrementAttribute positionAttribute = output.addAttribute(PositionIncrementAttribute.class);
|
||||||
|
|
||||||
|
int inputIndex = -1;
|
||||||
|
boolean recordedForPosition = false;
|
||||||
|
output.reset();
|
||||||
|
while (output.incrementToken()) {
|
||||||
|
final int positionIncrement = positionAttribute.getPositionIncrement();
|
||||||
|
if (positionIncrement > 0) {
|
||||||
|
inputIndex += positionIncrement;
|
||||||
|
recordedForPosition = false;
|
||||||
|
}
|
||||||
|
if (inputIndex >= 0 && inputIndex < outputs.length && !recordedForPosition) {
|
||||||
|
outputs[inputIndex] = termAttribute.toString();
|
||||||
|
blackhole.consume(termAttribute);
|
||||||
|
recordedForPosition = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
output.end();
|
||||||
|
output.close();
|
||||||
|
|
||||||
|
for (int index = 0; index < outputs.length; index++) {
|
||||||
|
if (outputs[index] == null) {
|
||||||
|
outputs[index] = tokens[index];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return outputs;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a direct Radixor evaluator using the contracted dictionary trie.
|
||||||
|
*
|
||||||
|
* @return direct Radixor stemmer
|
||||||
|
* @throws IOException if the trie cannot be loaded
|
||||||
|
*/
|
||||||
|
private static Stemmer createGermanRadixorStemmer() throws IOException {
|
||||||
|
return new RadixorBenchmarkStemmer(StemmerPatchTrieLoader.loadCompiled(
|
||||||
|
StemmerPatchTrieLoader.Language.DE_DE, true,
|
||||||
|
ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS))::stem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Adds Lucene lower-case normalization.
|
||||||
|
*
|
||||||
|
* @param input token stream
|
||||||
|
* @return normalized token stream
|
||||||
|
*/
|
||||||
|
private static TokenStream lowercase(final TokenStream input) {
|
||||||
|
return new LowerCaseFilter(input);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Adds Lucene German normalization for light and minimal filters.
|
||||||
|
*
|
||||||
|
* @param input token stream
|
||||||
|
* @return normalized token stream
|
||||||
|
*/
|
||||||
|
private static TokenStream germanNormalize(final TokenStream input) {
|
||||||
|
return new GermanNormalizationFilter(lowercase(input));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Direct or filter stemmer adapter used by this benchmark.
|
||||||
|
*/
|
||||||
|
@FunctionalInterface
|
||||||
|
private interface GoldstandardStemmer {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs one complete token list.
|
||||||
|
*
|
||||||
|
* @param tokens input tokens
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return per-token outputs
|
||||||
|
* @throws IOException if filter processing fails
|
||||||
|
*/
|
||||||
|
String[] stem(String[] tokens, Blackhole blackhole) throws IOException;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Deterministic direct word stem function.
|
||||||
|
*/
|
||||||
|
@FunctionalInterface
|
||||||
|
private interface Stemmer {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stems one token.
|
||||||
|
*
|
||||||
|
* @param token input token
|
||||||
|
* @return stemmed token
|
||||||
|
*/
|
||||||
|
String stem(String token);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Immutable parsed CISTEM gold standard corpus.
|
||||||
|
*/
|
||||||
|
private static final class GermanGoldstandardCorpus {
|
||||||
|
|
||||||
|
private final String[][] clusters;
|
||||||
|
|
||||||
|
GermanGoldstandardCorpus(final String[][] clusters) {
|
||||||
|
this.clusters = clusters;
|
||||||
|
}
|
||||||
|
|
||||||
|
String[][] clusters() {
|
||||||
|
return this.clusters;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Aggregated quality result for one benchmark operation.
|
||||||
|
*
|
||||||
|
* @param truePositives true positives
|
||||||
|
* @param falsePositives false positives
|
||||||
|
* @param falseNegatives false negatives
|
||||||
|
* @param evaluatedClusters evaluated clusters
|
||||||
|
* @param evaluatedTokens evaluated tokens
|
||||||
|
*/
|
||||||
|
private record GoldstandardResult(long truePositives, long falsePositives, long falseNegatives,
|
||||||
|
long evaluatedClusters, long evaluatedTokens) {
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,387 @@
|
|||||||
|
/*******************************************************************************
|
||||||
|
* Copyright (C) 2026, Leo Galambos
|
||||||
|
* All rights reserved.
|
||||||
|
*
|
||||||
|
* Redistribution and use in source and binary forms, with or without
|
||||||
|
* modification, are permitted provided that the following conditions are met:
|
||||||
|
*
|
||||||
|
* 1. Redistributions of source code must retain the above copyright notice,
|
||||||
|
* this list of conditions and the following disclaimer.
|
||||||
|
*
|
||||||
|
* 2. Redistributions in binary form must reproduce the above copyright notice,
|
||||||
|
* this list of conditions and the following disclaimer in the documentation
|
||||||
|
* and/or other materials provided with the distribution.
|
||||||
|
*
|
||||||
|
* 3. Neither the name of the copyright holder nor the names of its contributors
|
||||||
|
* may be used to endorse or promote products derived from this software
|
||||||
|
* without specific prior written permission.
|
||||||
|
*
|
||||||
|
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||||
|
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||||
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||||
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||||
|
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||||
|
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||||
|
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||||
|
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||||
|
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||||
|
* POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
******************************************************************************/
|
||||||
|
package org.egothor.stemmer.benchmark;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.io.InputStream;
|
||||||
|
import java.text.ParseException;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Locale;
|
||||||
|
import java.util.Objects;
|
||||||
|
import java.util.concurrent.TimeUnit;
|
||||||
|
|
||||||
|
import org.apache.lucene.analysis.LowerCaseFilter;
|
||||||
|
import org.apache.lucene.analysis.TokenStream;
|
||||||
|
import org.apache.lucene.analysis.hunspell.Dictionary;
|
||||||
|
import org.apache.lucene.analysis.hunspell.HunspellStemFilter;
|
||||||
|
import org.apache.lucene.analysis.hunspell.SortingStrategy;
|
||||||
|
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
|
||||||
|
import org.apache.lucene.analysis.tokenattributes.PositionIncrementAttribute;
|
||||||
|
import org.egothor.stemmer.StemmerPatchTrieLoader;
|
||||||
|
import org.openjdk.jmh.annotations.AuxCounters;
|
||||||
|
import org.openjdk.jmh.annotations.Benchmark;
|
||||||
|
import org.openjdk.jmh.annotations.BenchmarkMode;
|
||||||
|
import org.openjdk.jmh.annotations.Fork;
|
||||||
|
import org.openjdk.jmh.annotations.Level;
|
||||||
|
import org.openjdk.jmh.annotations.Measurement;
|
||||||
|
import org.openjdk.jmh.annotations.Mode;
|
||||||
|
import org.openjdk.jmh.annotations.OutputTimeUnit;
|
||||||
|
import org.openjdk.jmh.annotations.Param;
|
||||||
|
import org.openjdk.jmh.annotations.Scope;
|
||||||
|
import org.openjdk.jmh.annotations.Setup;
|
||||||
|
import org.openjdk.jmh.annotations.State;
|
||||||
|
import org.openjdk.jmh.annotations.Warmup;
|
||||||
|
import org.openjdk.jmh.infra.Blackhole;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Emits exact-root agreement metrics for the benchmark-only Hunspell comparisons.
|
||||||
|
*
|
||||||
|
* <p>
|
||||||
|
* This class mirrors the existing Hunspell throughput setup but adds
|
||||||
|
* quality-style accuracy counters for every Hunspell language dictionary used
|
||||||
|
* in benchmark-only throughput comparisons.
|
||||||
|
* </p>
|
||||||
|
*/
|
||||||
|
@BenchmarkMode(Mode.AverageTime)
|
||||||
|
@OutputTimeUnit(TimeUnit.NANOSECONDS)
|
||||||
|
@Warmup(iterations = 0)
|
||||||
|
@Measurement(iterations = 1, time = 1, timeUnit = TimeUnit.MILLISECONDS)
|
||||||
|
@Fork(0)
|
||||||
|
public class HunspellStemmerComparisonBenchmarkQuality {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shared quality corpus and Hunspell dictionary for a selected language.
|
||||||
|
*/
|
||||||
|
@State(Scope.Benchmark)
|
||||||
|
public static class SharedState {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Selected language case.
|
||||||
|
*/
|
||||||
|
@Param({ "ENGLISH", "CZECH", "GERMAN", "SPANISH", "FRENCH", "DUTCH", "POLISH", "UKRAINIAN" })
|
||||||
|
public String languageCaseName;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Selected language descriptor.
|
||||||
|
*/
|
||||||
|
private HunspellLanguageCase languageCase;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Complete language dictionary corpus and expected roots.
|
||||||
|
*/
|
||||||
|
private LanguageBenchmarkCorpus.Corpus corpus;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parsed benchmark-only Hunspell dictionary.
|
||||||
|
*/
|
||||||
|
private Dictionary dictionary;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Initializes quality resources.
|
||||||
|
*
|
||||||
|
* @throws IOException if corpus or dictionary loading fails
|
||||||
|
* @throws ParseException if the Hunspell dictionary cannot be parsed
|
||||||
|
*/
|
||||||
|
@Setup(Level.Trial)
|
||||||
|
public void setUp() throws IOException, ParseException {
|
||||||
|
this.languageCase = HunspellLanguageCase.valueOf(this.languageCaseName);
|
||||||
|
this.corpus = LanguageBenchmarkCorpus.createFullCorpus(this.languageCase.radixorLanguage());
|
||||||
|
this.dictionary = loadDictionary(this.languageCase);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* JMH auxiliary counters for exact-root agreement.
|
||||||
|
*/
|
||||||
|
@State(Scope.Thread)
|
||||||
|
@AuxCounters(AuxCounters.Type.EVENTS)
|
||||||
|
public static class AccuracyCounters {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of exact-root matches.
|
||||||
|
*/
|
||||||
|
public long correctMatches;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of evaluated tokens.
|
||||||
|
*/
|
||||||
|
public long evaluatedTokens;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of exact-root matches where the input token differs from the
|
||||||
|
* expected root.
|
||||||
|
*/
|
||||||
|
public long changedCorrectMatches;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of evaluated tokens where the input token differs from the expected
|
||||||
|
* root.
|
||||||
|
*/
|
||||||
|
public long changedEvaluatedTokens;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of exact-root matches where the input token is already the expected
|
||||||
|
* root.
|
||||||
|
*/
|
||||||
|
public long rootPreservedMatches;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Number of evaluated tokens where the input token is already the expected
|
||||||
|
* root.
|
||||||
|
*/
|
||||||
|
public long rootEvaluatedTokens;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resets counters before each measured iteration.
|
||||||
|
*/
|
||||||
|
@Setup(Level.Iteration)
|
||||||
|
public void reset() {
|
||||||
|
this.correctMatches = 0L;
|
||||||
|
this.evaluatedTokens = 0L;
|
||||||
|
this.changedCorrectMatches = 0L;
|
||||||
|
this.changedEvaluatedTokens = 0L;
|
||||||
|
this.rootPreservedMatches = 0L;
|
||||||
|
this.rootEvaluatedTokens = 0L;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Evaluates exact-root agreement for the selected Hunspell dictionary.
|
||||||
|
*
|
||||||
|
* @param sharedState shared quality state
|
||||||
|
* @param counters JMH auxiliary counters
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return exact-root match count
|
||||||
|
* @throws IOException if Lucene token streaming fails
|
||||||
|
*/
|
||||||
|
@Benchmark
|
||||||
|
public int luceneHunspellStemFilterAccuracy(final SharedState sharedState, final AccuracyCounters counters,
|
||||||
|
final Blackhole blackhole) throws IOException {
|
||||||
|
final String[] actualStems = firstHunspellOutputs(sharedState.corpus.tokens(), sharedState.dictionary,
|
||||||
|
blackhole);
|
||||||
|
final String[] tokens = sharedState.corpus.tokens();
|
||||||
|
final String[] expectedRoots = sharedState.corpus.expectedRoots();
|
||||||
|
|
||||||
|
int correct = 0;
|
||||||
|
int changedCorrect = 0;
|
||||||
|
int changedEvaluated = 0;
|
||||||
|
int rootPreserved = 0;
|
||||||
|
int rootEvaluated = 0;
|
||||||
|
for (int index = 0; index < actualStems.length; index++) {
|
||||||
|
final String token = tokens[index];
|
||||||
|
final String expectedRoot = expectedRoots[index];
|
||||||
|
final boolean exact = Objects.equals(expectedRoot, actualStems[index]);
|
||||||
|
if (exact) {
|
||||||
|
correct++;
|
||||||
|
}
|
||||||
|
if (Objects.equals(token, expectedRoot)) {
|
||||||
|
rootEvaluated++;
|
||||||
|
if (exact) {
|
||||||
|
rootPreserved++;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
changedEvaluated++;
|
||||||
|
if (exact) {
|
||||||
|
changedCorrect++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
counters.correctMatches += correct;
|
||||||
|
counters.evaluatedTokens += actualStems.length;
|
||||||
|
counters.changedCorrectMatches += changedCorrect;
|
||||||
|
counters.changedEvaluatedTokens += changedEvaluated;
|
||||||
|
counters.rootPreservedMatches += rootPreserved;
|
||||||
|
counters.rootEvaluatedTokens += rootEvaluated;
|
||||||
|
return correct;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Extracts the first emitted Hunspell stem for each input token.
|
||||||
|
*
|
||||||
|
* @param tokens token corpus
|
||||||
|
* @param dictionary Hunspell dictionary
|
||||||
|
* @param blackhole result sink
|
||||||
|
* @return first emitted term per input token
|
||||||
|
* @throws IOException if Lucene streaming fails
|
||||||
|
*/
|
||||||
|
private static String[] firstHunspellOutputs(final String[] tokens, final Dictionary dictionary,
|
||||||
|
final Blackhole blackhole) throws IOException {
|
||||||
|
final String[] outputs = new String[tokens.length];
|
||||||
|
final BenchmarkTokenStream input = new BenchmarkTokenStream(tokens);
|
||||||
|
final TokenStream output = new HunspellStemFilter(new LowerCaseFilter(input), dictionary, true);
|
||||||
|
final CharTermAttribute termAttribute = output.addAttribute(CharTermAttribute.class);
|
||||||
|
final PositionIncrementAttribute positionAttribute = output.addAttribute(PositionIncrementAttribute.class);
|
||||||
|
int inputIndex = -1;
|
||||||
|
boolean recordedForPosition = false;
|
||||||
|
|
||||||
|
output.reset();
|
||||||
|
while (output.incrementToken()) {
|
||||||
|
final int positionIncrement = positionAttribute.getPositionIncrement();
|
||||||
|
if (positionIncrement > 0) {
|
||||||
|
inputIndex += positionIncrement;
|
||||||
|
recordedForPosition = false;
|
||||||
|
}
|
||||||
|
if (inputIndex >= 0 && inputIndex < outputs.length && !recordedForPosition) {
|
||||||
|
outputs[inputIndex] = termAttribute.toString();
|
||||||
|
recordedForPosition = true;
|
||||||
|
}
|
||||||
|
blackhole.consume(termAttribute);
|
||||||
|
}
|
||||||
|
output.end();
|
||||||
|
output.close();
|
||||||
|
|
||||||
|
for (int index = 0; index < outputs.length; index++) {
|
||||||
|
if (outputs[index] == null) {
|
||||||
|
outputs[index] = tokens[index];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return outputs;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Loads a benchmark-only Hunspell dictionary from generated resources.
|
||||||
|
*
|
||||||
|
* @param languageCase selected language case
|
||||||
|
* @return parsed dictionary
|
||||||
|
* @throws IOException if dictionary resources cannot be read
|
||||||
|
* @throws ParseException if dictionary parsing fails
|
||||||
|
*/
|
||||||
|
private static Dictionary loadDictionary(final HunspellLanguageCase languageCase) throws IOException,
|
||||||
|
ParseException {
|
||||||
|
final ClassLoader classLoader = HunspellStemmerComparisonBenchmarkQuality.class.getClassLoader();
|
||||||
|
final String basePath = "hunspell/" + languageCase.hunspellResourceCode() + "/index.";
|
||||||
|
try (InputStream affixStream = openRequiredResource(classLoader, basePath + "aff");
|
||||||
|
InputStream dictionaryStream = openRequiredResource(classLoader, basePath + "dic")) {
|
||||||
|
return new Dictionary(affixStream, List.of(dictionaryStream), true, SortingStrategy.inMemory());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Opens a required classpath resource.
|
||||||
|
*
|
||||||
|
* @param classLoader class loader
|
||||||
|
* @param path resource path
|
||||||
|
* @return resource stream
|
||||||
|
*/
|
||||||
|
private static InputStream openRequiredResource(final ClassLoader classLoader, final String path) {
|
||||||
|
final InputStream stream = classLoader.getResourceAsStream(path);
|
||||||
|
if (stream == null) {
|
||||||
|
throw new IllegalStateException("Missing benchmark-only Hunspell resource: " + path);
|
||||||
|
}
|
||||||
|
return stream;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Benchmark language mapping.
|
||||||
|
*/
|
||||||
|
private enum HunspellLanguageCase {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* English Hunspell dictionary over the Radixor English corpus.
|
||||||
|
*/
|
||||||
|
ENGLISH("en", StemmerPatchTrieLoader.Language.US_UK),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Czech Hunspell dictionary over the Radixor Czech corpus.
|
||||||
|
*/
|
||||||
|
CZECH("cs", StemmerPatchTrieLoader.Language.CS_CZ),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* German Hunspell dictionary over the Radixor German corpus.
|
||||||
|
*/
|
||||||
|
GERMAN("de", StemmerPatchTrieLoader.Language.DE_DE),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Spanish Hunspell dictionary over the Radixor Spanish corpus.
|
||||||
|
*/
|
||||||
|
SPANISH("es", StemmerPatchTrieLoader.Language.ES_ES),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* French Hunspell dictionary over the Radixor French corpus.
|
||||||
|
*/
|
||||||
|
FRENCH("fr", StemmerPatchTrieLoader.Language.FR_FR),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Dutch Hunspell dictionary over the Radixor Dutch corpus.
|
||||||
|
*/
|
||||||
|
DUTCH("nl", StemmerPatchTrieLoader.Language.NL_NL),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Polish Hunspell dictionary over the Radixor Polish corpus.
|
||||||
|
*/
|
||||||
|
POLISH("pl", StemmerPatchTrieLoader.Language.PL_PL),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ukrainian Hunspell dictionary over the Radixor Ukrainian corpus.
|
||||||
|
*/
|
||||||
|
UKRAINIAN("uk", StemmerPatchTrieLoader.Language.UK_UA);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Wooorm/dictionaries resource code.
|
||||||
|
*/
|
||||||
|
private final String hunspellResourceCode;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Matching Radixor language.
|
||||||
|
*/
|
||||||
|
private final StemmerPatchTrieLoader.Language radixorLanguage;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a language mapping.
|
||||||
|
*
|
||||||
|
* @param hunspellResourceCode Hunspell resource code
|
||||||
|
* @param radixorLanguage Radixor language
|
||||||
|
*/
|
||||||
|
HunspellLanguageCase(final String hunspellResourceCode, final StemmerPatchTrieLoader.Language radixorLanguage) {
|
||||||
|
this.hunspellResourceCode = hunspellResourceCode.toLowerCase(Locale.ROOT);
|
||||||
|
this.radixorLanguage = radixorLanguage;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Returns the Hunspell resource code.
|
||||||
|
*
|
||||||
|
* @return resource code
|
||||||
|
*/
|
||||||
|
String hunspellResourceCode() {
|
||||||
|
return this.hunspellResourceCode;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Returns the matching Radixor language.
|
||||||
|
*
|
||||||
|
* @return Radixor language
|
||||||
|
*/
|
||||||
|
StemmerPatchTrieLoader.Language radixorLanguage() {
|
||||||
|
return this.radixorLanguage;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -473,6 +473,19 @@ public class MultiLanguageStemmerComparisonBenchmark {
|
|||||||
filterState.germanMinimalStem.run(sharedState.german.tokens, blackhole);
|
filterState.germanMinimalStem.run(sharedState.german.tokens, blackhole);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs CISTEM directly over the German corpus.
|
||||||
|
*
|
||||||
|
* @param sharedState shared benchmark state
|
||||||
|
* @param blackhole result sink
|
||||||
|
*/
|
||||||
|
@Benchmark
|
||||||
|
public void germanCistem(final SharedState sharedState, final Blackhole blackhole) {
|
||||||
|
for (final String token : sharedState.german.tokens) {
|
||||||
|
blackhole.consume(Cistem.stem(token));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Runs Radixor over the Spanish corpus.
|
* Runs Radixor over the Spanish corpus.
|
||||||
*
|
*
|
||||||
|
|||||||
@@ -144,6 +144,7 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
"GERMAN_LUCENE_GERMAN_STEM_FILTER",
|
"GERMAN_LUCENE_GERMAN_STEM_FILTER",
|
||||||
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER",
|
"GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER",
|
||||||
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER",
|
"GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER",
|
||||||
|
"GERMAN_CISTEM",
|
||||||
"SPANISH_RADIXOR",
|
"SPANISH_RADIXOR",
|
||||||
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER",
|
"SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER",
|
||||||
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER",
|
"SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER",
|
||||||
@@ -178,46 +179,32 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
"UKRAINIAN_RADIXOR",
|
"UKRAINIAN_RADIXOR",
|
||||||
"UKRAINIAN_MORFOLOGIK_DIRECT",
|
"UKRAINIAN_MORFOLOGIK_DIRECT",
|
||||||
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER",
|
"UKRAINIAN_LUCENE_MORFOLOGIK_FILTER",
|
||||||
"SNOWBALL_DANISH_RADIXOR",
|
|
||||||
"SNOWBALL_DANISH_DIRECT",
|
"SNOWBALL_DANISH_DIRECT",
|
||||||
"SNOWBALL_DANISH_LUCENE_FILTER",
|
"SNOWBALL_DANISH_LUCENE_FILTER",
|
||||||
"SNOWBALL_DUTCH_RADIXOR",
|
|
||||||
"SNOWBALL_DUTCH_DIRECT",
|
"SNOWBALL_DUTCH_DIRECT",
|
||||||
"SNOWBALL_DUTCH_LUCENE_FILTER",
|
"SNOWBALL_DUTCH_LUCENE_FILTER",
|
||||||
"SNOWBALL_FINNISH_RADIXOR",
|
|
||||||
"SNOWBALL_FINNISH_DIRECT",
|
"SNOWBALL_FINNISH_DIRECT",
|
||||||
"SNOWBALL_FINNISH_LUCENE_FILTER",
|
"SNOWBALL_FINNISH_LUCENE_FILTER",
|
||||||
"SNOWBALL_FRENCH_RADIXOR",
|
|
||||||
"SNOWBALL_FRENCH_DIRECT",
|
"SNOWBALL_FRENCH_DIRECT",
|
||||||
"SNOWBALL_FRENCH_LUCENE_FILTER",
|
"SNOWBALL_FRENCH_LUCENE_FILTER",
|
||||||
"SNOWBALL_GERMAN_RADIXOR",
|
|
||||||
"SNOWBALL_GERMAN_DIRECT",
|
"SNOWBALL_GERMAN_DIRECT",
|
||||||
"SNOWBALL_GERMAN_LUCENE_FILTER",
|
"SNOWBALL_GERMAN_LUCENE_FILTER",
|
||||||
"SNOWBALL_HUNGARIAN_RADIXOR",
|
|
||||||
"SNOWBALL_HUNGARIAN_DIRECT",
|
"SNOWBALL_HUNGARIAN_DIRECT",
|
||||||
"SNOWBALL_HUNGARIAN_LUCENE_FILTER",
|
"SNOWBALL_HUNGARIAN_LUCENE_FILTER",
|
||||||
"SNOWBALL_ITALIAN_RADIXOR",
|
|
||||||
"SNOWBALL_ITALIAN_DIRECT",
|
"SNOWBALL_ITALIAN_DIRECT",
|
||||||
"SNOWBALL_ITALIAN_LUCENE_FILTER",
|
"SNOWBALL_ITALIAN_LUCENE_FILTER",
|
||||||
"SNOWBALL_NORWEGIAN_BOKMAL_RADIXOR",
|
|
||||||
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT",
|
"SNOWBALL_NORWEGIAN_BOKMAL_DIRECT",
|
||||||
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER",
|
"SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER",
|
||||||
"SNOWBALL_NORWEGIAN_NYNORSK_RADIXOR",
|
|
||||||
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT",
|
"SNOWBALL_NORWEGIAN_NYNORSK_DIRECT",
|
||||||
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER",
|
"SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER",
|
||||||
"SNOWBALL_PORTUGUESE_RADIXOR",
|
|
||||||
"SNOWBALL_PORTUGUESE_DIRECT",
|
"SNOWBALL_PORTUGUESE_DIRECT",
|
||||||
"SNOWBALL_PORTUGUESE_LUCENE_FILTER",
|
"SNOWBALL_PORTUGUESE_LUCENE_FILTER",
|
||||||
"SNOWBALL_RUSSIAN_RADIXOR",
|
|
||||||
"SNOWBALL_RUSSIAN_DIRECT",
|
"SNOWBALL_RUSSIAN_DIRECT",
|
||||||
"SNOWBALL_RUSSIAN_LUCENE_FILTER",
|
"SNOWBALL_RUSSIAN_LUCENE_FILTER",
|
||||||
"SNOWBALL_SPANISH_RADIXOR",
|
|
||||||
"SNOWBALL_SPANISH_DIRECT",
|
"SNOWBALL_SPANISH_DIRECT",
|
||||||
"SNOWBALL_SPANISH_LUCENE_FILTER",
|
"SNOWBALL_SPANISH_LUCENE_FILTER",
|
||||||
"SNOWBALL_SWEDISH_RADIXOR",
|
|
||||||
"SNOWBALL_SWEDISH_DIRECT",
|
"SNOWBALL_SWEDISH_DIRECT",
|
||||||
"SNOWBALL_SWEDISH_LUCENE_FILTER",
|
"SNOWBALL_SWEDISH_LUCENE_FILTER",
|
||||||
"SNOWBALL_YIDDISH_RADIXOR",
|
|
||||||
"SNOWBALL_YIDDISH_DIRECT",
|
"SNOWBALL_YIDDISH_DIRECT",
|
||||||
"SNOWBALL_YIDDISH_LUCENE_FILTER"
|
"SNOWBALL_YIDDISH_LUCENE_FILTER"
|
||||||
})
|
})
|
||||||
@@ -343,6 +330,7 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
GERMAN_LUCENE_GERMAN_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
GERMAN_LUCENE_GERMAN_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
||||||
GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
GERMAN_LUCENE_GERMAN_LIGHT_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
||||||
GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER(StemmerPatchTrieLoader.Language.DE_DE),
|
||||||
|
GERMAN_CISTEM(StemmerPatchTrieLoader.Language.DE_DE),
|
||||||
SPANISH_RADIXOR(StemmerPatchTrieLoader.Language.ES_ES),
|
SPANISH_RADIXOR(StemmerPatchTrieLoader.Language.ES_ES),
|
||||||
SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER(StemmerPatchTrieLoader.Language.ES_ES),
|
SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER(StemmerPatchTrieLoader.Language.ES_ES),
|
||||||
SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER(StemmerPatchTrieLoader.Language.ES_ES),
|
SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER(StemmerPatchTrieLoader.Language.ES_ES),
|
||||||
@@ -377,52 +365,36 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
UKRAINIAN_RADIXOR(StemmerPatchTrieLoader.Language.UK_UA),
|
UKRAINIAN_RADIXOR(StemmerPatchTrieLoader.Language.UK_UA),
|
||||||
UKRAINIAN_MORFOLOGIK_DIRECT(StemmerPatchTrieLoader.Language.UK_UA),
|
UKRAINIAN_MORFOLOGIK_DIRECT(StemmerPatchTrieLoader.Language.UK_UA),
|
||||||
UKRAINIAN_LUCENE_MORFOLOGIK_FILTER(StemmerPatchTrieLoader.Language.UK_UA),
|
UKRAINIAN_LUCENE_MORFOLOGIK_FILTER(StemmerPatchTrieLoader.Language.UK_UA),
|
||||||
SNOWBALL_DANISH_RADIXOR(StemmerPatchTrieLoader.Language.DA_DK, SnowballLanguageCase.DANISH),
|
|
||||||
SNOWBALL_DANISH_DIRECT(StemmerPatchTrieLoader.Language.DA_DK, SnowballLanguageCase.DANISH),
|
SNOWBALL_DANISH_DIRECT(StemmerPatchTrieLoader.Language.DA_DK, SnowballLanguageCase.DANISH),
|
||||||
SNOWBALL_DANISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.DA_DK, SnowballLanguageCase.DANISH),
|
SNOWBALL_DANISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.DA_DK, SnowballLanguageCase.DANISH),
|
||||||
SNOWBALL_DUTCH_RADIXOR(StemmerPatchTrieLoader.Language.NL_NL, SnowballLanguageCase.DUTCH),
|
|
||||||
SNOWBALL_DUTCH_DIRECT(StemmerPatchTrieLoader.Language.NL_NL, SnowballLanguageCase.DUTCH),
|
SNOWBALL_DUTCH_DIRECT(StemmerPatchTrieLoader.Language.NL_NL, SnowballLanguageCase.DUTCH),
|
||||||
SNOWBALL_DUTCH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NL_NL, SnowballLanguageCase.DUTCH),
|
SNOWBALL_DUTCH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NL_NL, SnowballLanguageCase.DUTCH),
|
||||||
SNOWBALL_FINNISH_RADIXOR(StemmerPatchTrieLoader.Language.FI_FI, SnowballLanguageCase.FINNISH),
|
|
||||||
SNOWBALL_FINNISH_DIRECT(StemmerPatchTrieLoader.Language.FI_FI, SnowballLanguageCase.FINNISH),
|
SNOWBALL_FINNISH_DIRECT(StemmerPatchTrieLoader.Language.FI_FI, SnowballLanguageCase.FINNISH),
|
||||||
SNOWBALL_FINNISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.FI_FI, SnowballLanguageCase.FINNISH),
|
SNOWBALL_FINNISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.FI_FI, SnowballLanguageCase.FINNISH),
|
||||||
SNOWBALL_FRENCH_RADIXOR(StemmerPatchTrieLoader.Language.FR_FR, SnowballLanguageCase.FRENCH),
|
|
||||||
SNOWBALL_FRENCH_DIRECT(StemmerPatchTrieLoader.Language.FR_FR, SnowballLanguageCase.FRENCH),
|
SNOWBALL_FRENCH_DIRECT(StemmerPatchTrieLoader.Language.FR_FR, SnowballLanguageCase.FRENCH),
|
||||||
SNOWBALL_FRENCH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.FR_FR, SnowballLanguageCase.FRENCH),
|
SNOWBALL_FRENCH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.FR_FR, SnowballLanguageCase.FRENCH),
|
||||||
SNOWBALL_GERMAN_RADIXOR(StemmerPatchTrieLoader.Language.DE_DE, SnowballLanguageCase.GERMAN),
|
|
||||||
SNOWBALL_GERMAN_DIRECT(StemmerPatchTrieLoader.Language.DE_DE, SnowballLanguageCase.GERMAN),
|
SNOWBALL_GERMAN_DIRECT(StemmerPatchTrieLoader.Language.DE_DE, SnowballLanguageCase.GERMAN),
|
||||||
SNOWBALL_GERMAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.DE_DE, SnowballLanguageCase.GERMAN),
|
SNOWBALL_GERMAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.DE_DE, SnowballLanguageCase.GERMAN),
|
||||||
SNOWBALL_HUNGARIAN_RADIXOR(StemmerPatchTrieLoader.Language.HU_HU, SnowballLanguageCase.HUNGARIAN),
|
|
||||||
SNOWBALL_HUNGARIAN_DIRECT(StemmerPatchTrieLoader.Language.HU_HU, SnowballLanguageCase.HUNGARIAN),
|
SNOWBALL_HUNGARIAN_DIRECT(StemmerPatchTrieLoader.Language.HU_HU, SnowballLanguageCase.HUNGARIAN),
|
||||||
SNOWBALL_HUNGARIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.HU_HU, SnowballLanguageCase.HUNGARIAN),
|
SNOWBALL_HUNGARIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.HU_HU, SnowballLanguageCase.HUNGARIAN),
|
||||||
SNOWBALL_ITALIAN_RADIXOR(StemmerPatchTrieLoader.Language.IT_IT, SnowballLanguageCase.ITALIAN),
|
|
||||||
SNOWBALL_ITALIAN_DIRECT(StemmerPatchTrieLoader.Language.IT_IT, SnowballLanguageCase.ITALIAN),
|
SNOWBALL_ITALIAN_DIRECT(StemmerPatchTrieLoader.Language.IT_IT, SnowballLanguageCase.ITALIAN),
|
||||||
SNOWBALL_ITALIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.IT_IT, SnowballLanguageCase.ITALIAN),
|
SNOWBALL_ITALIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.IT_IT, SnowballLanguageCase.ITALIAN),
|
||||||
SNOWBALL_NORWEGIAN_BOKMAL_RADIXOR(StemmerPatchTrieLoader.Language.NB_NO,
|
|
||||||
SnowballLanguageCase.NORWEGIAN_BOKMAL),
|
|
||||||
SNOWBALL_NORWEGIAN_BOKMAL_DIRECT(StemmerPatchTrieLoader.Language.NB_NO,
|
SNOWBALL_NORWEGIAN_BOKMAL_DIRECT(StemmerPatchTrieLoader.Language.NB_NO,
|
||||||
SnowballLanguageCase.NORWEGIAN_BOKMAL),
|
SnowballLanguageCase.NORWEGIAN_BOKMAL),
|
||||||
SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NB_NO,
|
SNOWBALL_NORWEGIAN_BOKMAL_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NB_NO,
|
||||||
SnowballLanguageCase.NORWEGIAN_BOKMAL),
|
SnowballLanguageCase.NORWEGIAN_BOKMAL),
|
||||||
SNOWBALL_NORWEGIAN_NYNORSK_RADIXOR(StemmerPatchTrieLoader.Language.NN_NO,
|
|
||||||
SnowballLanguageCase.NORWEGIAN_NYNORSK),
|
|
||||||
SNOWBALL_NORWEGIAN_NYNORSK_DIRECT(StemmerPatchTrieLoader.Language.NN_NO,
|
SNOWBALL_NORWEGIAN_NYNORSK_DIRECT(StemmerPatchTrieLoader.Language.NN_NO,
|
||||||
SnowballLanguageCase.NORWEGIAN_NYNORSK),
|
SnowballLanguageCase.NORWEGIAN_NYNORSK),
|
||||||
SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NN_NO,
|
SNOWBALL_NORWEGIAN_NYNORSK_LUCENE_FILTER(StemmerPatchTrieLoader.Language.NN_NO,
|
||||||
SnowballLanguageCase.NORWEGIAN_NYNORSK),
|
SnowballLanguageCase.NORWEGIAN_NYNORSK),
|
||||||
SNOWBALL_PORTUGUESE_RADIXOR(StemmerPatchTrieLoader.Language.PT_PT, SnowballLanguageCase.PORTUGUESE),
|
|
||||||
SNOWBALL_PORTUGUESE_DIRECT(StemmerPatchTrieLoader.Language.PT_PT, SnowballLanguageCase.PORTUGUESE),
|
SNOWBALL_PORTUGUESE_DIRECT(StemmerPatchTrieLoader.Language.PT_PT, SnowballLanguageCase.PORTUGUESE),
|
||||||
SNOWBALL_PORTUGUESE_LUCENE_FILTER(StemmerPatchTrieLoader.Language.PT_PT, SnowballLanguageCase.PORTUGUESE),
|
SNOWBALL_PORTUGUESE_LUCENE_FILTER(StemmerPatchTrieLoader.Language.PT_PT, SnowballLanguageCase.PORTUGUESE),
|
||||||
SNOWBALL_RUSSIAN_RADIXOR(StemmerPatchTrieLoader.Language.RU_RU, SnowballLanguageCase.RUSSIAN),
|
|
||||||
SNOWBALL_RUSSIAN_DIRECT(StemmerPatchTrieLoader.Language.RU_RU, SnowballLanguageCase.RUSSIAN),
|
SNOWBALL_RUSSIAN_DIRECT(StemmerPatchTrieLoader.Language.RU_RU, SnowballLanguageCase.RUSSIAN),
|
||||||
SNOWBALL_RUSSIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.RU_RU, SnowballLanguageCase.RUSSIAN),
|
SNOWBALL_RUSSIAN_LUCENE_FILTER(StemmerPatchTrieLoader.Language.RU_RU, SnowballLanguageCase.RUSSIAN),
|
||||||
SNOWBALL_SPANISH_RADIXOR(StemmerPatchTrieLoader.Language.ES_ES, SnowballLanguageCase.SPANISH),
|
|
||||||
SNOWBALL_SPANISH_DIRECT(StemmerPatchTrieLoader.Language.ES_ES, SnowballLanguageCase.SPANISH),
|
SNOWBALL_SPANISH_DIRECT(StemmerPatchTrieLoader.Language.ES_ES, SnowballLanguageCase.SPANISH),
|
||||||
SNOWBALL_SPANISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.ES_ES, SnowballLanguageCase.SPANISH),
|
SNOWBALL_SPANISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.ES_ES, SnowballLanguageCase.SPANISH),
|
||||||
SNOWBALL_SWEDISH_RADIXOR(StemmerPatchTrieLoader.Language.SV_SE, SnowballLanguageCase.SWEDISH),
|
|
||||||
SNOWBALL_SWEDISH_DIRECT(StemmerPatchTrieLoader.Language.SV_SE, SnowballLanguageCase.SWEDISH),
|
SNOWBALL_SWEDISH_DIRECT(StemmerPatchTrieLoader.Language.SV_SE, SnowballLanguageCase.SWEDISH),
|
||||||
SNOWBALL_SWEDISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.SV_SE, SnowballLanguageCase.SWEDISH),
|
SNOWBALL_SWEDISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.SV_SE, SnowballLanguageCase.SWEDISH),
|
||||||
SNOWBALL_YIDDISH_RADIXOR(StemmerPatchTrieLoader.Language.YI, SnowballLanguageCase.YIDDISH),
|
|
||||||
SNOWBALL_YIDDISH_DIRECT(StemmerPatchTrieLoader.Language.YI, SnowballLanguageCase.YIDDISH),
|
SNOWBALL_YIDDISH_DIRECT(StemmerPatchTrieLoader.Language.YI, SnowballLanguageCase.YIDDISH),
|
||||||
SNOWBALL_YIDDISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.YI, SnowballLanguageCase.YIDDISH);
|
SNOWBALL_YIDDISH_LUCENE_FILTER(StemmerPatchTrieLoader.Language.YI, SnowballLanguageCase.YIDDISH);
|
||||||
|
|
||||||
@@ -504,6 +476,7 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
tokenFilter(input -> new GermanLightStemFilter(germanNormalize(input)));
|
tokenFilter(input -> new GermanLightStemFilter(germanNormalize(input)));
|
||||||
case GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER ->
|
case GERMAN_LUCENE_GERMAN_MINIMAL_STEM_FILTER ->
|
||||||
tokenFilter(input -> new GermanMinimalStemFilter(germanNormalize(input)));
|
tokenFilter(input -> new GermanMinimalStemFilter(germanNormalize(input)));
|
||||||
|
case GERMAN_CISTEM -> direct(createGermanCistemStemmer());
|
||||||
case SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER ->
|
case SPANISH_LUCENE_SPANISH_LIGHT_STEM_FILTER ->
|
||||||
tokenFilter(input -> new SpanishLightStemFilter(lowercase(input)));
|
tokenFilter(input -> new SpanishLightStemFilter(lowercase(input)));
|
||||||
case SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER ->
|
case SPANISH_LUCENE_SPANISH_MINIMAL_STEM_FILTER ->
|
||||||
@@ -561,6 +534,15 @@ public class StemmerComparisonBenchmarkQuality {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Creates a CISTEM stemmer adapter.
|
||||||
|
*
|
||||||
|
* @return German stem function
|
||||||
|
*/
|
||||||
|
private static Stemmer createGermanCistemStemmer() {
|
||||||
|
return Cistem::stem;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Direct stemmer function.
|
* Direct stemmer function.
|
||||||
*/
|
*/
|
||||||
|
|||||||
Binary file not shown.
Reference in New Issue
Block a user