Egothor/Radixor
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add CISTEM and Hunspell benchmarks and refresh results

Browse Source
This commit is contained in:
Leo Galambos
2026-07-06 01:51:33 +02:00
parent a52e82933f
commit 049f44e697
33 changed files with 1410 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/benchmarks/reference/candidates.md
View File

@@ -4,20 +4,28 @@ Implemented benchmark methods are documented on the per-language pages under [La
## Included Candidate Families
The current benchmark pages include Radixor baselines, Lucene language filters where the language matches a bundled Radixor resource, Lucene Stempel and Morfologik paths where applicable, official Snowball Java stemmers where same-language comparison is available, and selected English-specific non-Lucene baselines such as OpenNLP Porter and Paice/Husk Lancaster.
The current benchmark pages include Radixor baselines, Lucene language filters where the language matches a bundled Radixor resource, Lucene Stempel and Morfologik paths where applicable, official Snowball Java stemmers where same-language comparison is available, benchmark-only CISTEM German stemmer evaluation, benchmark-only Hunspell comparisons, and selected English-specific non-Lucene baselines such as OpenNLP Porter and Paice/Husk Lancaster.
Benchmark-only Hunspell comparisons use bundled benchmark dictionaries and the Lucene HunspellStemFilter adapter over the selected language token streams.
The CISTEM candidate is implemented in `src/jmh/java/org/egothor/stemmer/benchmark/Cistem.java` and follows the original MIT-licensed upstream implementation from Leonie Weissweiler's CISTEM project.
CISTEM German gold-standard files are not vendored in this repository. The Gradle JMH resource preparation tasks download `goldstandard1.txt` and `goldstandard2.txt` from the upstream CISTEM repository into generated build resources.
Direct stemmer APIs and Lucene TokenFilter paths are documented separately on language pages. TokenFilter rows include TokenStream, attribute, and required normalization overhead. Direct rows measure exposed direct APIs.
For the benchmark refresh used in this documentation build:
- Hunspell families are included in `HunspellStemmerComparisonBenchmark` (speed) and `HunspellStemmerComparisonBenchmarkQuality` (quality for all benchmark languages in this corpus). The legacy
`EnglishHunspellStemmerComparisonBenchmarkQuality` result is retained for continuity.
- CISTEM quality is present in the published per-language results under `GERMAN_CISTEM`. CISTEM speed is present as `germanCistem` in `MultiLanguageStemmerComparisonBenchmark`.
## Evaluated But Skipped Candidates
| Candidate | Language | Link/source | Reason skipped |
| --- | --- | --- | --- |
| Lucene Arabic, Bulgarian, Bengali, Sorani, Greek, Galician, Hindi, Indonesian, Latvian, Telugu filters | Various | `lucene-analysis-common` | No bundled same-language Radixor resource in this repository snapshot. |
| Lucene analyzer-only paths | Multiple | Lucene analyzers | Full analyzers mix tokenization, stop-word handling, and other behavior; direct filters are used where available. |
| Lucene HunspellStemFilter | Multiple | `lucene-analysis-common` | Requires external Hunspell dictionaries not resolved as benchmark-only resources here. |
| Lucene StemmerOverrideFilter | Multiple | `lucene-analysis-common` | Override map facility, not a stemmer algorithm. |
| Additional Snowball Lovins | English | Official Snowball Java distribution | No Lovins Java stemmer was present in the selected Snowball Java distribution. |
| Lemur Project Krovetz Stemmer | English | Lemur Project | Lucene KStem represents the Krovetz-style path without adding separate dependency and license risk. |
| Smile Lancaster / Paice-Husk | English | Smile NLP | Smile is large for one stemmer; Paice/Husk is included through a smaller benchmark-only generated path. |
| CISTEM German stemmer | German | `https://github.com/LeonieWeissweiler/CISTEM` | Clean benchmark-only Java integration was not completed in this phase. |
| `stemmerEval` reference repository | Multiple | `https://github.com/endredy/stemmerEval` | Used only as a candidate reference; no code or data copied. |