Files

feat!: modularize stemmer models and release infrastructure

Move bundled stemmer dictionaries from the core artifact into independently
versioned model modules. Add model discovery and explicit model-loading APIs,
a standard model aggregate, a model BOM, and dedicated model and catalog
release workflows.

Add full PoliMorf integration, model provenance and licensing validation,
streaming model-input verification, strict dependency verification, consumer
resolution tests, Configuration Cache compatibility, and expanded JMH,
quality, documentation, and release checks.

Upgrade the CycloneDX and JMH Gradle plugins and remove Gradle 10 and Java
compiler deprecations.

BREAKING CHANGE: The core Radixor artifact no longer contains bundled stemmer
dictionaries. Applications must add the required model artifacts, the standard
model aggregate, or model dependencies managed through the Radixor model BOM.

2026-07-22 23:33:28 +02:00

5.6 KiB

Raw Blame History

Programmatic Usage

Radixor code and model data are separate runtime components. Every example on this page requires org.egothor:radixor:<radixor-version> as an implementation dependency and at least one model JAR as a runtime dependency. The core JAR contains no stemmer.gz.

For complete dependency patterns, lifecycle guidance, and troubleshooting, use Model Selection and Loading. The generated model catalog records the current artifacts, versions, checksums, and provenance.

1. Minimal use: the Polish default

Dependency prerequisite:

implementation 'org.egothor:radixor:<radixor-version>'
runtimeOnly 'org.egothor:radixor-model-pl-pl-unimorph:1.0.0'

import org.egothor.stemmer.CompiledPatchCommand;
import org.egothor.stemmer.FrequencyTrie;
import org.egothor.stemmer.ReductionMode;
import org.egothor.stemmer.StemmerPatchTrieLoader;

final FrequencyTrie<CompiledPatchCommand> trie =
        StemmerPatchTrieLoader.loadCompiled(
                StemmerPatchTrieLoader.Language.PL_PL,
                true,
                ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);

final String word = "koty";
final CompiledPatchCommand patch = trie.get(word);
final String stem = patch == null ? word : patch.apply(word);

Language.PL_PL resolves to pl-pl-unimorph. The loader creates the registry internally through the thread context class loader.

2. Explicit model selection

Dependency prerequisite: replace or supplement the default dependency with runtimeOnly 'org.egothor:radixor-model-pl-pl-polimorf:1.0.0'.

final StemmerModelRegistry registry = StemmerModelRegistry.fromContextClassLoader();
final StemmerModelDescriptor polimorf = registry.require("pl-pl-polimorf");
final FrequencyTrie<CompiledPatchCommand> trie =
        StemmerPatchTrieLoader.loadCompiled(
                polimorf,
                true,
                ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);

The stable model-ID overload performs the same exact selection without a separately retained registry:

final FrequencyTrie<CompiledPatchCommand> trie =
        StemmerPatchTrieLoader.loadCompiled(
                "pl-pl-polimorf",
                true,
                ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);

3. Multiple variants for one language

Dependency prerequisite: both radixor-model-pl-pl-unimorph:1.0.0 and radixor-model-pl-pl-polimorf:1.0.0 at runtime.

final StemmerModelRegistry registry = StemmerModelRegistry.fromContextClassLoader();
final StemmerModelDescriptor unimorph = registry.require("pl-pl-unimorph");
final StemmerModelDescriptor polimorf = registry.require("pl-pl-polimorf");

final FrequencyTrie<CompiledPatchCommand> unimorphTrie =
        StemmerPatchTrieLoader.loadCompiled(unimorph, true, reductionMode);
final FrequencyTrie<CompiledPatchCommand> polimorfTrie =
        StemmerPatchTrieLoader.loadCompiled(polimorf, true, reductionMode);

final StemmerModelDescriptor defaultPolish =
        registry.requireDefault(StemmerPatchTrieLoader.Language.PL_PL);
if (!"pl-pl-unimorph".equals(defaultPolish.id())) {
    throw new IllegalStateException(
            "Unexpected default Polish model: " + defaultPolish.id());
}

The tries remain independent. Radixor does not merge models or infer an alternative default from classpath order.

4. Discovery

Dependency prerequisite: whichever model artifacts the application intends to discover.

final StemmerModelRegistry registry = StemmerModelRegistry.fromContextClassLoader();

for (final StemmerModelDescriptor descriptor : registry.models()) {
    System.out.printf("%s %s %s %s/%d%n",
            descriptor.id(), descriptor.language(), descriptor.version(),
            descriptor.format(), descriptor.formatVersion());
}

final java.util.List<StemmerModelDescriptor> polish =
        registry.findByLanguage(StemmerPatchTrieLoader.Language.PL_PL);

Results use deterministic model-ID order. See Built-in Languages for default interpretation and the generated catalog for provenance.

5. Advanced ClassLoader selection

Dependency prerequisite: the model JAR must be visible to the selected loader.

final ClassLoader applicationLoader = application.getClass().getClassLoader();
final StemmerModelRegistry isolatedRegistry =
        StemmerModelRegistry.fromClassLoader(applicationLoader);

This form is useful for plugin containers, isolated application servers, and tests. It can discover a different set from the thread context loader. See ClassLoader troubleshooting.

6. Error handling

Dependency prerequisite: none beyond core; this example demonstrates an absent optional model.

try {
    StemmerModelRegistry.fromContextClassLoader().require("pl-pl-polimorf");
} catch (final StemmerModelNotFoundException exception) {
    System.err.println(exception.getMessage());
}

Missing models never produce an empty trie or arbitrary fallback. Duplicate IDs, unsupported formats, malformed descriptors, missing resources, and checksum mismatches are also fatal. The full exception mapping and remediation table are in Model Selection and Loading.

5.6 KiB Raw Blame History