feat!: modularize stemmer models and release infrastructure

Move bundled stemmer dictionaries from the core artifact into independently
versioned model modules. Add model discovery and explicit model-loading APIs,
a standard model aggregate, a model BOM, and dedicated model and catalog
release workflows.

Add full PoliMorf integration, model provenance and licensing validation,
streaming model-input verification, strict dependency verification, consumer
resolution tests, Configuration Cache compatibility, and expanded JMH,
quality, documentation, and release checks.

Upgrade the CycloneDX and JMH Gradle plugins and remove Gradle 10 and Java
compiler deprecations.

BREAKING CHANGE: The core Radixor artifact no longer contains bundled stemmer
dictionaries. Applications must add the required model artifacts, the standard
model aggregate, or model dependencies managed through the Radixor model BOM.
This commit is contained in:
2026-07-22 23:33:28 +02:00
parent 9c5b9e331b
commit e7800b29c9
250 changed files with 7297 additions and 905 deletions

View File

@@ -1,14 +1,14 @@
# Fast Track
This page is the shortest path from an empty Java project to a working Radixor stemmer.
It deliberately uses a bundled dictionary and the preferred compiled-command runtime API, so the
It deliberately uses an external model artifact and the preferred compiled-command runtime API, so the
first result does not require writing a dictionary, running the CLI compiler, or understanding
reduction internals.
Use this page when the goal is:
- add the dependency,
- load a bundled language resource,
- load a registered language model,
- stem a token,
- know where to go next.
@@ -23,14 +23,14 @@ groupId: org.egothor
artifactId: radixor
```
Use the current published version from Maven Central. The snippets below use `3.0.0`; replace it
with the version you deploy if a newer release is available.
Radixor 4 is not yet represented by a published release in this working tree. Replace the version placeholder with the reviewed release you deploy.
For a Gradle project:
```kotlin
dependencies {
implementation("org.egothor:radixor:3.0.0")
implementation("org.egothor:radixor:<radixor-version>")
runtimeOnly("org.egothor:radixor-model-us-uk-default:1.0.0")
}
```
@@ -40,17 +40,23 @@ For a Maven project:
<dependency>
<groupId>org.egothor</groupId>
<artifactId>radixor</artifactId>
<version>3.0.0</version>
<version>${radixor.version}</version>
</dependency>
<dependency>
<groupId>org.egothor</groupId>
<artifactId>radixor-model-us-uk-default</artifactId>
<version>1.0.0</version>
<scope>runtime</scope>
</dependency>
```
Radixor targets modern Java and has a dependency-light runtime core. The project documentation and
benchmarks assume a current JDK; Java 21 or newer is the practical baseline for current releases.
## 2. Load A Bundled Dictionary
## 2. Load An External Model Dictionary
The fastest path is to use a bundled dictionary through `StemmerPatchTrieLoader.Language`.
This example uses the bundled English resource, `US_UK`.
The fastest path is to use a registered model through `StemmerPatchTrieLoader.Language`.
This example uses `US_UK`, whose default ID is `us-uk-default`; the runtime model dependency above must be present.
```java
import java.io.IOException;
@@ -81,12 +87,11 @@ public final class RadixorFirstStem {
}
```
The loaded `FrequencyTrie<CompiledPatchCommand>` is immutable and can be shared across request
threads. Load it once during application startup and reuse it for indexing and query processing.
The loaded `FrequencyTrie<CompiledPatchCommand>` has no mutating API. Load it once during application startup, publish it safely through application-owned lifecycle code, and reuse it for indexing and query processing.
## 3. Choose A Language Resource
## 3. Choose a Language Default or Explicit Model
Bundled dictionaries are exposed as enum constants. Common examples:
Language defaults are exposed as enum constants. Common examples:
| Language | Enum constant |
| --- | --- |
@@ -102,6 +107,8 @@ Bundled dictionaries are exposed as enum constants. Common examples:
The full list, writing-direction notes, and benchmark links are in
[Built-in Languages](built-in-languages.md).
Polish has two models. `Language.PL_PL` selects `pl-pl-unimorph`; load the alternative explicitly with `StemmerPatchTrieLoader.loadCompiled("pl-pl-polimorf", true, reductionMode)`, or retain a registry and pass `registry.require("pl-pl-polimorf")` to the descriptor overload. See [Model Selection and Loading](model-selection-and-loading.md). Full PoliMorf construction requires substantially more startup heap than ordinary models; the repository verifies it in a dedicated 6 GiB test JVM.
## 4. Use The Same Stemmer On Both Sides
For search, use the same Radixor configuration during indexing and query processing. A typical
@@ -118,7 +125,7 @@ limited to lookup and patch application.
## 5. Next Step For Production
The fast path compiles a bundled dictionary during startup. That is convenient for evaluation and
The fast path parses and compiles a registered model dictionary during startup. That is convenient for evaluation and
small services. For larger deployments, compile once, persist a `.radixor.gz` artifact, and load
that binary artifact at runtime.
@@ -126,5 +133,6 @@ Continue with:
- [Integration Deep Dive](integration-deep-dive.md) for production lifecycle guidance.
- [Loading and Building Stemmers](programmatic-loading-and-building.md) for all loading APIs.
- [Built-in Languages](built-in-languages.md) for bundled resources and dictionary locations.
- [Model Selection and Loading](model-selection-and-loading.md) for model dependencies, variants, and failures.
- [Built-in Languages](built-in-languages.md) for defaults and optional variants.
- [Benchmarking](benchmarking.md) for speed and quality interpretation.