feat!: modularize stemmer models and release infrastructure
Move bundled stemmer dictionaries from the core artifact into independently versioned model modules. Add model discovery and explicit model-loading APIs, a standard model aggregate, a model BOM, and dedicated model and catalog release workflows. Add full PoliMorf integration, model provenance and licensing validation, streaming model-input verification, strict dependency verification, consumer resolution tests, Configuration Cache compatibility, and expanded JMH, quality, documentation, and release checks. Upgrade the CycloneDX and JMH Gradle plugins and remove Gradle 10 and Java compiler deprecations. BREAKING CHANGE: The core Radixor artifact no longer contains bundled stemmer dictionaries. Applications must add the required model artifacts, the standard model aggregate, or model dependencies managed through the Radixor model BOM.
This commit is contained in:
@@ -1,14 +1,14 @@
|
||||
# Fast Track
|
||||
|
||||
This page is the shortest path from an empty Java project to a working Radixor stemmer.
|
||||
It deliberately uses a bundled dictionary and the preferred compiled-command runtime API, so the
|
||||
It deliberately uses an external model artifact and the preferred compiled-command runtime API, so the
|
||||
first result does not require writing a dictionary, running the CLI compiler, or understanding
|
||||
reduction internals.
|
||||
|
||||
Use this page when the goal is:
|
||||
|
||||
- add the dependency,
|
||||
- load a bundled language resource,
|
||||
- load a registered language model,
|
||||
- stem a token,
|
||||
- know where to go next.
|
||||
|
||||
@@ -23,14 +23,14 @@ groupId: org.egothor
|
||||
artifactId: radixor
|
||||
```
|
||||
|
||||
Use the current published version from Maven Central. The snippets below use `3.0.0`; replace it
|
||||
with the version you deploy if a newer release is available.
|
||||
Radixor 4 is not yet represented by a published release in this working tree. Replace the version placeholder with the reviewed release you deploy.
|
||||
|
||||
For a Gradle project:
|
||||
|
||||
```kotlin
|
||||
dependencies {
|
||||
implementation("org.egothor:radixor:3.0.0")
|
||||
implementation("org.egothor:radixor:<radixor-version>")
|
||||
runtimeOnly("org.egothor:radixor-model-us-uk-default:1.0.0")
|
||||
}
|
||||
```
|
||||
|
||||
@@ -40,17 +40,23 @@ For a Maven project:
|
||||
<dependency>
|
||||
<groupId>org.egothor</groupId>
|
||||
<artifactId>radixor</artifactId>
|
||||
<version>3.0.0</version>
|
||||
<version>${radixor.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.egothor</groupId>
|
||||
<artifactId>radixor-model-us-uk-default</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
```
|
||||
|
||||
Radixor targets modern Java and has a dependency-light runtime core. The project documentation and
|
||||
benchmarks assume a current JDK; Java 21 or newer is the practical baseline for current releases.
|
||||
|
||||
## 2. Load A Bundled Dictionary
|
||||
## 2. Load An External Model Dictionary
|
||||
|
||||
The fastest path is to use a bundled dictionary through `StemmerPatchTrieLoader.Language`.
|
||||
This example uses the bundled English resource, `US_UK`.
|
||||
The fastest path is to use a registered model through `StemmerPatchTrieLoader.Language`.
|
||||
This example uses `US_UK`, whose default ID is `us-uk-default`; the runtime model dependency above must be present.
|
||||
|
||||
```java
|
||||
import java.io.IOException;
|
||||
@@ -81,12 +87,11 @@ public final class RadixorFirstStem {
|
||||
}
|
||||
```
|
||||
|
||||
The loaded `FrequencyTrie<CompiledPatchCommand>` is immutable and can be shared across request
|
||||
threads. Load it once during application startup and reuse it for indexing and query processing.
|
||||
The loaded `FrequencyTrie<CompiledPatchCommand>` has no mutating API. Load it once during application startup, publish it safely through application-owned lifecycle code, and reuse it for indexing and query processing.
|
||||
|
||||
## 3. Choose A Language Resource
|
||||
## 3. Choose a Language Default or Explicit Model
|
||||
|
||||
Bundled dictionaries are exposed as enum constants. Common examples:
|
||||
Language defaults are exposed as enum constants. Common examples:
|
||||
|
||||
| Language | Enum constant |
|
||||
| --- | --- |
|
||||
@@ -102,6 +107,8 @@ Bundled dictionaries are exposed as enum constants. Common examples:
|
||||
The full list, writing-direction notes, and benchmark links are in
|
||||
[Built-in Languages](built-in-languages.md).
|
||||
|
||||
Polish has two models. `Language.PL_PL` selects `pl-pl-unimorph`; load the alternative explicitly with `StemmerPatchTrieLoader.loadCompiled("pl-pl-polimorf", true, reductionMode)`, or retain a registry and pass `registry.require("pl-pl-polimorf")` to the descriptor overload. See [Model Selection and Loading](model-selection-and-loading.md). Full PoliMorf construction requires substantially more startup heap than ordinary models; the repository verifies it in a dedicated 6 GiB test JVM.
|
||||
|
||||
## 4. Use The Same Stemmer On Both Sides
|
||||
|
||||
For search, use the same Radixor configuration during indexing and query processing. A typical
|
||||
@@ -118,7 +125,7 @@ limited to lookup and patch application.
|
||||
|
||||
## 5. Next Step For Production
|
||||
|
||||
The fast path compiles a bundled dictionary during startup. That is convenient for evaluation and
|
||||
The fast path parses and compiles a registered model dictionary during startup. That is convenient for evaluation and
|
||||
small services. For larger deployments, compile once, persist a `.radixor.gz` artifact, and load
|
||||
that binary artifact at runtime.
|
||||
|
||||
@@ -126,5 +133,6 @@ Continue with:
|
||||
|
||||
- [Integration Deep Dive](integration-deep-dive.md) for production lifecycle guidance.
|
||||
- [Loading and Building Stemmers](programmatic-loading-and-building.md) for all loading APIs.
|
||||
- [Built-in Languages](built-in-languages.md) for bundled resources and dictionary locations.
|
||||
- [Model Selection and Loading](model-selection-and-loading.md) for model dependencies, variants, and failures.
|
||||
- [Built-in Languages](built-in-languages.md) for defaults and optional variants.
|
||||
- [Benchmarking](benchmarking.md) for speed and quality interpretation.
|
||||
|
||||
Reference in New Issue
Block a user