feat!: modularize stemmer models and release infrastructure
Move bundled stemmer dictionaries from the core artifact into independently versioned model modules. Add model discovery and explicit model-loading APIs, a standard model aggregate, a model BOM, and dedicated model and catalog release workflows. Add full PoliMorf integration, model provenance and licensing validation, streaming model-input verification, strict dependency verification, consumer resolution tests, Configuration Cache compatibility, and expanded JMH, quality, documentation, and release checks. Upgrade the CycloneDX and JMH Gradle plugins and remove Gradle 10 and Java compiler deprecations. BREAKING CHANGE: The core Radixor artifact no longer contains bundled stemmer dictionaries. Applications must add the required model artifacts, the standard model aggregate, or model dependencies managed through the Radixor model BOM.
This commit is contained in:
@@ -1,6 +1,149 @@
|
||||
# Migration and Backward Compatibility
|
||||
|
||||
This page describes the migration from repeated serialized patch-command application to compiled patch commands.
|
||||
## Radixor 3.x to 4.x architecture migration
|
||||
|
||||
Radixor 3.x published algorithm classes and language dictionaries together as `org.egothor:radixor`. Radixor 4 keeps that established coordinate for the algorithmic core but removes every dictionary from the core JAR. Applications must now choose independently versioned model artifacts. This is deliberately source-compatible where practical and deliberately different at runtime.
|
||||
|
||||
### Before and after: dependencies
|
||||
|
||||
| Deployment | 3.x | 4.x |
|
||||
|---|---|---|
|
||||
| Core | `org.egothor:radixor:<3.x-version>` included dictionaries | `org.egothor:radixor:<radixor-version>` contains code only |
|
||||
| Minimal Polish | No separate data dependency | Add `radixor-model-pl-pl-unimorph:1.0.0` |
|
||||
| All defaults | Implicitly embedded | Add optional `radixor-models-standard:<catalog-version>` |
|
||||
| Optional Polish variant | Not independently selectable | Add and explicitly select `radixor-model-pl-pl-polimorf:1.0.0` |
|
||||
|
||||
Gradle, preserving the previous Polish default:
|
||||
|
||||
```groovy
|
||||
dependencies {
|
||||
implementation 'org.egothor:radixor:<radixor-version>'
|
||||
runtimeOnly 'org.egothor:radixor-model-pl-pl-unimorph:1.0.0'
|
||||
}
|
||||
```
|
||||
|
||||
Gradle, broad default coverage:
|
||||
|
||||
```groovy
|
||||
dependencies {
|
||||
implementation 'org.egothor:radixor:<radixor-version>'
|
||||
runtimeOnly 'org.egothor:radixor-models-standard:<catalog-version>'
|
||||
}
|
||||
```
|
||||
|
||||
Maven, preserving the Polish default:
|
||||
|
||||
```xml
|
||||
<dependency>
|
||||
<groupId>org.egothor</groupId>
|
||||
<artifactId>radixor</artifactId>
|
||||
<version>${radixor.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.egothor</groupId>
|
||||
<artifactId>radixor-model-pl-pl-unimorph</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
```
|
||||
|
||||
### Before and after: API behavior
|
||||
|
||||
Language-oriented calls remain source-compatible:
|
||||
|
||||
```java
|
||||
final FrequencyTrie<CompiledPatchCommand> polish =
|
||||
StemmerPatchTrieLoader.loadCompiled(
|
||||
StemmerPatchTrieLoader.Language.PL_PL,
|
||||
true,
|
||||
ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);
|
||||
```
|
||||
|
||||
In 4.x this call creates a registry and resolves `Language.PL_PL.defaultModelId()`, which is `pl-pl-unimorph`. Source compatibility does not imply runtime classpath compatibility: the call fails with `StemmerModelNotFoundException` unless that model is visible.
|
||||
|
||||
Explicit selection enables multiple variants:
|
||||
|
||||
```java
|
||||
final StemmerModelRegistry registry = StemmerModelRegistry.fromContextClassLoader();
|
||||
final StemmerModelDescriptor polimorf = registry.require("pl-pl-polimorf");
|
||||
final FrequencyTrie<CompiledPatchCommand> trie =
|
||||
StemmerPatchTrieLoader.loadCompiled(
|
||||
polimorf,
|
||||
true,
|
||||
ReductionMode.MERGE_SUBTREES_WITH_EQUIVALENT_RANKED_GET_ALL_RESULTS);
|
||||
```
|
||||
|
||||
The existing `load(String, ...)` overload means a filesystem path. The compiled `loadCompiled(String, boolean, ReductionMode)` overload now means a stable model ID; use the `Path` overload for a filesystem dictionary. Descriptor-based compiled loading avoids rediscovery when an application retains a registry. See [Model Selection and Loading](model-selection-and-loading.md) for complete examples.
|
||||
|
||||
### Polish migration scenarios
|
||||
|
||||
1. **Preserve previous default behavior:** add `radixor-model-pl-pl-unimorph` and keep using `Language.PL_PL`.
|
||||
2. **Use PoliMorf:** add `radixor-model-pl-pl-polimorf` and call `registry.require("pl-pl-polimorf")`.
|
||||
3. **Deploy both:** add both runtime artifacts and load each descriptor by ID. They are not merged.
|
||||
4. **Verify selection:** compare `registry.requireDefault(Language.PL_PL).id()` with `pl-pl-unimorph` through normal application control flow or a JUnit assertion, and inspect `registry.findByLanguage(Language.PL_PL)`.
|
||||
5. **Diagnose absence:** read the exact `StemmerModelNotFoundException` message, then inspect the production `runtimeClasspath` rather than changing dependency order.
|
||||
|
||||
UniMorph and PoliMorf are not interchangeable quality datasets. They can differ in vocabulary, provenance, licensing, and stemming outputs.
|
||||
|
||||
Model migration does not erase source obligations. Each migrated UniMorph artifact packages its
|
||||
language-specific notice with upstream attribution, Radixor modifications and contribution
|
||||
statement, ShareAlike terms, and the canonical CC BY-SA 3.0 URI. The original imports did not
|
||||
record exact UniMorph commits, so descriptors use
|
||||
`source.revision=not-recorded-in-legacy-import` and disclose that fact. Future model imports must
|
||||
record an exact upstream revision and source-archive checksum.
|
||||
|
||||
### Compatibility table
|
||||
|
||||
| Dimension | 4.x migration status |
|
||||
|---|---|
|
||||
| Source compatibility | Language-oriented loader signatures remain; external model dependencies are new |
|
||||
| Binary compatibility | Removing resources is a major-version boundary; review all deployed artifacts |
|
||||
| Runtime classpath | At least one selected model JAR is required |
|
||||
| Model format | Descriptor format `radixor-dictionary-tsv-gzip` version `1` is validated by the registry |
|
||||
| Model IDs | Stable runtime identities, independent of artifact discovery order |
|
||||
| Core Maven coordinate | Remains `org.egothor:radixor` |
|
||||
| Release versions | Core, each model, upstream source, format, and catalog versions evolve separately |
|
||||
|
||||
### Upgrade checklist
|
||||
|
||||
- Update the core dependency.
|
||||
- Choose individual model artifacts or the standard pack.
|
||||
- Put resource-only model dependencies on the production runtime classpath.
|
||||
- Verify `Language.defaultModelId()` mappings used by the application.
|
||||
- Inspect shaded, minimized, plugin, or modular packaging for indexes and resources.
|
||||
- Run application-level vocabulary and output regression tests.
|
||||
- Track model artifact versions and checksums separately from the core version.
|
||||
|
||||
### Roll back model choice
|
||||
|
||||
To return from optional PoliMorf to the default UniMorph behavior, add or retain `radixor-model-pl-pl-unimorph`, stop requesting `pl-pl-polimorf`, and load `Language.PL_PL` or explicitly request `pl-pl-unimorph`. Do not change the language constant. Remove the unused PoliMorf runtime dependency after verifying no explicit lookup still needs it.
|
||||
|
||||
Rolling the whole application back to 3.x instead requires restoring the reviewed 3.x core dependency and removing 4.x model assumptions. Do not combine 3.x embedded resources with the 4.x registry architecture.
|
||||
|
||||
Core, model, and catalog releases are independent:
|
||||
|
||||
```bash
|
||||
git tag -a "release@4.0.0" -m "Release Radixor 4.0.0"
|
||||
git tag -a "model/pl-pl-polimorf@1.0.0" -m "Release Polish PoliMorf model 1.0.0"
|
||||
git tag -a "models-catalog@2026.1" -m "Release Radixor model catalog 2026.1"
|
||||
```
|
||||
|
||||
A core tag publishes only the root `org.egothor:radixor` software artifacts, never model JARs. A model tag validates and publishes exactly its matching module, never core, standard, BOM, JMH, or the multilingual quality suite. A catalog tag publishes only BOM and standard aggregate metadata. Local model dry-run:
|
||||
|
||||
The catalog artifacts are POM-only: `radixor-models-standard` carries runtime dependencies on the 20 defaults, while `radixor-models-bom` carries dependency-management constraints for all 21 individual models. Neither publishes an empty binary, sources, or Javadoc JAR. This Maven BOM is distinct from the root CycloneDX SBOM report under `build/reports/sbom/`.
|
||||
|
||||
```bash
|
||||
./tools/parse-model-release-tag.sh "model/pl-pl-polimorf@1.0.0" .
|
||||
./gradlew --no-daemon :models:pl-pl-polimorf:check
|
||||
./gradlew --no-daemon :models:pl-pl-polimorf:validateModelRelease -PmodelReleaseVersion=1.0.0
|
||||
./gradlew --no-daemon :models:pl-pl-polimorf:packageModelReleaseCandidate -PmodelReleaseVersion=1.0.0
|
||||
```
|
||||
|
||||
Model format compatibility is descriptor-level and does not alter migrated bytes. Version 1 is `radixor-dictionary-tsv-gzip`. Model versions come from each module's `model-version.txt` or the matching explicit release property; catalog version comes from `models/catalog-version.txt`; only core uses Git-derived `release@` versioning.
|
||||
|
||||
The model catalog used by the published documentation is generated under `build/mkdocs-source/`. Neither generated Markdown nor rendered MkDocs output belongs in Git.
|
||||
|
||||
The remainder of this page describes the earlier migration from repeated serialized patch-command application to compiled patch commands.
|
||||
|
||||
## Summary
|
||||
|
||||
|
||||
Reference in New Issue
Block a user