feat!: modularize stemmer models and release infrastructure
Move bundled stemmer dictionaries from the core artifact into independently versioned model modules. Add model discovery and explicit model-loading APIs, a standard model aggregate, a model BOM, and dedicated model and catalog release workflows. Add full PoliMorf integration, model provenance and licensing validation, streaming model-input verification, strict dependency verification, consumer resolution tests, Configuration Cache compatibility, and expanded JMH, quality, documentation, and release checks. Upgrade the CycloneDX and JMH Gradle plugins and remove Gradle 10 and Java compiler deprecations. BREAKING CHANGE: The core Radixor artifact no longer contains bundled stemmer dictionaries. Applications must add the required model artifacts, the standard model aggregate, or model dependencies managed through the Radixor model BOM.
This commit is contained in:
23
models/fi-fi-default/build.gradle
Normal file
23
models/fi-fi-default/build.gradle
Normal file
@@ -0,0 +1,23 @@
|
||||
plugins {
|
||||
id 'org.egothor.radixor.model'
|
||||
}
|
||||
|
||||
radixorModel {
|
||||
modelId = 'fi-fi-default'
|
||||
language = 'FI_FI'
|
||||
displayName = 'Finnish default model'
|
||||
defaultModel = true
|
||||
sourceName = 'UniMorph'
|
||||
sourceVersion = 'not-recorded-in-legacy-import'
|
||||
sourceRevision = 'not-recorded-in-legacy-import'
|
||||
sourceProject = 'UniMorph'
|
||||
sourceRepository = 'https://github.com/unimorph/fin'
|
||||
sourceDataset = 'UniMorph Finnish morphological dataset (`fin`)'
|
||||
sourceRevisionStatus = 'not-recorded-in-legacy-import'
|
||||
sourceLicense = 'CC-BY-SA-3.0'
|
||||
sourceLicenseUri = 'https://creativecommons.org/licenses/by-sa/3.0/'
|
||||
sourceAttribution = 'UniMorph and Wikipedia contributors'
|
||||
sourceVerificationDate = '2026-07-22'
|
||||
transformationsSummary = 'Cleaning, normalization, grouping inflected forms by lemma, deduplication, filtering invalid rows, reformatting into Radixor dictionary groups, GZip packaging, and generation of runtime descriptor and checksum metadata'
|
||||
noticeFileName = 'NOTICE-model-data.txt'
|
||||
}
|
||||
1
models/fi-fi-default/model-version.txt
Normal file
1
models/fi-fi-default/model-version.txt
Normal file
@@ -0,0 +1 @@
|
||||
1.0.0
|
||||
38
models/fi-fi-default/src/modelInput/NOTICE-model-data.txt
Normal file
38
models/fi-fi-default/src/modelInput/NOTICE-model-data.txt
Normal file
@@ -0,0 +1,38 @@
|
||||
Radixor model-data notice
|
||||
|
||||
Radixor-derived model data
|
||||
|
||||
Copyright (C) 2026, Leo Galambos.
|
||||
|
||||
Copyright and, where applicable, database rights are claimed in the
|
||||
Radixor-specific selection, verification, cleaning, normalization,
|
||||
grouping, deduplication, filtering, reformatting, metadata preparation,
|
||||
and packaging of this model, to the extent protected by applicable law.
|
||||
|
||||
The underlying morphological data remains attributed to UniMorph and
|
||||
the upstream contributors identified in this notice.
|
||||
|
||||
This derived model data, including Radixor's protectable contributions,
|
||||
is distributed under Creative Commons Attribution-ShareAlike 3.0
|
||||
Unported.
|
||||
|
||||
Model ID: fi-fi-default
|
||||
Radixor language: FI_FI
|
||||
Source project: UniMorph
|
||||
Official repository: https://github.com/unimorph/fin
|
||||
Upstream dataset: UniMorph Finnish morphological dataset (`fin`)
|
||||
Upstream lexical source: Wikipedia
|
||||
Attribution: UniMorph and Wikipedia contributors
|
||||
License:
|
||||
Creative Commons Attribution-ShareAlike 3.0 Unported
|
||||
Canonical license URI: https://creativecommons.org/licenses/by-sa/3.0/
|
||||
Source revision: not-recorded-in-legacy-import
|
||||
Revision status: not-recorded-in-legacy-import
|
||||
|
||||
The exact UniMorph commit used for the original Radixor import was not recorded. The model remains attributed to the official UniMorph language repository and is distributed under the repository's stated data license.
|
||||
|
||||
Radixor modifications: Cleaning, normalization, grouping inflected forms by lemma, deduplication, filtering invalid rows, reformatting into Radixor dictionary groups, GZip packaging, and generation of runtime descriptor and checksum metadata.
|
||||
|
||||
The derived model data is distributed under CC BY-SA 3.0. UniMorph supplies morphological data; Radixor constructs its own patch-command trie at runtime. Neither UniMorph nor any upstream contributor endorses Radixor.
|
||||
|
||||
Upstream information verified: 2026-07-22
|
||||
BIN
models/fi-fi-default/src/modelInput/stemmer.gz
Normal file
BIN
models/fi-fi-default/src/modelInput/stemmer.gz
Normal file
Binary file not shown.
Reference in New Issue
Block a user