feat(python): add native distribution and release infrastructure

- add the Rust-backed Python API with PyStemmer compatibility
- distribute standard compiled models as a separate Python package
- generate model artifacts during builds instead of storing them in Git
- add GitHub release and Pages-backed package index workflows
- add Python tests, benchmarks, documentation, and Gradle integration
- refresh the documentation site, branding, and language benchmarks
This commit is contained in:
2026-08-10 22:34:32 +02:00
parent b45e143c84
commit 5e3d3c7c7d
139 changed files with 11420 additions and 747 deletions

View File

@@ -0,0 +1,12 @@
Radixor Standard Model Data License
The model data in this distribution, including Radixor's protectable
selection, transformation, metadata, and packaging contributions, is licensed
under Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0):
https://creativecommons.org/licenses/by-sa/3.0/
Each model's attribution, upstream provenance, and license details are recorded
in radixor_models_standard/notices/<model-id>/NOTICE-model-data.txt. Those
notices form part of this distribution and must be retained with redistributed
model data.

View File

@@ -0,0 +1,4 @@
include LICENSE-MODEL-DATA.txt
include README.md
recursive-include radixor_models_standard *.json *.rxc *.txt
global-exclude stemmer.gz *.gz __pycache__ *.py[cod]

View File

@@ -0,0 +1,26 @@
# Radixor standard models
This pure-Python distribution supplies Radixor's 20 precompiled standard
language models. It is installed automatically by `pip install radixor`; users
normally do not import it directly.
The catalog version is `2026.1`. Individual model versions recorded in the
generated `radixor_models_standard/manifest.json` are currently `1.0.0`. The
optional Polish PoliMorf model is intentionally not part of the standard
catalog.
The first Python model distribution is released as `1.0.0`; its version is
independent of both the catalog identity and the individual model versions.
The checked-in descriptors use `0.0.0` as a deliberate non-release placeholder.
The release workflow creates an isolated project below `build/`, injects the Git
tag version, and deterministically compiles all model resources there.
Only gzip-compressed Radixor v7 (`.rxc`) tries are shipped. They are release
artifacts, not checked-in repository files. Canonical textual dictionaries
remain in the Radixor source repository and are not included in this wheel or
source distribution. Model data is licensed under CC BY-SA 3.0; see
`LICENSE-MODEL-DATA.txt` and the generated per-model notices.
The checked-out directory is intentionally only a packaging skeleton and is
not directly buildable as the complete data distribution. From the repository
root, use `./gradlew pythonBuildStandardModels`; the generated project and its
wheel/sdist are written below `build/python/`.

View File

@@ -0,0 +1,29 @@
[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
[project]
name = "radixor-models-standard"
version = "0.0.0"
requires-python = ">=3.9"
description = "Precompiled standard language models for Radixor"
readme = "README.md"
license = "CC-BY-SA-3.0"
license-files = ["LICENSE-MODEL-DATA.txt"]
keywords = ["stemming", "nlp", "linguistics", "model-data"]
classifiers = [
"Programming Language :: Python :: 3",
"Topic :: Text Processing :: Linguistic",
]
[tool.setuptools]
include-package-data = true
[tool.setuptools.packages.find]
where = ["."]
include = ["radixor_models_standard*"]
[tool.setuptools.package-data]
radixor_models_standard = ["manifest.json"]
"radixor_models_standard.models" = ["*.rxc"]
"radixor_models_standard.notices" = ["*/*.txt"]

View File

@@ -0,0 +1,37 @@
###############################################################################
# Copyright (C) 2026, Leo Galambos
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
###############################################################################
"""Installed resources for the Radixor 2026.1 standard model catalog."""
CATALOG_VERSION = "2026.1"
__version__ = "0.0.0"
__all__ = ["CATALOG_VERSION"]

View File

@@ -0,0 +1,32 @@
###############################################################################
# Copyright (C) 2026, Leo Galambos
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
###############################################################################
"""Compiled model resources; not a public Python API."""

View File

@@ -0,0 +1,32 @@
###############################################################################
# Copyright (C) 2026, Leo Galambos
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
###############################################################################
"""Per-model attribution and license notice resources."""