feat(python): add native distribution and release infrastructure

- add the Rust-backed Python API with PyStemmer compatibility
- distribute standard compiled models as a separate Python package
- generate model artifacts during builds instead of storing them in Git
- add GitHub release and Pages-backed package index workflows
- add Python tests, benchmarks, documentation, and Gradle integration
- refresh the documentation site, branding, and language benchmarks
This commit is contained in:
2026-08-10 22:34:32 +02:00
parent b45e143c84
commit 5e3d3c7c7d
139 changed files with 11420 additions and 747 deletions

130
docs/python/quick-start.md Normal file
View File

@@ -0,0 +1,130 @@
# Python Quick Start
Radixor's Python implementation is a native Rust extension with a Python API.
It uses the same learned patch-command model and version 7 compiled-trie format
as the Java implementation, without requiring a JVM.
## 1. Install the runtime and standard models
Create an isolated environment and install Radixor:
```bash
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
```
=== "PyPI"
```bash
python -m pip install --only-binary=:all: radixor
```
=== "GitHub Releases"
```bash
python -m pip install --only-binary=:all: \
--index-url https://leogalambos.github.io/Radixor/python/simple/ radixor
```
PyPI publication is pending. The GitHub option becomes live when the first
Python GitHub Releases populate the Pages-backed package index. See
[Installation and Builds](installation.md) for availability and source builds.
The `radixor` wheel contains code. Its required
`radixor-models-standard` dependency contains 20 precompiled models. The
standard package excludes textual source dictionaries and optional PoliMorf
data, which keeps startup on the direct compiled-model path.
## 2. Select and reuse a stemmer
Construct a stemmer once and retain it for the lifetime of the application:
```python
from radixor import Stemmer
english = Stemmer("en")
polish = Stemmer("pl")
print(english.stem("running")) # 'run'
print(polish.stem("koty"))
```
Short aliases such as `en`, `de`, and `pl` resolve to the documented default
model IDs. A full ID such as `us-uk-default` selects the same model explicitly.
The complete mapping is listed under [Built-in Languages](../built-in-languages.md).
## 3. Prefer batch calls for collections
Crossing the Python/native boundary once per collection is substantially more
efficient than a Python loop of scalar calls:
```python
words = ["running", "studies", "better", "cars"]
stems = english.stem_batch(words)
```
`stem_batch()` preserves input order and returns one item per word. Entries can
be `None` when the trie has no applicable patch command.
Repeated natural-language tokens use a bounded result cache shared by the
scalar and batch APIs. Its default capacity is 10,000 entries, matching
PyStemmer; choose another bound or pass `0` to disable it:
```python
english = Stemmer("en", cache_size=10_000)
uncached = Stemmer("en", cache_size=0)
```
The cache covers `stem()`, `stemWord()`, `stem_batch()`, and `stemWords()`;
the `stem_all*()` methods are not cached.
## 4. Migrate from PyStemmer
Radixor exposes PyStemmer's familiar scalar and batch method names:
```python
stemmer = Stemmer("en")
stemmer.stemWord("running")
stemmer.stemWords(["running", "unknown_word"])
```
`stemWord()` and `stemWords()` return unmatched input unchanged. This removes
the `None` fallback checks required by Radixor's original `stem()` and
`stem_batch()` methods, so most migration work is limited to the package import
and dependency change.
## 5. Load a custom compiled model
The standard installation covers the maintained default catalog. A custom
version 7 model can be loaded directly:
```python
custom = Stemmer(compiled="models/domain-english.rxc")
```
To compile a maintained textual dictionary during a preparation step:
```python
from radixor import compile
compile("dictionaries/domain.tsv.gz", "models/domain-english.rxc", language="en")
```
Deploy the resulting `.rxc` file and load it at application startup. See
[Dictionary Compilation](model-compilation.md) for format interoperability and
the production compilation profile.
## 6. Production checklist
- Pin compatible `radixor` and `radixor-models-standard` releases in the
application's dependency lock.
- Construct and reuse stemmers instead of rebuilding them per request.
- Use batch calls for token collections.
- Choose `stem*` or `stemWord*` semantics deliberately for unmatched words.
- Treat custom dictionaries and compiled models as trusted application input.
- Regression-test representative vocabulary before changing model versions.
Continue with [Installation and Builds](installation.md) for wheel/platform
details, [Usage and API](usage.md) for the complete call surface, or
[Performance](performance.md) for benchmark methodology and results.