Files
Radixor/docs/python/installation.md
Leo Galambos 5e3d3c7c7d feat(python): add native distribution and release infrastructure
- add the Rust-backed Python API with PyStemmer compatibility
- distribute standard compiled models as a separate Python package
- generate model artifacts during builds instead of storing them in Git
- add GitHub release and Pages-backed package index workflows
- add Python tests, benchmarks, documentation, and Gradle integration
- refresh the documentation site, branding, and language benchmarks
2026-08-10 22:34:32 +02:00

8.0 KiB
Raw Blame History

Installing and building (Linux, Windows, macOS)

The Python package ships as an abi3 wheel — a single binary per OS/architecture that works on CPython ≥ 3.9 (including 3.14) through the stable ABI. Most users just pip install; building from source is only needed for development or unsupported platforms.

Install from PyPI

PyPI is the intended primary index once the Radixor projects are approved and published there:

python -m pip install --only-binary=:all: radixor

PyPI publication is not live yet. Until the radixor and radixor-models-standard project pages exist, this command cannot install the project.

Install compiled packages from GitHub

Python releases are published as immutable GitHub Release assets. A small PEP 503 index on GitHub Pages exposes both packages to pip:

python -m pip install --only-binary=:all: \
  --index-url https://leogalambos.github.io/Radixor/python/simple/ radixor

The index links directly to checksummed wheel assets in GitHub Releases; Pages does not duplicate the package files. It is not live until the first Python model and native releases have been published. This was verified before the initial release: the URL returned HTTP 404 and the repository contained no Python Release assets.

Do not configure the GitHub index as an --extra-index-url: pip does not prioritize one index over another. Use it as the sole --index-url, as shown above. The binary-only constraint also prevents an accidental source build with an unprepared toolchain.

Wheels are provided for Linux (manylinux), Windows, and macOS (x8664 and Apple Silicon). A source distribution is also published; installing it triggers a source build, which needs the toolchain described below.

Install or build from the GitHub source repository

Building requires the Rust toolchain, a linker for the target platform, and maturin. The crate and its dependencies contain no project C/C++ sources, but the selected Rust target still needs its normal platform linker and SDK.

git clone https://github.com/leogalambos/Radixor
cd Radixor
python -m venv python/.venv
# activate the venv (see per-OS note below)
pip install maturin build setuptools wheel pytest
./gradlew pythonBuildStandardModels
pip install --no-deps build/python/dist/standard/radixor_models_standard-0.0.0-py3-none-any.whl
cd python
maturin develop --release        # compile + install into the venv
pytest -q                        # run the test suite

For a reproducible application build, check out a release tag or exact commit instead of a moving branch. Repository descriptors deliberately use the non-release placeholder 0.0.0; release workflows inject the version from the Git tag into isolated staging trees. Consequently, source-checkout development installs use --no-deps for the generated development model wheel, while published packages carry normal release versions and dependency resolution works automatically.

The native distribution requires radixor-models-standard>=1.0,<2.0; an installation of radixor resolves it automatically. The local --no-deps command installs the generated data wheel for development without contacting a package index.

The installed package source (package index, environment, and sys.path) is the model-provider trust boundary. Manifest SHA-256 checks detect accidental corruption after installation; they do not authenticate a malicious provider.

Integrity and provenance

Every GitHub Release contains SHA256SUMS for its wheel and source archives. The release workflows also create GitHub artifact attestations for those archives. After downloading a release, maintainers and users can verify it with:

sha256sum --check SHA256SUMS
gh attestation verify radixor-<version>-<wheel-tags>.whl \
  --repo leogalambos/Radixor

Python packages do not reuse the OpenPGP key configured for Java/Maven Central publications. Java's SIGNING_KEY and SIGNING_PASSWORD produce Maven signatures; Python currently uses release checksums plus GitHub's identity-bound build-provenance attestation. A future PyPI publication should use PyPI Trusted Publishing and its supported attestations rather than copying the Java signing mechanism.

Build through Gradle

From the repository root, the supported build entry point creates the native wheel/sdist and pure standard-model wheel/sdist:

./gradlew pythonBuild

Artifacts are written below build/python/dist/; they are not installed into the invoking interpreter. ./gradlew pythonVerifyDistributions also validates archive contents, dependency metadata, checksums, v7 headers, and a fresh offline wheel-only installation. Platform convenience tasks are also available:

./gradlew pythonBuildLinux
./gradlew pythonBuildWindows
./gradlew pythonBuildMacos

The task matching the current host delegates to pythonBuild. A non-host task uses the default Rust target for that operating system and therefore succeeds only when its Rust target, linker, and platform SDK are installed. Override a default with pythonLinuxTarget, pythonWindowsTarget, or pythonMacosTarget. For example:

./gradlew pythonBuildWindows -PpythonWindowsTarget=x86_64-pc-windows-gnu

Use -PpythonExecutable=/path/to/python or -PmaturinExecutable=/path/to/maturin when those tools are not on PATH. These Gradle tasks are the repository integration; direct maturin commands remain useful while developing inside python/.

Prerequisites per platform

=== "Linux"

```bash
# Rust (rustup); most distros already ship Python 3.9+
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
python3 -m venv .venv && source .venv/bin/activate
pip install maturin pytest
```
Producing distributable `manylinux` wheels is easiest with
`maturin build --release` inside the official maturin/`manylinux` container.

=== "macOS"

```bash
brew install rustup-init && rustup-init -y      # or: curl https://sh.rustup.rs | sh
python3 -m venv .venv && source .venv/bin/activate
pip install maturin pytest
maturin develop --release
```
Both Apple Silicon (`aarch64-apple-darwin`) and Intel
(`x86_64-apple-darwin`) are supported; `maturin build --release --target
universal2-apple-darwin` produces a universal wheel.

=== "Windows"

```powershell
winget install -e --id Rustlang.Rustup
py -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install maturin pytest
maturin develop --release
```
The self-contained GNU toolchain avoids needing Visual Studio Build Tools:
```powershell
rustup toolchain install stable-x86_64-pc-windows-gnu
rustup default stable-x86_64-pc-windows-gnu
```
(The MSVC toolchain also works if you already have the C++ Build Tools.)

Python 3.14 (and newer than your PyO3 knows about)

Because the extension targets the stable ABI, it links against interpreters newer than the PyO3 version was released for. If a build against a very new CPython refuses, set the forward-compatibility flag once in the build shell:

=== "Linux / macOS"

```bash
export PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1
maturin develop --release
```

=== "Windows (PowerShell)"

```powershell
$env:PYO3_USE_ABI3_FORWARD_COMPATIBILITY = "1"
maturin develop --release
```

Verifying the build

python -c "from radixor import Stemmer; print(Stemmer('en').stem('running'))"   # -> run
pytest -q

Notes and caveats

  • Model packaging. Neither runtime distribution contains textual dictionaries. radixor-models-standard ships 20 compiled gzip v7 resources, the checksum/provenance manifest, and CC BY-SA 3.0 notices; optional pl-pl-polimorf is excluded.
  • Catalog compatibility. Radixor 4.1 accepts model-distribution major 1 (>=1.0,<2.0) carrying the independent 2026.1 catalog identity. Missing, incompatible, or corrupt data produces an actionable error before native loading.
  • Toolchain PATH. After installing rustup, open a fresh shell (or ensure ~/.cargo/bin is on PATH) so maturin can find cargo/rustc.