Egothor/Radixor
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: prepare Radixor 3.0.0 with contracted tries and compiled patch commands

Browse Source 
Introduce contracted compiled patch tries for faster lookup, make compiled
patch commands the primary runtime path, refresh stemmer benchmarks and
documentation, and restructure the documentation for 3.0.0 onboarding.

BREAKING CHANGE: Radixor 3.0.0 promotes compiled patch-command APIs and
new compiled trie artifacts as the primary runtime integration model.
This commit is contained in:
Leo Galambos
2026-07-03 18:44:39 +02:00
parent df4552b113
commit 38620d7e71
101 changed files with 11235 additions and 727 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/lookup-edge-optimization.md
View File

@@ -6,7 +6,10 @@ Compiled trie nodes (`CompiledNode`) use three lookup strategies when resolving
2. linear scan for very small child counts,
3. binary search over sorted edge labels.
This page explains the dense path, what `maxExpandedIndex` controls, and how to tune it.
This page explains the dense path, what `maxExpandedIndex` controls, and how to tune it. These
edge lookup strategies operate after trie reduction and uniform-subtree contraction. If lookup
reaches an accepting contracted leaf, no child edge search is needed for the remaining input
characters.
## Runtime model of one node
@@ -75,7 +78,7 @@ The value never changes lookup semantics. It only changes the in-memory structur
This threshold is **not** stored in `TrieMetadata`.
- The binary format stores only trie payload and semantic metadata (`reduction`, `traversal`,
case/diacritic settings, and stream version).
case/diacritic settings, contraction settings, and stream version).
- `maxExpandedIndex` is chosen when materializing nodes in memory.
- You can therefore keep one persisted artifact and load it with different in-memory
trade-offs depending on deployment constraints.
@@ -135,15 +138,17 @@ At artifact load time, you can tune the same trade-off independently of persiste
```java
import java.nio.file.Path;
import org.egothor.stemmer.CompiledPatchCommand;
import org.egothor.stemmer.FrequencyTrie;
import org.egothor.stemmer.StemmerPatchTrieLoader;
var defaultLookup = StemmerPatchTrieLoader.loadBinary(
final FrequencyTrie<CompiledPatchCommand> defaultLookup = StemmerPatchTrieLoader.loadBinaryCompiled(
Path.of("stemmers", "english.radixor.gz"));
var fastLookup = StemmerPatchTrieLoader.loadBinary(
final FrequencyTrie<CompiledPatchCommand> fastLookup = StemmerPatchTrieLoader.loadBinaryCompiled(
Path.of("stemmers", "english.radixor.gz"), 1024);
var compactLookup = StemmerPatchTrieLoader.loadBinary(
final FrequencyTrie<CompiledPatchCommand> compactLookup = StemmerPatchTrieLoader.loadBinaryCompiled(
Path.of("stemmers", "english.radixor.gz"), 0);
```