Refine stemmer core, compiled trie workflow, tests, and public documentation

Browse Source

feat: implement Compile CLI for building binary stemmer tables from source dictionaries
feat: add loading support for persisted compiled tries, including GZip-compressed binaries
feat: add a builder path for recreating a writable trie from a compiled trie
feat: expose read-only value/count access for compiled trie entries
feat: support deterministic NOOP patch encoding for identical source and target words

fix: make value selection deterministic for equal frequencies using length and lexical tie-breakers
fix: preserve valid alternative reductions during trie optimization and reduction
fix: correct patch command edge cases discovered in round-trip and malformed-input tests
fix: address persistence and compiled-trie handling defects found during implementation review
fix: resolve test failures and behavioral regressions uncovered by PMD and JUnit runs

refactor: reorganize trie-related support types into dedicated packages and classes
refactor: simplify the core FrequencyTrie design toward a cleaner practical architecture
refactor: improve compiled/read-only trie boundaries without restoring mutability
refactor: clean up internal reduction, serialization, and helper structure

test: add professional JUnit coverage for stemmer core classes
test: split trie tests into dedicated test classes per production type
test: improve parameterized tests for readability, diagnostics, and edge-case traceability
test: cover positive, negative, malformed, persistence, and round-trip scenarios
test: verify compiled dictionaries against source inputs using getAll semantics

docs: write public README and supplementary Markdown documentation for project publishing
docs: document architecture, reduction model, built-in languages, and operational guidance
docs: clarify reverse-word storage, mutable construction, and compiled-trie runtime behavior
docs: remove placeholders, vague buzzwords, and unexplained terminology from the documentation
docs: improve examples and wording for professional reader-facing project guidance

chore: align project materials with the practical Radix scope and Egothor/Stempel lineage
chore: raise overall project quality through documentation review and test hardening

...

This commit is contained in:

Leo Galambos

2026-04-13 02:10:46 +02:00

parent 15248c92c9

commit 038514bad0

64 changed files with 190190 additions and 20 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

10353

src/main/resources/da_dk/stemmer Normal file

File diff suppressed because it is too large Load Diff

17059

src/main/resources/de_de/stemmer Normal file

File diff suppressed because it is too large Load Diff

12909

src/main/resources/es_es/stemmer Normal file

File diff suppressed because it is too large Load Diff

7836

src/main/resources/fr_fr/stemmer Normal file

File diff suppressed because it is too large Load Diff

17095

src/main/resources/it_it/stemmer Normal file

File diff suppressed because it is too large Load Diff

21062

src/main/resources/nl_nl/stemmer Normal file

File diff suppressed because it is too large Load Diff

9689

src/main/resources/no_no/stemmer Normal file

File diff suppressed because it is too large Load Diff

15268

src/main/resources/pt_pt/stemmer Normal file

File diff suppressed because it is too large Load Diff

20676

src/main/resources/ru_ru/stemmer Normal file

File diff suppressed because it is too large Load Diff

12962

src/main/resources/sv_se/stemmer Normal file

File diff suppressed because it is too large Load Diff

19495

src/main/resources/us_uk.profi/stemmer Normal file

File diff suppressed because it is too large Load Diff

14760

src/main/resources/us_uk/stemmer Normal file

File diff suppressed because it is too large Load Diff