All Versions
7
Latest Version
Avg Release Cycle
-
Latest Release
-

Changelog History

  • v0.5.0 Changes

    • faster, more efficient code
    • ⬇️ dropped support for Python 3.5

  • v0.4.0 Changes

    • 🆕 new languages: Armenian, Greek, Macedonian, Norwegian (Bokmål), and Polish
    • language data reviewed for: Dutch, Finnish, German, Hungarian, Latin, Russian, and Swedish
    • 🚚 Urdu removed of language list due to issues with the data
    • ➕ add support for Python 3.10 and drop support for Python 3.4
    • 👌 improved decomposition and tokenization algorithms

  • v0.3.0 Changes

    • 👌 improved models and disambiguation
    • 👌 improved tokenization
    • extended rules for German

  • v0.2.2 Changes

    • Work on decomposition rules
    • Reviewed language data
    • Cleaner code

  • v0.2.1 Changes

    • 👍 Better decomposition into subwords by greedy algorithm
    • First benchmarks and data-based corrections: German, French, English, Spanish

  • v0.2.0 Changes

    • Languages added: Danish, Dutch, Finnish, Georgian, Indonesian, Latin, Latvian, Lithuanian, Luxembourgish, Turkish, Urdu
    • 👌 Improved word pair coverage
    • Tokenization functions added
    • Limit greediness and range of potential candidates

  • v0.1.0 Changes

    • 🚀 First release on PyPI