ftfy v3.0 Release Notes
Release Date: 2013-08-26 // over 10 years ago-
- Generally runs faster
- Idempotent
- Simplified decoding logic
- Understands more encodings and more kinds of mistakes
- Takes options that enable or disable particular normalization steps
- Long line handling: now the time-consuming step (
fix_text_encoding) will be consistently skipped on long lines, but all other fixes will apply - ✅ Tested on millions of examples from Twitter, ensuring a near-zero rate of false positives