chardet test data

Test data files for the chardet Universal Encoding Detector. Each file is organized into directories named {encoding} or {encoding}-{language} (e.g., big5-chinese, utf-8-english, cp037-breton).

See CATALOG.md for a full listing of every file's provenance and characteristics.

Data quality

Run scripts/check_test_data.py to verify that all files decode correctly with their labeled encoding and pass quality checks (mojibake, control characters, language/script mismatches):

python3 scripts/check_test_data.py .

Contributing

Contributions of openly-licensed test data are welcome at https://github.com/chardet/chardet.

License

Each test file is copyright its respective publisher.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

chardet test data

Data quality

Contributing

License

About

Releases

Packages

Contributors 10

Languages

Folders and files

Latest commit

History

Repository files navigation

chardet test data

Data quality

Contributing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors 10

Languages