# chardet: A Fast and Accurate Python Character Encoding Detector

This repository profile is provided by osrepos.com, an open source repository discovery platform.

Source: osrepos.com
Repository profile: https://osrepos.com/repo/chardet-chardet
Generated for open source discovery and AI-assisted research.

chardet is a powerful Python library designed for universal character encoding detection. The recently rewritten version 7 offers significant improvements in accuracy and speed, making it a robust solution for identifying text encodings. It provides features like language and MIME type detection, along with a flexible API for various use cases.

GitHub: https://github.com/chardet/chardet
OSRepos URL: https://osrepos.com/repo/chardet-chardet

## Summary

chardet is a powerful Python library designed for universal character encoding detection. The recently rewritten version 7 offers significant improvements in accuracy and speed, making it a robust solution for identifying text encodings. It provides features like language and MIME type detection, along with a flexible API for various use cases.

## Topics

- Python
- character encoding
- encoding detection
- text processing
- open source
- utility
- library

## Repository Information

Last analyzed by OSRepos: Sun Aug 02 2026 01:19:56 GMT+0100 (Western European Summer Time)
Detail views: 0
GitHub clicks: 0

## Safety Notice

OSRepos shares public repositories for knowledge and discovery only. Review source code, dependencies, licenses, and security implications before running or installing anything.

## Content

## Introduction

The `chardet` library is a powerful Python tool for automatically detecting character encodings. It's essential when dealing with text data from unknown sources, preventing mojibake and ensuring proper display and processing. The latest iteration, `chardet` version 7, represents a complete rewrite, delivering unparalleled performance and accuracy compared to its predecessors and alternatives.

## Installation

To install `chardet`, use pip:

bash
pip install chardet


## Examples

### Quick Start

python
import chardet

chardet.detect(b"Python is a great programming language for beginners and experts alike.")
# {'encoding': 'ascii', 'confidence': 1.0, 'language': 'en', 'mime_type': 'text/plain'}

# UTF-8 English with accented characters
chardet.detect("The naïve approach doesn't always work in complex systems.".encode("utf-8"))
# {'encoding': 'utf-8', 'confidence': 0.84, 'language': 'en', 'mime_type': 'text/plain'}

# Japanese EUC-JP
chardet.detect("????????????????????????EUC-JP????????????????????????????".encode("euc-jp"))
# {'encoding': 'EUC-JP', 'confidence': 1.0, 'language': 'ja', 'mime_type': 'text/plain'}

# Get all candidate encodings ranked by confidence
text = "Le café est une boisson très populaire en France et dans le monde entier."
results = chardet.detect_all(text.encode("windows-1252"))
for r in results[:4]:
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1252 0.32
# iso8859-15 0.32
# ISO-8859-1 0.32
# MacRoman 0.31


### Streaming Detection

For large files or network streams, use `UniversalDetector` to feed data incrementally:

python
from chardet import UniversalDetector

detector = UniversalDetector()
with open("unknown.txt", "rb") as f:
    for line in f:
        detector.feed(line)
        if detector.done:
            break
result = detector.close()
print(result)


### Encoding Era Filtering

Restrict detection to specific encoding eras to reduce false positives:

python
from chardet import detect_all
from chardet.enums import EncodingEra

data = "?????? ???????? ???????? ?????????? ????????? ? ?????????? ??????? ??????.".encode("windows-1251")

# All encoding eras are considered by default, 4 candidates across eras
for r in detect_all(data):
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1251 0.46
# MacCyrillic 0.42
# KZ1048 0.2
# ptcp154 0.2

# Restrict to modern web encodings, 1 confident result
for r in detect_all(data, encoding_era=EncodingEra.MODERN_WEB):
    print(r["encoding"], round(r["confidence"], 2))
# Windows-1251 0.46


### Encoding Filters

Restrict detection to specific encodings, or exclude encodings you don't want:

python
# Only consider UTF-8 and Windows-1252
chardet.detect(data, include_encodings=["utf-8", "windows-1252"])

# Consider everything except EBCDIC
chardet.detect(data, exclude_encodings=["cp037", "cp500"])


## Why Use chardet 7?

`chardet` version 7 stands out with remarkable improvements, making it a superior choice for character encoding detection:

*   **Exceptional Accuracy**: Achieves 99.3% accuracy on a diverse test set, significantly outperforming previous versions and other libraries like `charset-normalizer`.
*   **Blazing Fast Performance**: It is 47 times faster than `chardet` 6.0.0 with mypyc, and 1.5 times faster than `charset-normalizer` 3.4.6.
*   **Comprehensive Detection**: Beyond encoding, it offers 95.7% accurate language detection across 49 languages and identifies over 40 binary file formats via MIME type detection.
*   **Flexible Filtering**: Includes parameters like `include_encodings` and `exclude_encodings` to fine-tune detection, along with `encoding_era` filtering for modern web encodings.
*   **Broad Encoding Support**: Supports 99 encodings, including EBCDIC, Mac, DOS, and various European families.
*   **Modern Licensing**: Released under the permissive 0BSD license, a change from the previous LGPL.
*   **Thread-Safe API**: Designed for concurrent calls, scaling well on free-threaded Python.
*   **Familiar API**: Maintains the same public API (e.g., `detect()`, `detect_all()`, `UniversalDetector`, and the `chardetect` CLI) for an easy drop-in replacement.

## Links

*   **GitHub Repository**: [https://github.com/chardet/chardet](https://github.com/chardet/chardet)
*   **Official Documentation**: [https://chardet.readthedocs.io](https://chardet.readthedocs.io)