File Signature Magic Numbers: Why Extensions Lie and Headers Don't

Every file on disk starts with a header, and for most binary formats that header begins with a fixed byte sequence called the magic number (or file signature). Rename invoice.pdf to invoice.txt and the first four bytes still read 25 50 44 46 — %PDF. The extension is a label for humans; the signature is what software actually reads.

This guide covers the signatures every developer should recognize, how to inspect them, and how to use them to validate uploads.

Why Extensions Lie

An extension is just part of the file name. Anyone — or any malware — can change it with one rename:

• malware.exe becomes report.pdf
• a .jpg upload is actually a server-side script
• a "docx" is a ZIP bomb in disguise

If your upload endpoint trusts file.name or the Content-Type header the browser sent, you are trusting fully client-controlled data. Magic numbers can be forged too — but an attacker has to craft actual bytes, which filters out mislabeled files and the laziest attacks.

Common File Signatures

Notice that not every signature sits at offset 0. TAR puts ustar at byte 257. MP4's ftyp starts at byte 4. RIFF containers (WAV, AVI, WebP) open with RIFF but need the format tag at byte 8 to tell them apart.

Inspecting Signatures Yourself

The classic tools:

file --mime-type sample.pdf
xxd sample.png | head -2

file (libmagic) knows thousands of signatures; xxd gives you raw hex to check by eye. In Python:

with open("sample.jpg", "rb") as f:
    head = f.read(16)
print(head.hex())
# ffd8ffe000104a46494600...

Validating Uploads by Signature

A minimal detector in Python:

SIGNATURES = {
    b"\x89PNG\r\n\x1a\n": "png",
    b"\xff\xd8\xff": "jpg",
    b"GIF8": "gif",
    b"%PDF-": "pdf",
    b"PK\x03\x04": "zip",
    b"\x1f\x8b": "gz",
}

def detect(path):
    with open(path, "rb") as f:
        head = f.read(8)
    for sig, fmt in SIGNATURES.items():
        if head.startswith(sig):
            return fmt
    return "unknown"

In JavaScript, the npm package file-type does the same from a Buffer; in Java, Apache Tika is the standard. Whichever you pick, run the check server-side, before the file reaches storage.

Gotchas That Bite People

• ZIP is everywhere. DOCX, XLSX, PPTX, JAR and APK files are all ZIP archives — every one starts with 50 4B 03 04. The magic number alone cannot tell an Office document from any other archive; you have to inspect the contents for word/, xl/, or [Content_Types].xml.
• Legacy Office collides. Old .doc, .xls and .ppt share the OLE2 signature D0 CF 11 E0 A1 B1 1A E1. Distinguish them by their internal stream names, not the header.
• MP3 has two faces. A file may start with an ID3 tag or jump straight to a frame sync such as FF FB. Check both paths.
• Text formats have no signature. TXT, CSV, JSON and Markdown are just bytes — only heuristics or a full parse can validate them. A UTF-8 BOM (EF BB BF) is the closest thing to a signature, and it is optional.
• Polyglots exist. A crafted file can be valid as two formats at once. Signature checks stop accidents and lazy attacks, not determined attackers — layer them with size limits, extension allowlists, and real parsers.

Testing Your Detection Logic

Signature validation deserves the same test data as any other code path:

• Correct files: a real PNG sample, JPG sample, or PDF sample
• Mismatched extensions: rename each one and assert the detector still returns the true type
• Archive edge cases: an empty ZIP archive contains only the end-of-central-directory record, so its first bytes are 50 4B 05 06 instead of 50 4B 03 04 — decide how your validator handles that before your users find out

Magic numbers are the cheapest ground truth in file handling: four to eight bytes that no rename can fake. Trust them before you trust a file name.