Choosing CRC Polynomials for Instrument Protocols

Picking a CRC width and generator polynomial for an instrument link is a coverage decision, not a coding convenience: the polynomial degree sets the longest burst you can guarantee to catch, and the polynomial’s factorisation sets the Hamming distance you hold at a given frame length. This guide, part of Checksum & CRC Validation in the Data Capture, Validation & Metadata Sync section, explains how to match a polynomial to a protocol’s frame length and error model rather than copying whichever variant the last vendor used.

Selection Scope: Coverage Budget, Not Implementation

The decision here is upstream of writing any CRC loop. Given a link — a Modbus-RTU RS-485 bus, a spectrometer’s binary telemetry, a bespoke sensor frame — you have a frame length in bits, an expected bit-error rate from the physical layer, and a required residual (undetected) error rate the experiment can tolerate. The output is a CRC width r and a specific generator polynomial. The mechanics of running that polynomial byte by byte, table-driven, are covered in Implementing CRC32 validation for sensor data streams; the Modbus-specific parameterisation lives in Validating Modbus RTU CRC-16 Frames in Python. This page answers the prior question: which polynomial, and why.

Three constraints dominate. First, burst length: a CRC of degree r detects every burst error of length r bits or fewer, unconditionally. Second, Hamming distance (HD): the minimum number of bit errors that can go undetected, which depends jointly on the polynomial and the frame length — a polynomial that holds HD=6 for a 64-bit frame may collapse to HD=4 well before a 1500-byte frame. Third, random-error residual: for uniformly random corruption, roughly one frame in 2^r slips through regardless of polynomial. You cannot optimise all three with a single width; you choose the smallest r that satisfies the worst case your link presents.

Why Degree, Factorisation, and Frame Length Interact

A CRC is the remainder of the message polynomial M(x), shifted by the width r, modulo the generator G(x) over GF(2). An error pattern E(x) is undetected exactly when E(x) is divisible by G(x):

$$E(x)\mathrm{mod}G(x)=0⟺\text{corruption is undetected}$$

That single condition explains every guarantee. Any burst of length \le r is a polynomial of degree < r with a nonzero low term; it cannot be a multiple of a degree-r G(x), so it is always caught — the burst guarantee equals the degree. If G(x) contains the factor (x+1), then every E(x) of odd weight is caught, because (x+1) does not divide any odd-weight polynomial; this is why production polynomials are almost always chosen with that factor, giving all-odd-bit-error detection for free. Hamming distance is the subtler property: HD is the smallest error weight for which some E(x) of that weight is a multiple of G(x) within the frame length, and because the count of candidate multiples grows with frame length, HD is a decreasing step function of length. Koopman’s polynomial evaluations tabulate exactly where each polynomial drops from HD=6 to HD=4, and those breakpoints are the real selection criterion.

For an approximately random bit-error channel with bit-error probability p and frame length n bits, the probability that a corrupted frame passes undetected is bounded by the fraction of error patterns that are multiples of G(x):

$$P_{ud}\approx 2^{-r}(1-(1-p{)}^n)\le 2^{-r}$$

The 2^{-r} factor is the ceiling — CRC-16 lets at most one bad frame in 65 536 through; CRC-32 one in ~4.3 billion — and the (1-(1-p)^n) factor scales it by the chance the frame was corrupted at all. This bound is why widening from 16 to 32 bits buys four decimal orders of residual-error headroom, and why a short, clean link rarely needs more than CRC-16.

Coverage matrix: a polynomial’s Hamming distance is a decreasing step function of frame length, so the right choice depends on the longest frame the link carries, not the polynomial’s headline strength.

Polynomial Reference Matrix

The table below is the shortlist for laboratory links. The poly column is the standard bit-mask convention each protocol publishes; refin/refout capture whether input bytes and the output register are bit-reflected, and xorout the final inversion. Reproduce these four parameters plus init exactly and the check value falls out; get any one wrong and every frame is rejected.

Two rows deserve comment. CRC-16/MODBUS and CRC-16/CCITT-FALSE share error-detection strength — both are HD 4 across any realistic frame — but differ entirely in reflection and thus produce different check values; they are not interchangeable, and confusing them is the classic every-frame-rejected fault. CRC-32C holds its Hamming distance out to far longer frames than the classic CRC-32, which is why modern high-throughput loggers prefer it: the Castagnoli polynomial was selected specifically to push the HD=6 breakpoint past several kilobytes, and Intel’s SSE4.2 crc32 instruction computes it in hardware.

A Configurable CrcSpec and Recommender

The implementation encodes the six Rocksoft parameters (width, poly, init, refin, refout, xorout) as a single CrcSpec, drives a bit-serial crc() that reproduces any named variant from those parameters alone, and adds a recommend_crc() helper that maps a frame length and target Hamming distance to the narrowest sufficient width. Keeping the parameters as data, not code, is what lets one function reproduce CRC-8 through CRC-32 without special cases.

Copy
from __future__ import annotations

from dataclasses import dataclass


@dataclass(frozen=True)
class CrcSpec:
    """Rocksoft-model CRC parameters. One instance fully defines a variant."""

    width: int          # polynomial degree in bits (5, 8, 16, 32, ...)
    poly: int           # generator, MSB-first (non-reflected) representation
    init: int           # initial register value
    refin: bool         # reflect each input byte before processing
    refout: bool        # reflect the final register before xorout
    xorout: int         # value XORed into the final register

    def __post_init__(self) -> None:
        if not 1 <= self.width <= 64:
            raise ValueError(f"width {self.width} out of range 1..64")
        mask = (1 << self.width) - 1
        for name in ("poly", "init", "xorout"):
            if getattr(self, name) & ~mask:
                raise ValueError(f"{name}=0x{getattr(self, name):X} exceeds width {self.width}")


def _reflect(value: int, bits: int) -> int:
    """Reverse the low `bits` of `value` (LSB<->MSB)."""
    result = 0
    for _ in range(bits):
        result = (result << 1) | (value & 1)
        value >>= 1
    return result


def crc(data: bytes, spec: CrcSpec) -> int:
    """Compute the CRC of `data` under `spec`, bit-serial and table-free.

    Reproduces any named variant (CRC-8, CRC-16/MODBUS, CRC-32, ...) purely
    from its six parameters. For hot paths, precompute a 256-entry table; this
    form is the reference the table must agree with.
    """
    width, poly, mask = spec.width, spec.poly, (1 << spec.width) - 1
    topbit = 1 << (width - 1)
    reg = spec.init

    for byte in data:
        if spec.refin:
            byte = _reflect(byte, 8)
        reg ^= byte << (width - 8) if width >= 8 else byte >> (8 - width)
        for _ in range(8):
            reg = ((reg << 1) ^ poly) & mask if (reg & topbit) else (reg << 1) & mask

    if spec.refout:
        reg = _reflect(reg, width)
    return (reg ^ spec.xorout) & mask


# Named variants — the reference matrix, as data.
CRC8_SMBUS = CrcSpec(8, 0x07, 0x00, False, False, 0x00)
CRC16_MODBUS = CrcSpec(16, 0x8005, 0xFFFF, True, True, 0x0000)
CRC16_CCITT = CrcSpec(16, 0x1021, 0xFFFF, False, False, 0x0000)
CRC32_IEEE = CrcSpec(32, 0x04C11DB7, 0xFFFFFFFF, True, True, 0xFFFFFFFF)

# Frame-length breakpoints (bytes) below which each width holds the given HD.
# Derived from Koopman's polynomial evaluations for these generators.
_HD_LIMITS: dict[int, dict[int, int]] = {
    8:  {4: 8,    3: 128,  2: 1 << 30},
    16: {6: 15,   4: 4096, 3: 1 << 30},
    32: {8: 32,   6: 256,  4: 4096, 3: 1 << 30},
}


def recommend_crc(frame_len: int, target_hd: int) -> CrcSpec:
    """Return the narrowest standard CrcSpec holding `target_hd` at `frame_len`.

    `frame_len` is the payload length in bytes (checksum excluded). Raises if no
    listed width reaches the requested Hamming distance for that length.
    """
    if frame_len <= 0:
        raise ValueError("frame_len must be positive")
    by_width = {8: CRC8_SMBUS, 16: CRC16_MODBUS, 32: CRC32_IEEE}
    for width in (8, 16, 32):
        limit = _HD_LIMITS[width].get(target_hd)
        if limit is not None and frame_len <= limit:
            return by_width[width]
    raise ValueError(
        f"no listed width holds HD={target_hd} for a {frame_len}-byte frame; "
        "shorten the frame, lower the target HD, or use CRC-32C for long frames"
    )

Validating Against Published Check Values

Every standard variant publishes a check value: the CRC of the ASCII string "123456789". Reproducing it is the single fastest proof that the six parameters are wired correctly — if the check value matches, the polynomial, init, reflection, and xorout are all right, and only framing (byte range, trailer endianness) can still be wrong.

Copy
CHECK = b"123456789"

assert crc(CHECK, CRC32_IEEE) == 0xCBF43926, "CRC-32/IEEE check value"
assert crc(CHECK, CRC16_MODBUS) == 0x4B37, "CRC-16/MODBUS check value"
assert crc(CHECK, CRC16_CCITT) == 0x29B1, "CRC-16/CCITT-FALSE check value"
assert crc(CHECK, CRC8_SMBUS) == 0xF4, "CRC-8/SMBus check value"

# Recommender picks the narrowest sufficient width.
assert recommend_crc(6, target_hd=4) is CRC8_SMBUS       # tiny frame, 8 bits suffice
assert recommend_crc(200, target_hd=4) is CRC16_MODBUS   # medium frame needs 16
assert recommend_crc(200, target_hd=6) is CRC32_IEEE     # HD 6 at 200 B needs 32

# Property: any single-byte burst is always caught (burst <= width bits).
import os
base = os.urandom(64)
good = crc(base, CRC16_MODBUS)
for i in range(len(base)):
    flipped = bytearray(base)
    flipped[i] ^= 0xFF                                    # 8-bit burst < 16-bit width
    assert crc(bytes(flipped), CRC16_MODBUS) != good

Cross-check the bit-serial reference against the standard-library fast path where one exists: zlib.crc32(CHECK) & 0xFFFFFFFF == 0xCBF43926 must equal crc(CHECK, CRC32_IEEE). When they diverge, trust zlib and debug the CrcSpec. Reproducing the Modbus 0x4B37 on real bus traffic is the practical entry point covered in Validating Modbus RTU CRC-16 Frames in Python.

Selection Failure Modes

Reflected versus non-reflected mismatch. CRC-16/MODBUS (refin=refout=true, poly written 0xA001 reflected) and CRC-16/CCITT-FALSE (refin=refout=false, poly 0x1021) are frequently conflated because both are “CRC-16”. Feeding Modbus bytes through the CCITT parameters yields a stable, deterministic, wrong value and rejects every frame. Diagnose by computing the check value both ways: if one branch reproduces 0x4B37 and the other 0x29B1, the mismatch is reflection, not the polynomial.

Wrong init or xorout. A polynomial can be correct while init or xorout is not — CRC-16/CCITT-FALSE (init=0xFFFF) versus CRC-16/XMODEM (init=0x0000) share the 0x1021 polynomial but produce different check values. The symptom is identical to a polynomial error (all frames rejected), so always isolate it by reproducing the published check value before suspecting the wire.

Polynomial too weak for the frame length. A CRC-8 chosen for short status frames silently degrades to HD 2 once someone extends the payload past ~128 bytes — it will still catch all single-bit and burst errors, but two-bit errors begin slipping through at the 2^{-8} residual rate. This failure is invisible until a corrupted reading with a valid checksum appears in the archive. Route the resulting semantic anomaly through Error Code Categorization, and re-run recommend_crc() whenever a frame format grows.

Endianness of the appended CRC. Selection also fixes how the trailer is serialised: Modbus-RTU appends its CRC-16 low-byte-first (little-endian on the wire), while many big-endian instruments send the high byte first. A correct polynomial with a byte-swapped trailer rejects every frame. Confirm the trailer byte order against a captured known-good frame before decoding the payload in Binary & ASCII Format Parsing.

Related

• Checksum & CRC Validation — the verification gate and full algorithm reference matrix that consumes the polynomial chosen here.
• Validating Modbus RTU CRC-16 Frames in Python — the CRC-16/MODBUS parameterisation applied to live RS-485 traffic.
• Implementing CRC32 validation for sensor data streams — running the CRC-32 polynomial table-driven over a reassembled stream.
• Binary & ASCII Format Parsing — trailer byte order and payload decoding after the checksum clears.
• Error Code Categorization — classifying a checksum rejection into a retryable or fatal fault.

← Back to Checksum & CRC Validation

References

• Koopman, “Best CRC Polynomials” — Hamming-distance-versus-length tables for candidate generator polynomials.
• Williams, “A Painless Guide to CRC Error Detection Algorithms” — the Rocksoft parameter model (width/poly/init/refin/refout/xorout).
• Python zlib module documentation — reference CRC-32/IEEE implementation and its unsigned return contract.