BitPage

Incidents

Post-mortems with the full diagnosis: hypotheses, ruled-out theories, root cause, real numbers. How systems actually break.

2 articles

Mass segfaults on a ZFS host: not bad RAM, an unsigned underflow in zfs_fillpage()

A host that answers on no port while the kernel keeps logging is not failing memory. In OpenZFS before 2.4.4 and 2.3.9, zfs_fillpage() computes io_len = i_size - io_off in unsigned types: race an mmap read against an in-place truncate and the length underflows, so dmu_read() zero-fills physical memory far past the page. Fixed on 21 August 2026 in 2.4.4 and 2.3.9 — but not in 2.2.11, released the same minute.

Gmail bounced everything the relay sent: three broken auth mechanisms and a silent OpenDKIM

A relay with more than two years of uptime failed nearly every Gmail delivery because SPF, DKIM and forward-confirmed reverse DNS were broken independently. The root cause of the DKIM half was one word: tables mounted as refile: instead of file:, which makes OpenDKIM drop the key lookup and sign nothing without logging a line.