System Bugs Under Load, or How We Tried to Track Down a Bug in the OS Kernel

When a production database for a large and critical application crashes with a segmentation fault, the only thing that usually comes to mind is blaming developers for sloppy work. Then it crashes again. And again. But each time it fails in different places, under different circumstances, with no clear clues.

This is exactly what happened to Postgres Professional’s clients—and that’s where a year-long journey to find the root cause began.

We’ll talk about the issues we encountered, how we learned to reproduce the defect, the rookie mistakes we made along the way, and how it all ended (or didn’t).