Pixel Myth
← Field notes
Security

Log4Shell: the long night and what it left behind

Security field note

Forty-eight hours auditing client dependencies. The ones with an inventory finished on Friday. The rest finished on Tuesday.

The vulnerability was severe and everywhere in the ecosystem. Log4j was in transitive dependencies nobody had chosen, pulled in by other libraries. Your own code could be pristine and perfect. You were still exposed to the risk.

Forty-eight hours of auditing client dependencies and finding the vulnerability. The ones with Software Bill of Materials already in place finished their patches on Friday. Everyone else finished their inventories and patching on Tuesday after a stressful weekend.

That gave us the lesson directly and unmistakably: an SBOM was not compliance theater or audit paperwork. It was an operational necessity, a tool you needed in an incident, not something for quarterly audit reports.

We made dependency inventory automatic: generated and stored on every build that shipped to production. Instant visibility into what third-party code lived in every system. No discovery phase in emergencies. No emergency auditing of transitive dependencies when the next incident happened.

Three incidents followed: other vulnerabilities in other libraries, each dangerous in its own way. Each time, the inventory let us answer immediately: was it in our systems, where, and how exposed were we. The engineering paid for itself.

The lesson: compliance mechanisms that genuinely save engineering time and reduce mean-time-to-response during actual incidents are worth the upfront engineering investment. The ones that only matter for quarterly audits rarely justify what they cost.