Takeaway: Equifax got hacked because it sat on a 10/10 severity vulnerability for 2 months. Update your dang packages sooner!
If you've been following the news at all, you've probably heard of the Equifax hack. Some 145 million credit histories were stolen and now our social security numbers are basically public knowledge. Sweet.
What actually happened? Did some scary persons in black masks sitting around a table decide one day that they'd hack Equifax, and start typing furiously into computers until by some black magic they were in?
What actually happened is a whole lot less exciting.
On March 8, this 10/10 CVSS severity security vulnerability was announced in Apache Struts, a Java web framework used by a lot of large companies. You can find a nice writeup of the details of the hack by Nick Biasini over at Talos Intelligence. In short, you could put code in the Content-Type header of an HTTP request the system would execute it (!?). Note to self: don't use eval.
According to former Equifax CEO Richard F. Smith in his testimony before congress, the Department of Homeland Security told Equifax about the hack shortly thereafter. Apparently there was a policy that something like this had to be fixed within 48 hours. That is good!
The problem is that this memo got buried somehow.
And Equifax sat with an open security hole for 2 months.
When a security hole like this is publicly announced, companies now know they need to fix the problem, but hackers also know about the vulnerability. So from there it is a race against the clock. Will you update your software before hackers find you?
Apache Struts is used by a lot of high profile companies, and hackers began looking for running installs of this outdated version of Struts immediately. And it was only a matter of time before they found Equifax. Equifax was lucky - some companies get hacked way sooner than 2 months!
Mr. Smith ultimately blamed one unnamed IT person for dropping the ball. This is after boasting about his team of 225 security experts and billions of spending on security. If 225 security experts and billions of dollars can't apply a single security patch for two months, I think there is a bigger problem than just one person.
Humans make mistakes, and if a company of Equifax's size can be brought down by the mistake of a single human, that is a serious culture problem. And culture starts at the top. If a single person can break everything, let's change the system instead of casting blame.
In the book Antifragile, Nassim Taleb, discusses how humans have for centuries tried to get rid of problems by moralizing them. Greed in business can be combated by shaming people into not being greedy. Except this has never worked and never will as long as we are still humans. The solution, Taleb says, is to build systems which are not vulnerable to greed.
So it is with security. We need a method of doing security that is not vulnerable to a single human's mistake.
So what can we learn from this? How could we change the culture to prevent ball droppery like this in the future? How can you safeguard your SaaS app?
Equifax is a massive company of some 10,000 employees. Companies that large are going to have some measure of bureaucracy. Your company is not nearly as large, and as such you can move a lot faster. So congrats on not being too large!
Security problems need to be noticed and appropriately responded to relative to their severity. An easy remote code execution exploit is one of the most dangerous bugs possible. At the very least, monitor the announcements of the frameworks you use in your app.
But Equifax knew about the problem. It wasn't a matter of information there, it was a matter of visibility. I'm envisioning a system where anyone can announce a security problem in your app and assign a danger level to it, and the system loudly complains until the problem is fixed. What if Equifax had a TV dashboard in their IT department with a flashing RED ALERT warning until the problem was fixed? Security can easily go undetected until you get hacked, and by then it's too late.