Exploring Malware: Anatomy of an attack

"Be conservative in what you send, be liberal in what you accept."

This is Postel's Law, named after the "God of the Internet". Jon Postel was one of the engineers who build ARPANET, the precursor to the global communication we have today.

Few computer programs adhere to this tenet better than worms, a class of malware whose motives usually come secondary to the core directives of living things: survive and multiply. Worms thrive by abusing known vulnerabilities to infect computing systems, then spread.

We got hit by a memorable worm early in 2019. It was a remote code execution vulnerability affecting Jenkins, which we use to launch new features with every week. We didn’t appear to be directly targeted, the hacker had hit our server by its IP address instead of hostname. The relationship between IPs and hostnames is similar to the relationship between phone numbers and people, where it’s easier to look up someone’s phone number than it is to look up someone by a phone number, without calling first. “Be conservative in what you send.” This attack didn’t target all Java programs. The offending IP only attempted to exploit this vulnerability, and didn’t show security scans or other reconnaissance behavior.

The vulnerability was abused to execute a script on our system. We detected it when it started using all the CPU available to mine cryptocurrency, triggering alerts. It took some effort to deobfuscate what the program intended to do, but once it was refactored & cleaned up it was a brilliant example of fault tolerance and reliability. The worm worked by using Jenkins to download and execute a remote executable. It would try downloading it from a variety of sources, using different tools, and attempt execution with multiple interpreters. All of this redundancy was to try to get one method to work, and only one had to work on a given system. “[Be] liberal in what you accept.” This script didn’t make any assumptions about what tools the system had available to use, it would work on most Linux hosts. As if that wasn’t enough, the script would install itself as a cron, or job that runs regularly on some interval, and start a process to watch and reinstall it as a cron if it was removed. If I removed the cron, the running process would replace it, and if I stopped the process, the cron would replace it.

This script wouldn’t be a worm if it didn’t attempt to spread. This area is one of several where we were saved by dumb luck. This worm exploited a server that we use to make planned updates to most of our web application servers, so it needs access to function. The worm therefore had broad access, but failed because it wasn’t very good at finding servers to log into. It searched certain files for the names of the servers to log into, rather than scanning the network or something similar.

We could have been better protected from an attack like this if we’d had a more layered security approach. Defensive security is built on layers — access, authentication, and authorization, to start. That is, "can you reach this," "can we verify who you are", and "are you allowed to do that". Security is a domain that follows the "Swiss cheese model", where the more layers a potential threat has to make it through, the less likely it is to reach you, since every layer has "holes" or weaknesses. We were lucky this time, and no major damage occurred, but when a security incident like this occurs, you usually aren't so fortunate, so it's important to always think about layered defenses to mitigate possible business catastrophes.