We Wuz Hacked

For those of you who might have wondered about “where in the heck did opennms.org go” today, I stopped the server in order to figure out how our site got hacked.

It turns out it was a vulnerability in the old wiki, so it’s now gone. Please let me know if you needed anything off of that and I’ll try to put it back.

I run Debian, and I keep a pretty close eye on things, so as far as I can tell they weren’t able to install a rootkit or anything. All of their programs were owned by the web user.

It was OpenNMS that tipped me off that something was wrong. I got a notice that SMTP was down on opennms.org. I thought that was odd, since I use that for my outgoing mail, and it seemed to be working fine.

I decided to check the state of the server, and the load average was up around 4. Extremely unusual. Top showed a number of processes called “a1tsdos” as using up most of the CPU. A quick Google turned up no matches (one of the reasons I am writing this) and “locate” didn’t find it either, so I knew it must be a new file.

I found it in /var/tmp and it was owned by the web server. I moved it out of the way and killed all a1tsdos instances and stopped the web server. Then I called in the troops.

We found out that a process called “crond” was running:

crond == O Backdoor do OutLaw Group foi Executado com exito

and a1tsdos was described as:

A1TS -DoS Tool by Bim_Laden

The crond “app” had opened a TCP connection from our server to a site in Mexico City (I dropped a line to the ISP but I doubt it will do any good), and using that IP address I was able to track the attack.

I thought it might be due to the new wiki, Bitweaver, so I jumped on the #bitweaver channel. spiderr pointed me to an xmlrpc vulnerability that could have been the problem, so since we don’t use that functionality, I quickly removed it (it’s one of the nice things about BW – easy to remove things you don’t use).

Using the IP address and the web logs:

[12/Jul/2005:12:14:46 -0500] "GET /tiki-index.php HTTP/1.1" 200 25286 "http://www.google.com.mx/search?q=inurl:tiki-*.php&hl=es&lr=&start=20&sa=N" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

I could see that it was the ”old” Wiki that listed the vulnerability. Sure enough:

[12/Jul/2005:12:15:32 -0500] "GET /tiki-xmlrpc_services.php HTTP/1.1" 200 375 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"

There was the xmlrpc code that caused the problem.

I think everything is cool now, but the old wiki is gone. Please let me know if you need anything off of it. Outside of Bitweaver, everything else we use is off of Debian packages, so we should be covered. Sorry for any inconvenience.