And the moral of the story is don't use the same ssh keys on all servers, and don't use the same password for multiple boxes. -----Original Message----- From: Perry Lorier [mailto:perry(a)coders.net] Sent: Sunday, January 23, 2005 12:01 PM To: Waikato Linux Users Group Cc: WLUG Committee Mailing List Subject: [wlug-committee] Re: [wlug] Hoiho So some more information for those begging for it :) First, some back story... One of the lug members has been busy this week helping cleaning up after a lot of compromised servers. All the people that have been involved with this have had a busy week and I'm glad that it wasn't me having to deal with this :) So, on Saturday I was reading the root mail for hoiho and noticed that this user had used the "sudo" command. Now, he doesn't have access to sudo, so it sent me an email saying that he'd attempted it and it was blocked. Normally I get one of these every few weeks as people accidently type it on hoiho when they expect to be on their own machines, but it was suspicious enough that I phoned him. He claimed that he hadn't logged into hoiho for weeks. Hrm. Curious. So I poked around his home directory, there were a two subdirectories, "root" which contained what appeared to be a rootkit (new versions of programs like modprobe and lsmod being evidant) and a about 30 or so programs which seemed to be used for exploiting services to get root. Fortunately we had done a security upgrade of the software not 12 hours before the attack. The other directory was "a" and had a subdirectory ".access.log" and contained a concealed bnc. The users crontab was running a program to restart the bnc if it was ever killed every minute. That was enough for me, and I shut the machine down immediately and emailed everyone to tell them to change their passwords and to explain why the box was offline. We managed to organise to go and fetch the box. When we got back here with the box we booted it with a livecd (Ubuntu!) and mounted the filesystems ro,noexec,nosuid so we could investigate them. After much looking around we saw that the kiddie got in at about 2:30pm on Friday by logging into a users account via ssh using password authentication. He tried to use "sudo" (which generated the email), and "su" (which had shown up in the daily reports, but I hadn't read when I decided to kill the box). However, he doesn't seem to have got root. None of the exploits he left lying around seem to work on hoiho, and his rootkit binaries (modprobe etc) don't seem to be installed. However, as a precaution we're reinstalling the entire machine anyway and we're using this opertunity to update the software on the machine (including the wiki). So short answer: * Machine was compromised by the kiddie knowing a users password and sshing in directory. * Kiddie attempted to get root, and as far as we can tell, probably failed. * We're reinstalling anyway, and using the opertunity to upgrade the software (including the wiki). Hopefully the machine will be back up again sometime monday. I'd like to thank Jamie Curis, Craig McKenna, John McPhearson, Craig Box and Kyle Carter for their help with sorting out the machine. _______________________________________________ wlug-committee mailing list wlug-committee(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/wlug-committee