
We have reason to believe that the hoiho server (www.wlug.org.nz) may have been compromised. The machine has been shutdown pending an audit and probably a reinstall (to be on the safe side). The kiddie suspected of targetting this machine is well known for using password sniffers and other attacks to leverage accounts on other machines. If you have an account on hoiho (including a mail account) then I suggest that you change your password on any related machine immediately. Hopefully the kiddie hasn't successfully compromised the machine, but we're not taking any chances. The wiki will be offline until further notice. Sorry for any inconvenience, Perry Lorier

So some more information for those begging for it :) First, some back story... One of the lug members has been busy this week helping cleaning up after a lot of compromised servers. All the people that have been involved with this have had a busy week and I'm glad that it wasn't me having to deal with this :) So, on Saturday I was reading the root mail for hoiho and noticed that this user had used the "sudo" command. Now, he doesn't have access to sudo, so it sent me an email saying that he'd attempted it and it was blocked. Normally I get one of these every few weeks as people accidently type it on hoiho when they expect to be on their own machines, but it was suspicious enough that I phoned him. He claimed that he hadn't logged into hoiho for weeks. Hrm. Curious. So I poked around his home directory, there were a two subdirectories, "root" which contained what appeared to be a rootkit (new versions of programs like modprobe and lsmod being evidant) and a about 30 or so programs which seemed to be used for exploiting services to get root. Fortunately we had done a security upgrade of the software not 12 hours before the attack. The other directory was "a" and had a subdirectory ".access.log" and contained a concealed bnc. The users crontab was running a program to restart the bnc if it was ever killed every minute. That was enough for me, and I shut the machine down immediately and emailed everyone to tell them to change their passwords and to explain why the box was offline. We managed to organise to go and fetch the box. When we got back here with the box we booted it with a livecd (Ubuntu!) and mounted the filesystems ro,noexec,nosuid so we could investigate them. After much looking around we saw that the kiddie got in at about 2:30pm on Friday by logging into a users account via ssh using password authentication. He tried to use "sudo" (which generated the email), and "su" (which had shown up in the daily reports, but I hadn't read when I decided to kill the box). However, he doesn't seem to have got root. None of the exploits he left lying around seem to work on hoiho, and his rootkit binaries (modprobe etc) don't seem to be installed. However, as a precaution we're reinstalling the entire machine anyway and we're using this opertunity to update the software on the machine (including the wiki). So short answer: * Machine was compromised by the kiddie knowing a users password and sshing in directory. * Kiddie attempted to get root, and as far as we can tell, probably failed. * We're reinstalling anyway, and using the opertunity to upgrade the software (including the wiki). Hopefully the machine will be back up again sometime monday. I'd like to thank Jamie Curis, Craig McKenna, John McPhearson, Craig Box and Kyle Carter for their help with sorting out the machine.

And the moral of the story is don't use the same ssh keys on all servers, and don't use the same password for multiple boxes. -----Original Message----- From: Perry Lorier [mailto:perry(a)coders.net] Sent: Sunday, January 23, 2005 12:01 PM To: Waikato Linux Users Group Cc: WLUG Committee Mailing List Subject: [wlug-committee] Re: [wlug] Hoiho So some more information for those begging for it :) First, some back story... One of the lug members has been busy this week helping cleaning up after a lot of compromised servers. All the people that have been involved with this have had a busy week and I'm glad that it wasn't me having to deal with this :) So, on Saturday I was reading the root mail for hoiho and noticed that this user had used the "sudo" command. Now, he doesn't have access to sudo, so it sent me an email saying that he'd attempted it and it was blocked. Normally I get one of these every few weeks as people accidently type it on hoiho when they expect to be on their own machines, but it was suspicious enough that I phoned him. He claimed that he hadn't logged into hoiho for weeks. Hrm. Curious. So I poked around his home directory, there were a two subdirectories, "root" which contained what appeared to be a rootkit (new versions of programs like modprobe and lsmod being evidant) and a about 30 or so programs which seemed to be used for exploiting services to get root. Fortunately we had done a security upgrade of the software not 12 hours before the attack. The other directory was "a" and had a subdirectory ".access.log" and contained a concealed bnc. The users crontab was running a program to restart the bnc if it was ever killed every minute. That was enough for me, and I shut the machine down immediately and emailed everyone to tell them to change their passwords and to explain why the box was offline. We managed to organise to go and fetch the box. When we got back here with the box we booted it with a livecd (Ubuntu!) and mounted the filesystems ro,noexec,nosuid so we could investigate them. After much looking around we saw that the kiddie got in at about 2:30pm on Friday by logging into a users account via ssh using password authentication. He tried to use "sudo" (which generated the email), and "su" (which had shown up in the daily reports, but I hadn't read when I decided to kill the box). However, he doesn't seem to have got root. None of the exploits he left lying around seem to work on hoiho, and his rootkit binaries (modprobe etc) don't seem to be installed. However, as a precaution we're reinstalling the entire machine anyway and we're using this opertunity to update the software on the machine (including the wiki). So short answer: * Machine was compromised by the kiddie knowing a users password and sshing in directory. * Kiddie attempted to get root, and as far as we can tell, probably failed. * We're reinstalling anyway, and using the opertunity to upgrade the software (including the wiki). Hopefully the machine will be back up again sometime monday. I'd like to thank Jamie Curis, Craig McKenna, John McPhearson, Craig Box and Kyle Carter for their help with sorting out the machine. _______________________________________________ wlug-committee mailing list wlug-committee(a)list.waikato.ac.nz http://list.waikato.ac.nz/mailman/listinfo/wlug-committee

Yes, I see those 3B1 notebooks are on special at the Warehouse Stationary currently for 5 cents each... Oh well, I guess that's my 5 cents worth. Kyle Carter wrote:
And the moral of the story is don't use the same ssh keys on all servers, and don't use the same password for multiple boxes.

The question is, how much RAM does Steven Tindall put in them??
Yes, I see those 3B1 notebooks are on special at the Warehouse Stationary currently for 5 cents each...
Oh well, I guess that's my 5 cents worth.
Kyle Carter wrote:

The compromised user didn't happen to be zcat or mark? I had a script kiddie bashing against hostility briefly a couple weeks ago trying to log in as "zcat" and "mark". From a Romanian host no less. Regards On Sun, 2005-01-23 at 12:01 +1300, Perry Lorier wrote:
So some more information for those begging for it :)
First, some back story...
One of the lug members has been busy this week helping cleaning up after a lot of compromised servers. All the people that have been involved with this have had a busy week and I'm glad that it wasn't me having to deal with this :)
-- Oliver Jones <oliver(a)deeper.co.nz> Deeper Design Limited

Perry Lorier wrote:
So some more information for those begging for it :)
I'd like to thank Jamie Curis, Craig McKenna, John McPhearson, Craig Box and Kyle Carter for their help with sorting out the machine.
Thank you for all your hard work. cheers stuart -- Stuart Yeates stuart.yeates(a)computing-services.oxford.ac.uk OSS Watch http://www.oss-watch.ac.uk/ Humbul Humanities Hub http://www.humbul.ac.uk/
participants (6)
-
Judy & Lindsay Roberts
-
Kyle Carter
-
Lindsay Druett
-
Oliver Jones
-
Perry Lorier
-
Stuart Yeates