Back in the year 1999, long before I started my Asterisk days, I spent most of my time as a security consultant and cyber forensics expert. I remember that in those days, most of the hacks were script kiddies exploiting some Windows IIS well known hole, and you would usually get the “Hacked by Chinese” black display on your website – how annoying!
In any case, I’ve recently replaced my co-location firewall. I’ve migrated from a Linux system running IPtables with a manual script, to a fully blown IPCOP installation. Ok, so IPCOP is nothing more than a fancy GUI for IPtables, but hey, it makes my life a whole lot easier on the management side – and it’s very stable – so who am I to complain?
I’ve decided to run a small experiment, I wanted to setup a Linux box, with a root password of 123456. My question was this, how much time will pass from the moment the machine was up, on a new IP address, till the machine gets hacked – and more importantly, from where and what got installed on the machine?
So, the machine fired up for the first time at Fri Jul 25 23:19, believe it or not, the machine got hacked at Sat Jul 26 00:50. A mere 90 minutes into the air, and the machine got hacked. The funny thing was that at Sat Jul 26 03:09 it got hacked again to the same account, then at Sat Jul 26 03:21, which also closed the root access via SSH at this point. Following below is the last log:
root pts/0 77.127.137.52 Sat Jul 26 06:04 still logged in reboot system boot 2.6.18-53.1.14.e Sat Jul 26 06:02 (00:17) root pts/1 92.80.195.126 Sat Jul 26 03:21 - 03:24 (00:03) root pts/0 78.110.163.31 Sat Jul 26 03:09 - 05:20 (02:11) root pts/1 60.220.240.7 Sat Jul 26 00:50 - 00:50 (00:00) root pts/0 77.127.137.52 Fri Jul 25 23:24 - 01:39 (02:14) root tty1 Fri Jul 25 23:22 - 23:24 (00:01) reboot system boot 2.6.18-53.1.14.e Fri Jul 25 23:19 (07:00) root tty1 Fri Jul 25 22:14 - down (01:03) reboot system boot 2.6.18-53.1.14.e Fri Jul 25 21:58 (01:19)
I admit it, putting a machine on the open net, with a root password of 123456 and open root access to SSH – that’s kind of a honey pot the size of the grand canyon. But what amazed me here was not the speed, but actually the locations of the hacks: 60.220.240.7, 78.110.163.31 and 92.80.195.126. One hacker is in China, the other in Romania and the third in the UK. What is this? a real hacker? maybe 3 different robots scanning? – I can’t really tell here. However, the traces they left were interesting enough – which lead me to believe we’re talking about robot hacking.
First off, a look at /var/log/audit/audit.log immediately showed the logins – the hacker didn’t even remove the log file – marking of a script kiddie running an automated script. So, what did they leave on my box, let’s take a look. Running ‘netstat -apn | less’ would show me open ports, unless netstat was replaced. However, lets start with this:
tcp 0 0 172.31.31.16:34183 195.47.220.2:6667 ESTABLISHED 2940/crond
tcp 0 1 172.31.31.16:57263 195.54.102.4:6667 SYN_SENT 2940/crond
tcp 0 1 172.31.31.16:46043 195.68.221.221:6667 SYN_SENT 2940/crond
Ok, so this is most probably an IRC bot waiting for instructions from the hacker – till now nothing special. The script tries to masquerade the bot with a legitimate process name: crond. Well, that may fool a beginner Linux Sysadmin, however, seeing crond connecting to 3 other hosts at TCP 6667 – ok, that’s kind’a lame – no?
I wonder where he hid the script? maybe he replaced crond?
root@pbx:~ $ find / -name "crond" /usr/sbin/crond /var/tmp/.www/crond /var/lock/subsys/crond /etc/sysconfig/crond /etc/rc.d/init.d/crond /etc/pam.d/crond root@pbx:~ $
Hmm… /var/tmp/.www/crond looks promising, let’s see what’s in there:
root@pbx:~ $ ls -la /var/tmp/ total 24 drwxrwxrwt 4 root root 4096 Jul 26 2008 . drwxr-xr-x 25 root root 4096 Jul 25 2008 .. drwxr-xr-x 2 root root 4096 Jun 27 17:03 .spd drwxr-xr-x 4 501 502 4096 Jul 26 2008 .www
Yummy! Let’s check it out:
root@pbx:/var/tmp $ ll .spd/
total 1316
-rwxr-xr-x 1 root root 265 Nov 19 2005 gen-pass.sh
-rwxr-xr-x 1 root root 72 Jun 26 19:43 pass_file
-rwxr-xr-x 1 root root 21407 Nov 19 2005 pscan2
-rwxr-xr-x 1 root root 218 Jun 27 16:59 s
-rwxr-xr-x 1 root root 453972 Nov 19 2005 ss
-rwxr-xr-x 1 root root 842736 Jun 26 19:20 ssh-scan
-rwxr-xr-x 1 root root 312 Jun 27 17:02 x
root@pbx:/var/tmp $ ll .www/
total 888
-rwxr-xr-x 1 501 502 353 Jul 26 2008 1.user
-rwxr-xr-x 1 501 502 349 Jul 26 2008 2.user
-rwxr-xr-x 1 501 502 353 Mar 14 2009 3.user
-rwxr-xr-x 1 501 502 317 Nov 6 2007 autorun
-rw-r--r-- 1 root root 0 Jul 26 2008 belgian.seen
-rwxr-xr-x 1 501 502 942 May 15 2003 checkmech
-rwxr-xr-x 1 501 502 23237 May 15 2003 configure
-rwxr-xr-x 1 501 502 492135 Mar 4 2005 crond
-rwxr-xr-x 1 501 502 48 Jul 26 2008 cron.d
-rwxr-xr-x 1 501 502 171 Jul 26 2008 cutitas
-rwxr-xr-x 1 501 502 4147 May 15 2003 genuser
-rwxr-xr-x 1 501 502 157 Jul 25 17:36 LinkEvents
-rwxr-xr-x 1 501 502 0 Oct 15 2007 lucifer.seen
-rwxr-xr-x 1 501 502 2154 May 15 2003 Makefile
-rwxr-xr-x 1 501 502 14 Jul 26 2008 m.dir
-rwxr-xr-x 1 501 502 22882 May 15 2003 m.help
-rwxr-xr-x 1 501 502 748 May 15 2003 mkindex
-rwxr-xr-x 1 501 502 1043 Jul 26 2008 m.lev
-rwxr-xr-x 1 501 502 5 Jul 25 17:35 m.pid
-rwxr-xr-x 1 501 502 1068 Jul 26 2008 m.ses
-rwxr-xr-x 1 501 502 1675 Mar 25 2009 m.set
-rwxr-xr-x 1 501 502 167964 Mar 16 2001 pico
-rwxr-xr-x 1 501 502 84476 Jun 23 2006 pico.tgz
drwxr-xr-x 2 501 502 4096 Jul 23 15:48 r
-rwxr-xr-x 1 501 502 661 Jul 12 22:00 shadow}{700.seen
-rwxr-xr-x 1 501 502 661 Jul 12 22:00 shadow}{800.seen
-rwxr-xr-x 1 501 502 715 Jul 12 22:00 shadow}{900.seen
drwxr-xr-x 2 501 502 4096 Jul 23 15:51 src
-rw-r--r-- 1 root root 1842 Jul 26 2008 zak.seen
Looks like .spd is the SSH scanner and the .www directory contains the actual bot binary – ok, I can respect that. The contents of the cron.d file suggested that the script utilizes crontab to verify that the bot is always up and running – and examination of its code assured me of that.
So, what have we learned from the above: just one thing! When installing a server for the first time, DON’T USE A SILLY PASSWORD LIKE 123456 – EVEN NOT FOR THE INSTALLATION PHASE! Scanning robots appear to be scanning the entire Internet over and over and over again, doing so in seconds – so by the time you install your server, set it up completely, there is a good chance it will already be compromised.