Analyzing Cisco Pix firewall logs with FireGen Log Analyzer - A firewall administrator diary
Send your comments!

All the notes on this page are derived from our daily analysis of our Cisco Pix firewall logs using FireGen for Pix Log Analyzer, Version 2.0. To cross-reference the report sections mentioned in these articles, see the sample report.

March 31, 2004
The "Email (SMTP) Top outbound" section indicated a large number of connections from our Exchange server to IP 63.228.132.187. There was no name associated to this IP (no reverse lookup). Since our Exchange tried to SMTP to it, we telnet-ed to it on port 25 in order to find what is the response. The server reported with a qmail banner (saying is is godmail.com) but in the same time, the qmail refused any SMTP commands saying that "there is insufficient disk space". So most probably this is a Linux server with a full partition (read "poorly maintained"). Checking the SMTP queue on our Exchange server we found the message intended for the godmail.com, a NDR for same spam email. So one can basically cause a DoS (as it was the case here with the godmail.com qmail server) just by sending some spam out... The SMTP RFCs do not allow an SMTP server to discard emails for non-existing users without generating an NDR.
In the "Denied connections" section we noticed (with some satisfaction) several attempts from 216.5.163.55 to connect with SMTP - this is the spam server we detected with high traffic in one of the previous days. On the other hand, since the server tried several times, it basically generated more traffic than it would've have if it simply delivered same spam (caught by our antispam gateway). So the question is, what is worse, the spam or the extra traffic? However, the "extra traffic" affected the spammer as well, and since these are high-volume server, they are more sensible to extra load so blocking it it is worse for the spammer. Actually, this is one proposed method of discouraging spammers - have the sending email server solve a "puzzle" before accepting emails from it - this "puzzle" is actually a small application that takes some CPU from the sending server. For "normal" servers, this would not pose any problems - but for servers sending millions of emails per day, this would mean the CPU at 100% most of the time, requiring the spammers to buy more hardware, making their "enterprise" more costly.  We will keep denying this server to see when it will "give up". In the mean time, we will add more spammers on the access-list that denies their access.

March 30, 2004
The report listed again several "Deny IP spoof from (127.0.0.1)" messages (%PIX-2-106016), the target being 2 public IP addresses that we advertise through our Pix firewall (for our email and support web site). We ran again the report but having "Deny IP spoof from" as include keyword and we selected to view only the messages matching it. We did this in order to see if there is a time pattern for these messages. The "Messages matching include keywords" section revealed 12 such messages, all between 22:00 and 03:00 EST. In the same time, there was not such message from our second firewall, with an external IP address in the same range. Now, the second firewall is not redirecting traffic to an internal host (no "static" statements) and in the same time, it is running a different version of PIX OS. From all these we could extract the following conclusions:
1. These messages are not caused by the traffic from internal hosts as these are the hours least active
2. They are not caused by a worm as a worm wouldn't be active just within certain hours
3. If it is a hacker/script kiddie, they could be anywhere - the hours could be the "normal hacking hours"  for practically any region in the world
4. It could be some "residual" network traffic from the ISP (i.e. some regular scans, done a wee hours in order to minimize the impact on the network)
5. Newsgroup posts suggest that this might be some sort of denial of service attack. I do not think that this is the case - what would be the purpose of sending 10-12 such packets every day between 10 pm and 3 am? It would be great if all DoS attacks would be like this!
6. They may be a "false alarm" from the Pix firewall  - a bug in the OS?
7. May be caused by the DSL modems or other networking hardware on the external interface of the firewall

We believe that the cause is one of the points 4, 6 or 7. We will configure a redirection for the second firewall and this will take care of point 6. The results will be posted in the next days.

March 29, 2004
The "Email (SMTP) incoming connections" section revealed that one of the IP addresses that we monitored as being a potential high-volume spam server, made again the top 5. We have configured an access-list in our firewall so hosts like that are not being allowed to make any connection to our email servers. We are keeping the IP address on our "monitored ip addresses" list just to see how many times it will try to connect.
The "Denied protocols" section indicated an increased activity on scanning for Trojans, particularly, Subseven and Netbus. In the "Other protocols" section we noticed an internal PC that is synchronizing the time with ntp2.usno.navy.mil instead of the domain controller. We asked the help desk to look into it.

Pix Analysis Archives:
March 25 - March 26 2004
March 20 - March 23 2004
March 12 - March 19 2004
March 8 - March 11 2004
   

Featured Links GFI EventsManager - Network-wide event log management - Download free 30-day trial! Free Online Event Scanner - Scan your pc for high security events with GFI's free online service. EventID.Net Subscription - So much information for so little!

Legal - EventID.Net © 2001-2008 Altair Technologies Ltd., All rights reserved - Sign up for our Email Newsletter