CTO Tech Blog

Wouldn’t it be great if all unsolicited commercial email, or spam as it’s more commonly known, came with a tag in the subject line that identified it as spam?  Not surprisingly, the people who send spam don’t see the value in that idea.  The job of defending networks and users against spam is on the shoulders of network administrators and users.  Understanding what defences are available and how they work will help users and network administrators choose the best solution for their environment.

Spam filtering can be done in any of four points in the path of an email; as it is sent, before it reaches the destination server, on the destination server, or at the end client.  Checking for spam as it is sent seems, on the outside, to be futile since someone intending to broadcast spam would certainly not do it.  In fact, a large portion of spam is sent using malware that is unknowingly installed on end user hosts.  Checking for spam leaving a host would identify that there is a problem which could then be corrected using malware tools.  Companies that host their own mail servers, as well as many hosted servers check email for spam as it arrives or while it is being processed by the mail server.  Filtering spam before it arrives at the server reduces network traffic and isolates the server from malware that may be contained in spam, but often limits the ability for users to allow email that may appear to be spam from people or businesses that are known.  This is called white listing a sender or email domain.  Moving the spam filter to the mail server often adds simpler configuration tools to allow administrators or users to adjust the white list (desired) and blacklist (malicious) settings either on a global or per user basis.  Client based filtering allows users to individually decide how to identify spam and how to act on each message based on the rules that they configure.  Any or all of these types of filtering can be used depending on how much control a network administrator wants to retain or offload to users and most solutions are a hybrid mix.

Regardless of what stage on the email flow a filter is testing for spam, the same criteria can be used.  Most spam filters use a number of tests in a certain order with specific settings for action and logging for each test.  The best filter will allow all desirable mail through and block all spam, but since the spammers are constantly fighting the filters, new rules must constantly be implemented.  Having a subscription with the spam filter vendor allows the filter to be updated with defences against new spamming methods very soon after they are discovered.  Here are some of the more common methods included in spam filters to detect and protect against spam.

Heuristics/Bayesian analysis – These intelligent filters learn and use statistics to determine if a message is spam.

Reputation – Most anti-spam vendors keep databases of known spammers and tag any mail from their domain or IP address as spam.

Phishing – The scan engine looks for links in emails to known phishing sites and tags them.

RBL – Realtime or Relay blacklist – These are third party lists of reported spammers on the Internet.  These blacklists often list IPs and domains that have temporary issues with malware, or are part of a large range of addresses.

Header checking – The scan engine compares the SMTP and MIME email addresses in the message header to make sure they match as well as other header anomalies.  Email clients set the MIME address, while the server sets the SMTP address.  Spammers use this to mask the origin of an email.

Directory harvesting – Emails sent to multiple non-existent accounts in an email domain are marked as spam.  The term directory harvesting is used because by sending large lists of names to a mail server, most will bounce back as failed, but some may not.  Eventually the sender can, by process of elimination, build a list of valid email addresses on the domain.

SPF – Sender Policy Framework – A relatively new DNS record type that defines domain names and hosts that are authorized to send email on a given domain.  Although not widely used initially this method of protecting a domain has become more common.

rDNS – Reverse Domain Name Service – Where DNS takes a domain name and translates it to an IP address, rDNS looks at the IP address and ensures that the domain name in the email header matches the domain name that points to the IP.  This identifies servers that are relaying mail for other, typically unauthorized, domains.

Blacklist – A list on the anti-spam server that is generated automatically or by user intervention that identifies domains or IP addresses that users have reported as spam.

Keyword checking – A list generated automatically or by user intervention or words that are in undesirable emails.  These words could be profanity, pornographic, pharmaceutical, or any other s.

Regardless of which type of spam filtering is used and which tests are implemented, all but the most expensive appliances require configuration and some amount of learning.  Don’t expect a product to stop all spam right out of the box.  With some adjustments and ongoing updates they are great tools to keep users productive.

One of my earliest blogs was about security and I made a point of deterring people from going to web sites other than the big name, well known sites.  I was a little surprised that I didn’t get many emails telling me that I was being overcautious.  I did get one message pointing out that a large part of the value of surfing the net is finding new sites with new information.  I agree.  Is that contradictory?

I’m guilty of doing exactly what I said shouldn’t be done.  When I search for information I frequently click on links to sites that I’ve never been to belonging to companies I’ve never heard of.  I’m not immune to malware, but I do have a few tricks and tools up my sleeve to help make sure I’m protected.  The first and most important tool is education.  I’ve spent a great deal of time since I started this career learning about the methods that hackers use to attack computers.  This knowledge has helped me to develop habits that make me a less likely target for hackers.  While I believe that nothing will completely protect someone from malware and security breaches, I’ll share some tricks and tips that will certainly help.

First of all, any computer connected to the Internet should be fully patched and protected by firewalls.  Yes, that was plural.  Data travels between computers and the Internet in two directions.  Home and small business routers by default block all traffic coming in, but allow all traffic outbound.  They can typically be changed to block all but the necessary outbound traffic, but this requires quite a bit of knowledge and management to implement and maintain.  For outbound traffic I recommend a personal firewall such as the firewall built into recent versions of Microsoft Windows.  This firewall will typically prompt you if a port or program is blocked so that you can consent to allowing access.  If you don’t know what’s asking for access it’s best to say no.  My experience is mainly with Microsoft products, but patching applies to every operating system available today.  For Microsoft Windows, I recommend turning on automatic updates and checking to make sure that updates have been applied at least once a month.   Of course all computers should have up to date antivirus software installed as well.

Once you have this basic protection in place you’re ready to open a web browser.  If you’re searching for information there are many search engines available.  I like Google, but feel that it’s a personal preference, not because of any technological advantage.  When you get your results, look at the URL that is linked.  Most North American domains have a .com, .net, .org or .ca although there are some other new ones gaining popularity.  Phishing and hacking sites are often hosted in countries where law enforcement is not as likely to catch them, so unless you’re looking for something specifically in China, avoid domains ending with .cn for example.  Once you’ve clicked the link, if you see a lot of pop-ups or the page is not what you expected; leave.  Close your browser and any pop-ups.  It may already be too late, but there is a chance you’ve been quick enough to avoid a “drive by download”.

The Internet is a wonderful tool, but like anything popular it attracts people who hope to profit from people who don’t know how to protect themselves.  If you leave your purse on your car seat and your windows down, chances are that it will be stolen.  Basic protection will help avoid the majority of threats.

Email is a wonderful tool for communication, both at work and at home.  It's almost as common today to ask for someone's email address as their phone number.  Unfortunately most, if not all, of us have received some form of spam in our mailboxes as well as our intended communications.

Spam is officially known as unsolicited commercial email or UCE.  The more common term "spam" was reportedly borrowed from a Monty Pithon's Flying Circus comedy sketch of the same name.  The connection presumably being the annoying nature and apparent unavoidability of UCE and the similar use of the popular Hormel Foods canned ham in the sketch.  Unlike the food product, email spam comes in a variety of flavours.  Commonly we see prosthetic and drug advertising emails that use constantly changing techniques to attempt to thwart spam filters.  These messages frequently have return email addresses that are non existent or that have been hijacked from some innocent party (sender spoofing).  I'll walk through some methods of sending spam and discuss some techniques to help reduce the amount of spam you receive.

Since spammers, the people who originate spam email, need to protect themselves from being identified and blocked by intended recipients they have had to find ways to send a large number of emails from various locations in a very short period of time.  The most common method of doing this is to use an open mail relay somewhere on the Internet.  An open relay is a mail server that will forward email from anyone (or anyone who knows how to get around security measures) to any potential recipient on the Internet.   In the early days of spamming the sender would usually create a new email account on a large free mail hosting service such as Yahoo! or AOL and send emails to a large list of recipients.  As the hosting companies clamped down on this type of activity they began to use complex scripts with multiple open relay servers and large lists of recipients.  Today most commercial email providers block relaying, but many smaller companies who host their own email servers often don't have the technical skills required to turn off relaying while still allowing their users to send and receive emails that they need to.  The lists of email addresses that spammers send to are created using various methods.  Some companies sell lists of email addresses gathered from their web site.  Since email is sent unencrypted over the Internet, the addresses can often be gathered by "listening" to traffic between servers.  Sometimes they will even send email to random names in an email domain (a domain is the part after the @ in an email address) and see which ones don't bounce back to them (directory harvesting).

A few methods to help avoid spam are to not enter your email address in web forms or use it to enter draws.  In cases where you need to do this, consider creating a "spam account" with one of the online email providers like Yahoo! or GMail.  Use this account when you're dealing with companies on the web that you don't know.  If your Internet provider offers a spam filtering solution it may be worth subscribing to it.  On a larger network where email is hosted in house, an anti spam appliance or dedicated server software are the best defence as they offload the traffic from the mail server.  Some PC based spam filters work well, but they usually need a subscription to keep up to date so they can continue to work effectively.  Spammers are constantly changing their methods so new defences need to be defined and applied.  Junk mailboxes and blacklists that are built into applications such as Microsoft Outlook also help, but they tend to rely on filtering by the sender and these days the sender is changed with every mailing that goes out.

Just make sure you put my email address on your white list.

The best investment that anyone can make in malware defence is education.  In fact, I would go as far as saying that any investment made in education on this subject will be repaid in software cost savings and productivity increases.  Let me step back and clarify what I mean by malware and the potential costs to individuals and companies.

Malware is defined by Merriam-Webster Online as simply "software designed to interfere with a computer's normal functioning".  While most people would immediately put a computer virus in this category, they may be unaware of other types of malware such as spyware, trojans, root kits, and even adware to name a few.  Each of these types of software share the common trait of performing functions that the user of the computer does not benefit from and often does not know about.  In some cases they are installed on purpose, but perform functions other than or beyond what the installer believes they do.  In others they will be installed by deceiving the user or in some cases even without their knowledge.  The effect of malware on a computer user or company can range from disruptive popup windows and web surfing habit collecting at the low end to theft of personal information and data or destruction of data at the high end.  The cost of these events is difficult to estimate and would differ for each user and the severity of the infection.  In most cases users have antivirus software and firewalls at a minimum and the value of these items are abolished with a single infection of any severity.  The final cost is attributed to remediation.  Stolen data and personal information can be very difficult to retrieve and typically involves changing bank accounts and credit cards.  Lost data and corrupted operating systems can be retrieved and repaired in most cases by people with the right skills, but this is rarely an inexpensive endeavour.

So how can education help defend against this type of incursion?  Developing safer computing habits and knowing the tricks that malware creators use to attack systems can help avoid exposure.  Safe computing habits include not opening email from people you don't know and from whom you are not expecting mail.  If you do open an email, but are not 100 percent sure of why you received it, don't click on any links in the email.  Never send or post personal information of any kind in an email or on a web site.  Remember that your bank will never phone or email to ask you for personal or account information.  Don't visit web sites that you don't know or don't know to be legitimate.  Try to stick to big brand name sites such as Microsoft, Google, Walmart, etc.  Download and install all security related updates for your operating systems, firewalls and antivirus software.  Install and regularly update a reputable antivirus software package. 

Today I have listed some basic strategies that everyone should practice and in future posts I will drill deeper into different types of malware, explain how they attack computers and suggest strategies for defence.  If you'd like to jump ahead and learn more there are some well produced videos available at Watchguard's web site: http://www.watchguard.com/education/videos.asp  I would suggest starting with the security for beginners videos at the bottom of the page.  Watchguard is a manufacturer of security products for computer networks.