Shabbir’s Blog

Now a days, with millions of psychopaths all siphoning email addresses from everywhere, there is a chronic need for companies (both small and large) to have their own spam filtering services. 

While you can purchase software that does this off the shelf, such as those from MessageLabs, MacAfee, Symantec, Barracuda Networks and others, there’s nothing quite like having your own solution, that you look after, and that - most importantly - doesn’t cost you an arm and a leg.  In fact, this how-to will show you how to build one that costs you no money (unless you hire me to build it for you in which case I will I’m afraid have to charge you - but nothing like what these other guys charge).

Now this is quite a lengthy process, meaning that it will take some time, and is comprised of a number of steps.  We’ll go through each of them in some depth, with plenty of screen shots, so that life is made easy.  As with all my how-to’s any problems, questions, etc just get in touch, and I’ll be more than happy to assist.

So let’s outline the steps that we’ll need to go through.

1. Virtual Server (e.g. VMWare Based) or Physical Server?
2. Operating System Setup
3. Anti-Spam/Anti-Virus Filter Setup Preparation and Information Gathering
4. Actual Build of Spam/Anti-Virus Filter
5. DNS Modification
6. Testing
7. Deployment
8. Securing
9. Updating and Maintaining

So there we have it.  Nine steps, each of which need to be performed, and each of which is important in its own right.  Also, all of these steps have be completed in the order specified above, otherwise you end up with all manner of strange problems.

Since I don’t want this HOWTO be absolutely enormous, and also because my blogging software has size limitations on articles (doh!!), we’ll only cover the first two steps in this post, the second post will continue where this left off - starting at Step 3 on the list above.

So now, without wasting any more bandwidth on my waffle, let’s get started with Part 1 - Virtual Server or Physical Server.

Part 1 - Virtual Server or Physical Server

This is an important consideration, and depends entirely on the size of organisation that your new Spam/Anti-Virus Filter server is going to be catering for.  Personally, it is quite acceptable to use a VMWare Machine, with anything from 512MB to 2GB of RAM (depending on what you can spare, and what you feel is required - the beauty of VMWare is that it will allow you to modify the RAM that a machine has very easily (as long as your VMWare Server has sufficient RAM installed).  Also, it keeps your Spam/Anti-Virus Filter portable. 

Let me explain.

Say you are a small organisation, with 10 users.  You build up a VMWare based Spam/Anti-Virus Relay server hosted at a reliable Data Centre with a Third Party hosting provider (for example the excellent FastHosts).  Your company then grows, and it is impractical for you to continue with your current hosting provider (for any number of reasons, perhaps you have your own dedicated pipe into your shiny offices).  You simply copy the VMWare files onto a Server in your new hosting environment, link those files into the VMWare Console, and voila, after changing IPs (you should only really have to change one (1) IP Address - the Spam/Anti-Virus Filter’s public address), and Bob’s your proverbial uncle.

If on the other hand, you choose a Physical Server, you have to re-install, copy configuration files, mess about, and then if you’re lucky it’ll work, otherwise just a clean rebuild - and worse yet, you’ll lose a lot of statistical and spam-training data.

My personal recommendation - go with a Virtual Server.  Especially now that VMWare Server is a free download (even for commercial uses).

Just a quick note on VMWare, at the time of writing (December 2007), there is a VMWare Server 2.0 Beta available to download, I would advise against using Beta software for production environments, therefore, if you’re gong to be using a Virtual Machine, and you want it for a real world environment, then please make sure you download the last stable version (either for Windows or Linux, whatever your host platform is - host is the machine that you install the VMWare Server on, guests are the machines that run within the VMWare Server and provide various services).

You can download the latest STABLE version of VMWare from the following locations (one for Windows and one for Linux):

VMWare Server Version 1.0.4 (Windows)

VMWare Server Version 1.0.4 (Linux) - .tar.gz Format

VMWare Server Version 1.0.4 (Linux) - .rpm Format

VMWare Server Version 1.0.4 (All Versions)

You will have to register, and the lovely guys at VMWare will provide you with an Activation Number that will allow you to use your VMWare Server.

If you are going to be installing on a remote server, I would advise that you install the VMWare Client Package on your local machine, so that you can remotely manage your VMWare Guests easily.

VMWare Server Version 1.0.4 Client Package for Windows

VMWare Server Version 1.0.4 Client Package for Linux

Right then, now that you’ve decided where you’ll be installing your Anti-Spam / Anti-Virus Filter, so now you need to put the base operating system onto that machine.

Let’s move onto that:

Part 2 - Operating System Setup

You could use any number of operating systems, but I will go with Ubuntu over here, and this entire HOWTO uses Ubuntu as it’s primary operating system - you might get things to work with other Debian based flavours, or even non-Debian based flavours of Linux, but those are beyond the scope of this HOWTO. 

To assist those who have another distribution preferred, I will highlight those items that are Ubuntu only (and where possible give alternatives for Red Hat based Linux Versions, notably CentOS - but I don’t guarantee this, we’ll just have to see how this HOWTO proceeds ;))

Okay, so now we want to do an Ubuntu Install.  I chose Ubuntu 6.06 "Dapper" LTS (simply because this is a server platform, and it will require Long Term Support in the form of updates and patches from the OS Vendor.  Ubuntu 6.06LTS Server will be supported until 2011 according to the Ubuntu Web Site, whereas the current version (Gutsy Gibbon 7.10 which I use on my desktop and is absolutely lovely - but that’s another HOWTO for another day) will only be supported until 2009.  See: Ubuntu Dapper Release Notes.  I am led to believe that the next version of Ubuntu (8.04 Hardy Heron - Scheduled tentatively for sometime in June 2008) will be another LTS (Long Term Support version), and so, when that is launched, I will post instructions on how-to update from this install to a new Hardy Heron release.

So now that’s out of the way, let’s get cracking on the installation process.

I’m going to be building on a VMWare Virtual Machine, if you’re doing your build on a physical server, then that’s fine, you can ignore this initial portion where we setup the VMWare Guest Environment:

So now I’ve logged into my VMWare Console, and connected to my VMWare Server, I now have to setup a new Virtual Machine to build the Operating System on. 

Firstly, Click the "Create New Virtual Machine" button:

Now Click the "Next" button to get the next page of the Wizard:

You now need to make sure that you change the default selected radio-button from Typical to Custom (as shown in the image above).    Once you’ve done that click the "Next" button to continue:

Over here, you need to make sure you select the Guest Operating System type is set to Linux, and in the version drop down, you scroll to Ubuntu.  This will make sure that the Virtual Machine Settings are correct and safe for Ubuntu to function correctly, in terms of drivers and such things.

Non-Ubuntu Note: If you are not going to use Ubuntu, then you can use any of the flavours of Linux listed in the drop down.  For CentOS, you should use the Red Hat Enterprise Linux options - depending on your version of CentOS - further details can be found on the CentOS Website. 

Once you’ve made the selections as outlined above, click the "Next" button to continue onto the next page:

Over here you need to give your Virtual Machine a name.  This is something that you can choose yourself, this is the name that will be given to it’s directory, and the name by which it’ll be referenced with in the VMWare Console.  It is NOT the name that the actual Virtual Machine will have within the Operating System - although you could quite happily name them both the same.

For our purposes, I have chosen the name SpamAVFilter, since it describes what the Virtual Machine (VM) is for, and keeps things quite simple.

Once you have selected an appropriate name, and typed it into this page, click the "Next" button to continue:

Now you get to choose how many Virtual CPUs your Virtual Machine will have.  VMWare will automatically work out how many CPUs it can allocate to your VM, and present you with a list.  In my case, I can use up to two (2) CPUs for this Virtual Machine.  I will however, select to use only One, since this is an example build, and I have other virtual machines running that use up CPU Cycles also.  Also, VMWare’s support for 2-Way Virtual SMP is Experimental, so that’s an important consideration.  See VMWare Guest Operating System Notes for Ubuntu 6.06

If you are building this on a fresh machine, you might want to use all the horsepower you can for your Virtual Machine, again this will depend on how big your organisation is, and how many mail messages you expect your Anti-Spam / Anti-Virus Server to process.

Once you’ve chosen how many CPUs you want for your Virtual Machine, click the "Next" button to continue onto the next step:

This step is quite important, and deserves some explanation.  If you keep this Virtual Machine private, then only you as a user will be able to access it (in terms of file system access).  If you however, choose to make it public (by un-checking the checkbox "Make this virtual machine private" then all users will be able to access the files for this Virtual Machine.

Generally, I have found that un-checking this is acceptable, and indeed makes moving the machine from one VMWare Server to another considerably easier, otherwise you have to go through a nightmare of permissions.

For the purposes of this demonstration, I choose to NOT make this Virtual Machine private.  In a deployment scenario, it might be prudent to keep it private, but be aware that when it comes to moving the VM to another machine, you will have to modify access permissions for the VM Files. 

Once you’ve chosen to make the virtual machine private or not private, click the "Next" button to continue:

Now, come probably the most important screen in the Virtual Machine Creation Wizard.  This screen allows you to run the Virtual Machine as a particular user.  In my case, I have chosen to run it as the Administrator (not always a good idea, but this is only a demo), you might want to create a separate user (under Linux or Windows depending on what your VM Host Operating System is), or you can use the Local System Account or the User that Powered the Machine on.

You can also set the behaviour that the machine will exhibit when the Host Machine either starts up or shutdown (Windows is terrible at shutting down randomly - if not configured right, whenever it does Windows Updates).  This option, will define what will happen to this virtual machine, in the event of Windows Starting up and Windows Shutting down - naturally this is also the case for Linux (but Linux doesn’t randomly reboot after updates, go figure!).

Choose the options from the two drop-down lists that suite your requirements, and then click the "Next" button to continue:

This is where you define how much Memory (RAM) your Virtual Machine will have.  The limits that you have to abide by (Guest OS recommended minimum and Maximum Recommended Memory are provided, as well as the memory amount that the VMWare Server suggests you use.

In my case, I have chosen to go for 256MB, simply because this is a test system, if you are running it for a production environment, I would advise a minimum of 512MB, and ideally 1GB+ to give you the performance that you need.

Once you’ve chosen how much RAM you want your Virtual Machine to have, click the "Next" button to continue:

Make sure you select "Use Bridged Networking" over here for the moment, we’ll be adding more network interfaces to this Virtual Machine, but for the moment, we’ll take a bridged one, which will allow our VM to get to the physical Ethernet Interface on the server.  I will also discuss firewalling, hardening, securing and protecting your network  (in Step 8).

Click the "Next" button to continue:

This page, allows you to select the type of SCSI adaptor the Virtual Server will emulate for your Guest Operating System.  I would however recommend the SCSI LSI Logic adaptor, as that is what VMWare themselves recommend for Ubuntu.

Click the "Next" button to continue:

Now we have to create the Virtual Machine Disc, this will be our Virtual Hard Disc and is probably the most important file within the whole Virtual Machine.  Since we are creating a new virtual machine, we’ll be using the first option - which is to "Create A New Virtual Disk".

I would advise against the third option of using a physical disc, there are pros and cons to using a physical disc, but I won’t go into them here.

Click the "Next" button to continue:

On this screen, you can set the size of the hard disc for your virtual machine.  This is entirely up to you, and dependent on how much physical hard disc space you have.  For this how-to I will stick to an 8GB disc.

If you are building for a production environment, I would advise that you check the Allocate all disk space now checkbox, so that it allocates all the space at set-up time, and you get the best performance.  However, this does take some time.

I also don’t bother splitting the files into 2GB portions, while again there are pros and cons to this approach, that discussion is outside the scope of this how-to.

Once you have decided on your disc space and allocation, click the "Next" button to continue.

If you wish to name the file with a particular name, you can do so here, but I would advise, that unless you have a very good reason for doing so, let the name be as chosen by the VMWare Server (it’s based on the name you had chosen for the Virtual Machine - see above).

Now click the "Finish" button, and let the Virtual Machine be created.  Once the VMWare Server has completed the initial set-up of your virtual machine, you will see a screen similar to the one below:

Now we need to further edit our Virtual Machine, so click the Edit Virtual Machine Settings button, and you will see a screen similar to that below:

We have a number of tasks that we have to perform now, so firstly, let’s list the modifications we have to make:

1. Add extra network interfaces
2. Disable Floppy (Drive A:)
3. Mount Ubuntu 6.06 LTS Server Edition Image as CDROM

Firstly, we’ll add the extra network interface(s).  For this how-to I will only add one more Ethernet interface, but dependent on your network configuration, you might have to add more.

So to add a new network interface, click the Add + button, and you will see the following screen:

We now need to select in the list Ethernet Adapter and click the Next button:

Over here, I have chosen to have a second network adapter that is Host-Only, you may want to choose something else, say a custom network adapter - this would depend on the configuration of   your VMWare Server and this is outside the scope of this how-to.

So, select the type of network interface you would like to add, and click the "Finish" button.  You will now have a second network adapter on your Virtual Machine, as shown below:

Now we need to get the Ubuntu 6.06 LTS Server ISO Image from the Internet (or from a local directory on your Virtual Server Host, or even CDROM if you’ve burned the image to CD.  If you’ve burned the image to CD, you can skip this particular step, since I’m going to use an ISO image, we first need to download it:

Point your browser to http://www.ubuntu.com/getubuntu/download, select the Ubuntu 6.06 LTS Server Edition checkbox, and choose your machine’s architecture (for the VMWare version, since we’ve not chosen to have an Ubuntu 64bit setup (See above), you can use the Standard 32bit ISO, as shown below, and then use the drop down to select a download location near to you, then click the big green Start Download button:

You will then be presented with a download dialogue box (or link if the download is blocked and you’re using Internet Explorer).  Be sure to save the file to a sensible location:

Now let the download complete, and then copy the ISO file to your VMWare Server Host machine.  Now we will mount the ISO as a CDROM in the Virtual Machine:

Firstly, make sure that you are in the Edit Virtual Machine Settings screen.  From here, select the CDROM, and you will have a screen much like the one in the image below:

As you can see there are many options for how to get the CD or ISO Image into the Virtual Machine.  If you are using a physical CD, you can choose to have the CD in your machine (in which case choose the Client option and make sure you have selected the radio button that says Use a physical drive, if you want the CD in your Server, select the Host option.  Keep the Legacy Emulation option open unless you have a very good reason for not having it selected.  If you don’t know what it means, let it remain selected.

In the case for this how-to, we’re going to be using an ISO image, so we select the Use ISO image radio button, and browse to the appropriate location (where we’ve just downloaded the Ubuntu 6.06 LTS Server Edition ISO file to as shown below:

Then once you’ve found the appropriate ISO file, remember, when you browse the Virtual Machine, you’ll be browsing the directory structure on your Virtual Machine Host, not your local machine, so be sure that you’ve copied the ISO file to a location on your Virtual Server Host.

Once you’ve found the correct file, highlight it (as shown above), and click the OK button.  You will then have a screen that is similar to the one below:

Again, unless you have a good reason for changing the Virtual CD Type from ISO to SCSI, leave it be as IDE.

Now, your virtual machine is ready to go.  Click the OK button, and we’ll start up the Virtual Machine:

Now, we’re ready to install Ubuntu 6.06 LTS Server Edition and start our Anti-Spam / Anti-Virus filter box setup.

Press the Power on the virtual machine button, and let’s move onto Step 2.

Step 2 - Operating System Setup

One you press the Power on the virtual machine button, you will see a screen similar to the one below:

 

This will then boot the Ubuntu 6.06 LTS Server Edition ISO, and you will be presented with the Ubuntu Menu as shown below:

Over here, you have a number of options, we are only concerned with the first one, since we don’t want to install a LAMP (Linux, Apache, MySQL and PHP) Server, we simply want a base server install that we’ll customize through the course of this HOWTO.  For now, keep the first option selected and we’ll go through the prompts that you’ll get step by step to get a base Ubuntu Server build completed.

An important point to note about working within a VMWare machine, is that if you click inside the VM (which you’ll have to do to select the option, you can come out of the VM and into your own operating system by pressing the Alt and Ctrl keys on the left of your keyboard simultaneously.

Press enter to select the first option Install to the hard disk and then you’ll see the window shown below (after a few linux loading processes complete):

Here, you have to select the language that you’ll be installing and working in Ubuntu with.  Generally if you’re reading this chances are you’ll be an English speaker, and will want your install in English, but other languages are available.

Select your chosen language and press the Enter key on your keyboard to move to the next page:

Over here you need to select the location you’re in.  Since I’m in England, I’ll select United Kingdom and press the Enter key on your keyboard to continue.

The installer will attempt to detect your keyboard layout, if it is correct (as it is in my case, simply press the Enter key on your keyboard, otherwise, you can scroll down to the Find your layout by pressing some keys or Select from full keyboard list options.  If you are unsure as to whether the auto-selected layout (or even one you selected) is correct, you can Test whether this layout is correct.

Once you’re happy with your keyboard layout, press the Enter key on your keyboard to continue.  The installer will now perform some tasks during which you’ll see the progress bar move, much like in the image below:

After a while, once it’s finished doing those tests and retrieving various modules, since we have more than one network card installed, it will present you with the following screen:

Over here, you have to choose the primary network adaptor, in our case it will be the eth0 (that’s the first interface created when making the Virtual Machine as described in Step 1 (above).

Leave the selection as it is, and press the Enter key on your keyboard to continue:

The Ubuntu installer is now trying to obtain an IP address for the system via DHCP, if you have a DHCP server running then it will find an address for itself, this is generally the best way.  Once it has found an IP address it will present you with the following screen:

Over here you have to name your server.  I have called it anti-spam-av-filter, however in a production environment this is not a good name, since it gives away too much about the machine’s role and can provide valuable information to a potential nasty person who might wish your establishment some harm. 

Think very carefully about what you name this machine, and once you’ve decided type it in.  The default value is ubuntu this is also not a good idea in a production environment as it gives away the operating system, which is never a good idea.

Once you’ve decided on a name, type it in, and press the Enter key on your keyboard to continue:

The Ubuntu installer is now detecting your hard discs, once this is completed you will be presented with the following screen:

Over here, unless you have a  good reason to use LVM or manually edit the partition table, you can select the default option Erase entire disc, I would also advise against manually editing the partition table unless you have a good reason to do so, which is fine, but at the very least if you’re going to manually build the partitions, make sure the following partitions and mount points are created:

Mount Point	Size	Notes
/boot	50MB	Boot Partition (ext3 (primary))
SWAP	2GB	Swap partition (size is dependent on your system’s available memory, a general rule of thumb is double your physical memory)
/	2GB	Root Partition (ext3 (logical))
/var	1GB	Variable Data Partition (ext3 (logical)
/var/log	3GB	Variable Data Partition (Log Storage) (ext3 (logical))
/var/spool	2GB	Variable Data Partition (Spool) (ext3 (logical))
/usr	4GB	User Installed Programs (ext3 (logical))
/usr/local	2GB	User Installed Programs (ext3 (logical))
/home	REMAINDER	Home directories (Good place to put any extra space, you can re-partition if you run-out of space and use this) (ext3 (logical))

For this how-to, I will let the Ubuntu Installer use the entire VMWare 8GB disc we’ve created and let it build the partition table itself.

Select the first option (this how-to doesn’t cover the LVM or manual options, sorry - if you really need help with LVM or manual please get in touch and I’ll see what I can do to help out).  So select the first option, and press the Enter key on your keyboard to move on:

The Ubuntu installer will now compute the partition sizes and you will see the progress bar move a few times.  Then it will present you with a summary page - much like the one below:

Please be sure to read through what has been written.  Once you are confident and comfortable, you can choose to select the <Yes> option and press the Enter key on your keyboard.  (You can select the various options by using the TAB key on your keyboard to move between various options).

Once you’ve chosen to commit the changes and pressed Enter on your keyboard, the installer will go through the process of writing to your partition table (on the virtual machine), and once it completes, you will have a screen like the one below:

Over here the Ubuntu Installer is asking if your system’s hardware clock is set to UTC.  Generally, unless you have a good reason to do otherwise, just select the <Yes> button and press the Enter key on your keyboard to continue:

 

Over here you need to enter the Full Name for the new user.  Now here, we need to learn something that’s a little important about Ubuntu, and also forms part of the entire security mechanism within Ubuntu.

Ubuntu installs itself by default as a setuid system.  This essentially means that the root (or superuser) account is disabled by default, and you have elevate the privileges for any user by using "sudo", to run that command as root (superuser).

This is important, and has many advantages, and we will be making extensive use of this excellent security feature as part of our installation process.

Coming back to the screen here, you need to enter the Full Name not the login name, of a user that you will use to generally log into the system, and do your work.  This won’t be a root level user (meaning it won’t be a superuser), but through elevation we’ll be able to perform privileged tasks.

Generally it is bad practice to have a user that indicates an Administrator.  I have chosen "Spam Hater" as the full name of the user, when we come to the actual username, we’ll get more into this.  You don’t have to call the user Spam Hater, you can name it after yourself.  I could have named the user Shabbir Hassanally, and it would be just as acceptable, but since this is a server that does a particular task, I prefer to keep the username as obfuscated as possible - this will be made clearer once we see the next page.

For now, type in the full name of your user, and tab across to the <Continue> button and press Enter on your keyboard to continue:

Over here you have to choose a username, this is the username that you will be using to login to the system.  I have found it good practice to make is different from the Full Name, as the harder a username is to guess, the more secure the system is, since if someone who wishes to compromise your system doesn’t have the username, there is little chance of them getting the password, and if they have the password but not the right spelling of the username, the same thing will apply.

For this demo, I have chosen diespamdie, and this is the user that I will be logging in into the system to do the configuration of our Anti-Virus/Anti-Spam Filter.

Once you’ve chosen an appropriate username, tab across to the <Continue> button and press the Enter key on your keyboard to progress to the next page:

On this screen you must type a password, follow the advice given on the page, which says that a good password should be comprised of a mixture of letters, numbers and punctuation, and do change your password regularly. 

Once you have typed your password in, tab over to the <Continue> and press the Enter key on your keyboard to continue.

This screen requires you to re-enter the password you entered on the previous screen, to confirm it (this is also useful if you’ve mistyped it on the first screen, and prevents you from inadvertently getting locked out of your own system!!).

So, confirm your password (enter it in again), and then tab over to the <Continue> button and press the Enter key on your keyboard to continue:

The installer will now start the final installation of the  server software on your Virtual Machine.  You will see a number of screens like the ones below:

Until you reach the following screen, which indicates that the installation is nearly complete:

Over here you’re now ready to restart the Virtual Machine (or real machine if you’ve performed this installation on a real physical machine as opposed to a virtual machine).

Press the Enter key on your keyboard to restart into your fresh and new Ubuntu 6.06 LTS Server Edition Server.

Step 3 will continue in the next post (this one has already gotten obscenely long!!)

To be continued … we still have to install and build the Anti-Virus / Anti-Spam Filter portion - right now we’ve got a base operating system completed!! Lots more to do yet!

Warm Regards,

Shabbir