Email server vulnerability detection - a best practice checklist
Email server vulnerability detection - a best practice checklist

Providing proper security should be the number one priority for any company running their own email server, whether private or public.

Of course, you can never get rid of every single vulnerability out there, but there are definitely ways to detect and fix some of them. By following best practices and incorporating security measures when setting up an email server, you will be able to protect yourself from the most frequent and dangerous scenarios.

Unauthorised data access

One of the extremely widespread threats is when a perpetrator tries to access data without the correct authorisation. The best way to combat this is to set strong requirements for passwords used to access the server. This will protect you from brute-force attacks frequently used to bypass authentication.

Other security measures depend on the type of the server in question. However, what you can also do, is to use SMTP authentication.  

Data leakage

Personal data is a key target of malicious perpetrators. When you send an email containing sensitive personal information, it goes through completely unprotected channels. This message can be intercepted, resulting in sensitive data leakage.

One way to prevent this is to use encryption with both incoming and ongoing emails. You should use SSL/TLS encryption with POP3, IMAP, and SMTP protocols.

Spam emails

Spam emails are an extremely widespread and important security problem. From a server security standpoint, there are two types of spam:

  • Spam to own clients

  • Spam to other clients, where server acts as an Open Relay

To stop spam, you can install content filters on the mail server, or on a proxy (firewall, dedicated proxy component, etc.) used to protect access. Additionally, you can also block known spam servers based on publicly available blacklists, such as DNSBL, Spam URI RBL SURBL, and various local blacklists.

Preventing Open Relay is possible by configuring Mail Relay parameter of the server.

By checking the actual effectiveness of used content filters, we can test the effectiveness of our anti-spam protection.

Denial of service attacks

Denial of service attacks can be extremely damaging. When your email services is out, it means that crucial messages aren't going through, which means financial and reputational damage for the company. Another important factor is the time and effort it takes to restore the service.

One way to prevent such attacks is to limit the simultaneous, as well as a general number of SMTP server connections.

Final recommendations

You should start thinking about security as soon as you decide to run a server. Planning ahead is much more affordable and provides better protection in the end.

Here are the things you may want to consider if you planning to have email server:

  • Consider what services an email server should support and with what kind of data it will work

  • What security requirements does email server should have

  • What authentication methods will server have

  • What level of privilege will various server users have

  • How the server will be integrated into the existing network infrastructure

  • How the server will be managed

This set of questions leads you to consider the minimum level of security, necessary to prevent attacks.

Finally, regardless of what type of servers you use, there are basic universal recommendations, that everybody should follow. Down below you will find some of them:

  • Limit the attack surface on the server. This requires a specific network infrastructure. One example of such infrastructure is a server inside a protected perimeter with a single proxy serving external emails and having access to the private network. In the case of Exchange Server, Edge Transport can be used as such a proxy.

  • Don't forget to encrypt data in transit when possible. Don't use self-written SSL certificates, and instead choose them from reputable sources.

  • Don't forget about the basic protection. Reputable anti-viruses will compliment any built-in anti-malware capabilities.

  • Don't forget to apply updates whenever possible. They will protect you from popular malware and ROP attacks.

  • Create a backup server for emails and set at least two MX DNS records for smooth operation.

Contributed by Marcell Gogan a specialist within digital security solution business design and development, virtualisation and cloud computing R&D projects, establishment and management of software research direction.

*Note: The views expressed in this blog are those of the author and do not necessarily reflect the views of SC Media or Haymarket Media.