Joe Job Spam Run

The university received two spam run campaigns, the first uses a forged sender to make a university address look like the sender, the second uses forged university addresses (i.e. not accounts) in an outgoing campaign to other sites, resulting in backscatter messages to Oxford account holders. This page is to answer some of the common end user and IT officer queries relating to this.

One of my coworkers got a spam apparently sent from my address, have I or the mail server been hacked?

Probably not if it was in the recent long weekend (29th April – May 2nd). Queries this morning that I answered received a variation on the following:

If it was this weekend then it was part of a Joe job email run, specifically the key here is ‘address’ not account. Emails are like postcards, they can be signed as ‘from’ anyone.
In this case someone has used Oxford addresses as the forged sender address in a spam campaign. This is known as a “joe job”: http://en.wikipedia.org/wiki/Joe_job

I’m afraid this is an aspect of the way email works that is misused by the spammers.
Note that the spams of this type from this weekend have a spam score of 25-30. If on Nexus even the least sensitive option will filter these, there are instructions here:

http://www.oucs.ox.ac.uk/nexus/email/

I’m not keen on canned responses but there were too many queries not to prepare some form of template for it.

Are there any official university pages with generic information about spam?

Yes, try the chain and junk mail page

I’ve whitelisted ox.ac.uk as an incoming sender to my account…

Please don’t do this, it’s not needed and it will cause you more spam. Firstly the ‘from’ address on the incoming spam can be forged to an Oxford address and so bypass your filter settings, secondly mail from internal to internal hosts is not spam scored by the central mail relay. So for example mail from Engineering to Physics and similar will never have a spam score applied by the university mail relay when sent via internal servers.

I had spam sent to me and I blocked the sender address in my account but now I get the same message from another address, and another…

The sender address is forged, it’s like writing who the sender is on a postcard – you could write anything you like and therefore there isn’t much point to blocking based on sender address for the standard types of spam.

An email from a coworker in my same unit went to my spam folder, I thought you didn’t spam score internal mail?

Most likely you have Microsoft Outlook and the local client based mail filtering option is on. It uses rules from Microsoft rather than the university mail server rules and is best switched off due to a high number of false positives.

I’ve heard that you don’t filter any messages but pass them on to local units?

I was a bit surprised to hear someone suggest this. We don’t silently delete any email. We do SMTP time message rejection based on a number of criteria to reject the majority of spam and delivery attempts from compromised hosts, then we spam score the remainder and pass it on. We don’t accept mail and then silently drop it. We either accept and deliver or refuse to accept the message. We do, most certainly perform anti spam techniques on incoming mail to reject it as soon as possible. There are OUCS pages relating to the main mail relay.

Well then, why aren’t you filtering/rejecting this weekends messages?

We are. The messages getting to users inbox are the tip of the iceberg, the majority of connection attempts in this spam run will have been rejected at the first delivery stage by our mail servers using various techniques. The emails that are accepted are then spam scored.

Why don’t you just block the sending host?

In my experience of attempting this, only the more quasi-legal advertising companies use a single address or a handful of addresses or single network. We let the automatic blackilist updates that we receive take out the majority of sending hosts.

Why did this message get through? Here is my message header

From: university.address@ox.ac.uk
Sent: 03 May 2011 01:03
To: some.user@ox.ac.uk
Subject: from Cornelia
I'm an hot brunette girl, and I'm searching for a man to chat with [...]
I have registered my profile at:  www.some-site-beingspamvertised.ru

This isn’t a message header, there are instructions for message headers here: http://www.oucs.ox.ac.uk/email/headers/ it’s not that we’re being picky, the message headers tell us a lot of technical information about the message – which servers it went through and what score it got. Showing a message header usually results in the immediate explanation for a mail issue since the majority of the information needed is usually contained.

I don’t know anything about message headers, just tell me how to get them, I use Lotus notes..

The networks team that run the mail relay don’t know about your local mail clients, we only know about message delivery (e.g. from the outside world to Nexus or to your units own mail server or between internal mail servers). Your primary point of contact for your unit is your local IT officers who will know far more about what choices your unit has made and common issues and configuration with your chosen mail client than I or my team members.

I’m an IT officer, I’m looking at a message header, can you explain what’s going on roughly?

Yes, the first line we trust is where our mail relay takes the message. We know that the IP address it records as being the connecting server is correct (SMTP is TCP not UDP so sending packets with a forged IP address would rather difficult since a three way hand shake must complete – the server connects back to the address that contacted us), and any other lines before this may have been forged by the connecting server


Received: from 188-115-172-147.broadband.tenet.odessa.ua ([188.115.172.147])
by relay0.mail.ox.ac.uk with esmtp (Exim 4.75)
(envelope-from <kfaczek@sbe-ltd.co.uk>)
id 1QHCLG-00051H-15 for some-address@herald.ox.ac.uk;
Tue, 03 May 2011 10:56:30 +010

So in the above line, our mail server relay0.mail.ox.ac.uk has accepted the mail from a server at 188.115.172.147, it’s using the older email addresses the university used to use as it’s source of contact addresses. We don’t care about where our mail relay delivered the message next internally for this incident so these lines aren’t shown.


Received: from 188.115.172.147(helo=herald.ox.ac.uk) by herald.ox.ac.uk with
esmtpa (Exim 4.69) (envelope-from) id 1MM13H-1826ej-31 for
<some-address@herald.ox.ac.uk>; Tue, 3 May 2011 11:56:29 +020

Here the connecting server at 188.115.172.147 has added a totally fake log line, perhaps to try and confuse analysis and/or to see if some form of whitelist will cause the message to be accepted due the suggestion an internal mail server has already processed it.

[...]
x-oxmail-spam-level: ***********************************
x-oxmail-spam-status: score=35.1
tests=FH_HELO_EQ_D_D_D_D,HELO_DYNAMIC_IPADDR2,OX_RBL_MAPS[...]

This is the important bit, we’ve accepted the message so the sending host has passed a number of tests, but now we’ve spam scored the message.
Each test that is failed raises the score, we can see the message has a high spam score due to a high number of failed tests.

I sent a copy of some spam to the OUCS phishing address, they didn’t seem too keen…

They’re only resourced to tackle phishing incidents targeting university account credentials – they can take actions to prevent users accounts being compromised (we have a legal obligation not to send spam) but standard spam isn’t the same. The phishing contact address is staffed by members of the security and networks team that have other tasks and can’t manually tackle each individual spam.

Ok, well here is my message headers, you should do something about this…

[...]

x-oxmail-spam-level: ******************************
x-oxmail-spam-status: score=30.5

Please turn on your accounts filter options or assist your user you are supporting to do so. We recommend that anything over a spam score of 5 is probably spam, with the occasional false positive (hence we recommend moving it to a folder not automatically deleting it), Anything with a spam score of over 12 is always spam (with the specific exception of the university security team who email each other malware links as part of their daily work). This message scored over 30 which is high enough that even a very lax setting will filter out the message.

What about SPF! I’ve heard SPF will fix things like this and I use it on my personal domain…

SPF isn’t a great solution – there are knock on issues with implementing it, it doesn’t solve all that many problems and some political changes would have to be made. Your personal domain isn’t complex, if implemented at Oxford we’d need to enforce/ensure that everyone is using the university mail servers when sending as anyone@unit.ox.ac.uk and ensure they are not using external mail servers (such as that provided by their ISP). If we achieved that then we would probably implement DKIM instead as a better technology. Note that SPF and DKIM assist with anti spam techniques but do not cure it.

I run a department mail server so based on your advice I’m going to silently delete any mail with a score over 5

Please don’t do this, you will delete legitimate correspondence which is bad postmaster-ship: your users will come to think of email as silently unreliable and raise support queries to track each lost message. Email filtering is not a boolean (true or false, 1 or 0) operation. Messages over 5 are probably spam, with the occasional false positive, the recommendation is to filter these to a users spam folder. Messages over 12 should always be spam.

Ok, I run an internal mail server that accepts incoming mail from the central mail relays, what should I be doing?

Take note of the oxmail spam score – if there is no score (not 0, but no score at all – no x-oxmail-spam-level sign) then it’s come from an internal server and I’d recommend you don’t run your own spam filter on it, but deliver it to the user. It’s very rare that an internal address is compromised and sends to internal addresses.

  • The X-Omail score includes scoring from SMTP time checks, it’s recommended that you use the score or at least take it into account with your own scoring mechanism
  • We suggest you filter messages over a score of 5 to a users spam folder, so they can check for false positives, but you or your users might change that level
  • If your users have Outlook deployed to them, turn off the Outlook based local spam filtering as it causes issues and will flag internal mails. It does not relate to the scoring applied to the mail relay but is controlled by Microsoft.
  • Check postmaster@ and abuse@ your domain of ox.ac.uk work.
  • If Oxmail delivers something to your mailservers that your product flags as spam, please accept the message and spam score it to oblivion. If you drop the connection oxmail will have to assume your server had a network issue and will try again and again for 10 days then send a delivery failure message back to the sender (which if forged is backscatter and may result in a blacklisting of the university mail service).
  • Remember you will always have some degree of spam – there is no perfect cure on the internet to date, no matter what any vendor says or how clean your gmail account appears.
  • Turn off your unit level firewalls port 25/SMTP inspection function – it causes issues
  • Check your SMTP logs first before raising queries with OUCS and you’ll answer most of your queries

Are there any stats? Can I see the filtering in action?

Yes there are graphs linked from the mail relay statistics page. You can see the increase in hosts rejected this weekend due to being blacklisted on the lists we utilise on the rejections graph

I have more questions/you left something out

End users can email help@oucs.ox.ac.uk , IT officers can get in touch with us about aspects of the server at networks@oucs.ox.ac.uk

Posted in Mail Relay | 2 Comments

2 Responses to “Joe Job Spam Run”

  1. Stephanie Jenkins says:

    Please could you block [edit by Christopher Causer: email removed]

    Many members of the University joined it when it was about cycling, but it has been hacked and is sending out masses of porn.

    I cannot work out how to block it myself with Nexus

    • Christopher Causer says:

      Hi Stephanie,

      Any abuse of an email mailing list (in this case outside the University’s control) should be handled by them. We do not block mail on the mail relays, regardless of its content (except email we believe to contain viruses.)