Musings on Mac Malware

Apple Store, Fifth Avenue

Apple: under attack day and night


Over the past couple of weeks, OxCERT have been somewhat overwhelmed by Mac malware. This isn’t quite the first time we’ve dealt with problems on Macs – we’ve seen several compromised over the years through weak or exposed ssh credentials, and others infected as a result of installing pirated software. But with Flashback, the game has changed forever. We are seeing huge numbers of attacks of the sort that Windows users have had to contend with for years. Apple users, and indeed Apple themselves, just have not been ready. We are dealing with what is probably the biggest outbreak since Blaster struck the Windows world all the way back in the summer of 2003. That time OxCERT dealt with around 1000 incidents; we have seen several hundred Flashback incidents and they keep on coming.

What is Flashback?

Flashback is not in fact that new, it has been around in various forms since September 2011. Like much malware, multiple variants exist, as the attacks evolve to exploit new vulnerabilities, avoid detection and adapt to new purposes. Early versions required user interaction in order to execute, but in recent weeks the malware has been exploiting a vulnerability in Java, allowing for “drive-by” exploits where all a user has to do is to visit a webpage hosting malicious content (perhaps via a third-party advertisement).

Once on the system, Flashback gives the attackers the ability to do pretty much whatever they like with it, at least until someone stops them; it will depend on the particular variant and what the command-and-control systems tell it. In the Windows world, a common approach has been to capture users’ keystrokes and other information from the system in order to gain access to their online banking. Others may be interested in what other resources they can access, whether on the compromised system itself, or via resources to which the user has access. Some University users have access to some very sensitive data.

The Java vulnerability is one that Oracle fixed on 14 February. Anyone using Oracle’s auto-update mechanisms would have received this update shortly after, but under OS X, Java is distributed by Apple and most users have to wait for Apple to release the update. In this case, Apple did not release it until 3 April, some seven weeks later, by which time the vulnerability was being widely exploited. The reason behind this delay is unclear, but it is not a one-off: Apple have consistently lagged weeks behind Oracle on Java updates. For all we know there may be good operational reasons as to why it takes so long for Apple to release the updates, but as is usual they have been completely silent on the matter.

Java is not the only application being exploited – there are reports of others being targetted, such as emails containing malicious Word attachments. All too familiar in the Windows world of course.

“But Macs don’t get viruses!”

Sadly far too many users still appear to be under the misapprehension that “Macs don’t get viruses” in spite of decades of evidence to the contrary. Indeed, at the time of writing, Apple themselves still state that a Mac “doesn’t get PC viruses”. Technically true, perhaps, but very misleading: PCs get PC viruses, Macs get Mac viruses which may be extremely similar to that common on PCs, in spite of the “built-in defences”. (Note also the claim that “Apple responds quickly by providing software updates and security enhancements” – as we’ve seen, this depends very much on your definition of “quickly”.)

There was perhaps a time when the threat of viruses was sufficiently low that Mac users didn’t have to worry too much about having antivirus software installed, but that time is long gone. Apple’s “built-in defences” weren’t saving users from Flashback infections. It’s true recent versions OS X have a built-in antimalware capability, but it is extremely limited and no substitute for a proper third-party antivirus system. Sophos is widely used and supported in the University but no doubt most of the major players have equally good solutions.

What should Mac users do to protect themselves?

Really, it’s a case of taking the same precautions are required as on a Windows system.

  • Install antivirus, and ensure it updates frequently, preferably several times a day.
  • Keep the operating system up-to-date (but see below) – ensure Software Update checks on a daily basis, and that security-related updates are applied promptly.
  • Keep third-party applications up-to-date, especially anything that may handle untrusted data from the Internet. Browsers (eg Firefox, Chrome, Opera), mail clients (eg Thunderbird, Outlook), Flash, Java, Acrobat, Office.
  • Be wary. Don’t open email attachments you don’t expect, especially if from unknown senders. Only download software from trusted sources.
  • Enable the built-in firewall.
  • Disable or remove software you don’t use. Under OS X 10.7 (Lion), Apple now do this automatically with Java.

There is however a nasty catch with operating system updates, of which many users will be unaware: Apple security support lifetimes are much shorter than in the Windows world. This is an issue which we discuss further in a second post.

Ultimately though, the game has changed for Mac users. They can no longer sit smugly thinking that few people are going to bother attacking them – Macs are being attacked on a very significant scale, and complacency is asking for trouble. Mac malware has gone mainstream, and will likely remain so.

Posted in General Security | 10 Comments

Apple and security support

In a companion article I discuss Mac malware, and how this has recently become much more of a problem than has previously been the case. As well as Apple’s apparently slow response to a recent vulnerability, and general air of secrecy, one of the problems that the attacks have highlighted is Apple’s product support lifecycles, which are much shorter than in the Windows world. Many users are unaware of the issues and will not realise that their systems may be insecure as a result.

Software support

Mountain Lion

Mountain Lion is released this summer...

Let’s look first at software. To the best of our knowledge, Apple do not officially state their software support policy anywhere, but from what we can gather, only support the two most recent versions of OS X. Currently that is 10.6 (Snow Leopard) and 10.7 (Lion). 10.6 released in August 2009, which means that any Mac purchased prior to that date and not subsequently upgraded will be running a version which receives no security support. That’s for a system purchased under three years ago. Granted, users can upgrade – but at a cost. Users don’t like being told that they have to spend money. Moreover, 10.8 is due out sometime this summer – based on past experience, we can expect 10.6 systems to lose security support once that happens.

Snow Leopard

...but where will that leave security support for Snow Leopard?

Compare that to the situation in the Windows world. Windows XP was released in summer 2001, and will receive security support until April 2014 – twelve and a half years since it released, over seven since it ceased to be the “current” Windows version, and even for those who refused to touch Vista, nearly five years after the release of Windows 7. During that time functionality has been improved considerably by the release of three service packs, available at no charge. Windows Vista and 7 are both scheduled to get over ten years of security support. With around two years between OS X releases, Apple have been struggling to reach four years, and unless their support policy changes as they move to an iOS-style annual release cycle, that could come down to two years.

Now, granted, users can upgrade to a newer OS X release than their system came with. Plenty of users are unlikely to bother unless forced – their system seems perfectly adequate, why spend money and risk breaking it? One college has reported almost 50 systems known to their student registration system running OS X 10.5 or earlier.

Hardware support

But there comes a point at which a system can be upgraded no further. With Apple hardware, that can happen surprisingly rapidly. 10.6 does not support PowerPC-based hardware (last sold new during 2006); 10.7 does not support 32-bit Intel systems – most were superseded in late 2006, but in the case of the Mac Mini, not until August 2007. 10.8 is expected to release this summer and will drop support for even more recent systems, including any MacBook made before late 2008, although 10.7 should continue to be supported.

Thus in little over five years, it’s not just that the latest version of OS X may not run on your hardware, but that no currently-supported version of OS X will. If you want security support, buy a new system. Or change operating system. For old PowerPC systems, that limits you to certain distributions of open-source operating systems (e.g. OpenBSD or Debian GNU/Linux) – in all reality probably not something the average user is willing to consider. If 10.6 loses security support this summer, the best option for owners of early Intel Macs will most likely be to install Windows, heretical as it may sound to many Mac users. Vista’s probably good until 2017, and if the hardware’s up to Windows 7, they’ve got until 2020.

Five years is not a long time to retain a computer, especially in these cash-strapped times. In many departments, it will be their typical hardware replacement cycle, while others don’t even have one. Privately-owned systems may well remain in use until the hardware gives up. For the environmentally-conscious, throwing away perfectly functional machines is hugely wasteful (Apple have made considerable efforts to improve their green credentials in recent years), and the secondhand market relies on people not knowing or caring about the lack of security support, or being willing to run alternate operating systems.

In the PC world, many machines made last century can still make a decent stab at running Windows XP. It would be good to see Apple commit to their hardware being able to run a supported operating system for longer, with a minimum perhaps in the range seven to ten years. The supported operating system needn’t be what the machine shipped with for all that time, nor the current version, nor need all the whizzy new features of newer versions be available (just as Windows Vista would run better on old machines with Aero disabled). The important thing is that the hardware is still fit for use rather than the scrapheap.

The cost

Now, improving support lifetimes is going to cost money, and may deter a few customers from upgrading to newer Apple systems. But equally, customers are going to be less keen to stick with Apple if they learn that Apple are not looking after them. Apple may not have much a place in the enterprise market, but they do in the educational market, and people like us exist to ensure that people do care about security. And let’s face it, these days Apple are most definitely not short of cash.

Now, please don’t get me wrong: there is much about Apple that I like, and I use Apple products daily. I appreciate that Apple are also out there to make money. But they have been complacent in terms of their attitude to security and support, especially when compared to their chief competitor. Microsoft have learned a huge amount from past mistakes, support their products for many years, and these days I feel do an excellent job. By comparison, Apple appear to be making minimal effort, and are putting their customers at risk as a result

So in summary, I’d like to see from Apple the following:

  • Timely security updates
  • Greater openness regarding security issues
  • Minimum hardware and software support lifetimes stated clearly up-front
  • Longer operating system security support lifetimes: at least five years
  • Hardware that runs a supported operating system version for longer: minimum of seven years perhaps?

Whether anything will change any time soon remains to be seen, but as the threats towards Macs increase, surely Apple cannot afford to stand still.

Posted in General Security | 13 Comments

The weekend worm that wasn’t … yet


As I write this, it seems that my fears on Friday have not come true over the weekend. To date, things have been quiet. Nevertheless, we still expect a storm. We just don’t know when. While we wait, let’s consider the story so far.

The MS12-020 RDP vulnerability

On Tuesday 13 March, Microsoft released updates on their regular monthly cycle, including one addressing a vulnerability in the Windows Remote Desktop Protocol (RDP) service. This is remotely exploitable by an unauthenticated user. Microsoft themselves rate this as a “critical” vulnerability and likely to be exploited. In posting our own bulletin on Tuesday evening (sorry, only available within the University network), we drew particular attention to this, knowing that RDP is widely used within the University.

In the past there have been many notable computer worms – self-propagating malicious software. Examples include Code Red, Blaster, and SQL Slammer – the last one significant enough to paralyse a significant proportion of the global internet. It has been a while since we have seen a major new internet worm. In recent years there have been two major factors at work: firstly, Windows has shipped with a firewall enabled by default since the release of XP Service Pack 2 in 2004; secondly, attacks are increasingly the result of criminal gangs seeking to profit from their work, resulting in a trend towards stealthier malware. The most major worm in recent years has been Conficker, something that we still see on our network from time to time, particularly on systems belonging to visitors to the University.

Developments during the week

After internal discussion on Wednesday, we decided to perform a scan of the University network the following lunchtime for systems listening on the standard Microsoft RDP port, tcp/3389. We knew that RDP was widely used within the University but quite how much, we weren’t sure. We scanned all systems from within the University network initially, with followup probes from an external host as we suspected a common configuration would be for systems to be accessible from within the University but not open to the world at large. Our results showed several thousand systems listening on port 3389 and we proceeded to pass the results to the relevant IT officers on Thursday afternoon and Friday morning. Our scans could not determine whether systems had been patched against MS12-020 or even what software was listening on that port, merely that the port in question was open.

Some may ask why RDP isn’t firewalled within the university. We have always operated an extremely open network, with few centrally-imposed restrictions on what people may run; any attempt to impose restrictions tends to result in howls of protest. Only a few very specific threats are blocked across the site, and experience has shown us that anything for which we grant exceptions will tend to impose massive management overheads. Additionally, a perimeter firewall cannot protect against threats from elsewhere on the University network. It is therefore left to individual colleges and departments to operate their own firewalls according to their local requirements. From the centre, we can recommend particular configurations, but not dictate.

Microsoft themselves announced on Wednesday that they expected exploitation of MS12-020 “within thirty days”. If it took, say, a couple of weeks for the bad guys to develop an exploit then this is clearly of concern but gives IT staff a reasonable amount of time to address the problem, taking into account needs to test patch deployments and schedule necessary downtime.

The threat as seen on Friday

Unfortunately, the situation appeared to have changed by Friday lunchtime: reports stated that “proof of concept” exploit code had already appeared on message boards in China. Moving from proof-of-concept to a fully “weaponised” system may be a matter of days or even hours going by past experience. With hours to go before the weekend, this was cause for concern: under 72 hours had occurred since Microsoft had released patches and published their bulletin, exploit code had seemingly already been produced, and the weekend was fast approaching.

We felt that the chances of significant malicious activity before Monday morning were high enough to warrant a strong warning to IT officers. Something that would get their attention to the seriousness of the situation. As we stated, if our fears came true, there was a high probability of major disruption by Monday, requiring major effort to resolve. If our fears were unfounded, efforts had merely gone into securing systems sooner than might otherwise have been the case.

The important message to get across was to ensure all systems were patched if at all possible; if this wasn’t feasible before the weekend, to restrict access. A vulnerable system poses far less of a risk if it is only accessible from a handful of trusted hosts than if it is accessible from anywhere on the global internet.

Systems running RDP will not be a randomly-selected set of systems across the University, but will be heavily biased towards key assets: servers, desktops used by senior staff members, systems holding valuable or sensitive information. Compromises of such systems would clearly be far more damaging and time-consuming to address than compromises of a typical student-owned laptop.

The situation now

Now, the weekend has gone by without significant incident, as far as we are aware, although there has been a further development. On Friday evening, it became evident that the published exploit code was not the work of the “bad guys”. Instead, the code appeared to be that supplied to Microsoft by the researcher who originally discovered the vulnerability, and this had somehow been leaked. Possible sources for the leak are the discoverer, Microsoft or a partner in the Microsoft Active Protections Program, which shares vulnerability information with (supposedly) trusted industry partners; in time we may perhaps see more details of what actually happened.

What none of us can know is what is yet to come; all we can do is make educated guesses. We must still assume that an RDP worm will appear at any moment, The threat is real, it is still there. It is critical to ensure that appropriate protection is in place as soon as reasonably possible. Security researcher Dan Kaminsky estimates around 5 million globally-accessible systems running RDP; even if many have been patched, a very significant number remain vulnerable.

What lessons can we learn?

Much of what we have been saying to IT officers is not new. For years we have been recommending that services are not made globally-accessible unless absolutely necessary. For instance, a public webserver has to be able to serve pages to the world. It probably doesn’t require RDP to be globally accessible. Those needing RDP access most likely only come from a handful of hosts or networks, perhaps only internal to the University, perhaps their home addresses, perhaps also some external contractor has access. There may be a requirement to support mobile users or users without fixed IP addresses. Nevertheless for such users, perhaps a facility such as VPN would be acceptable, providing a “layered” security model. Successful attack requires compromising the VPN (or at least one VPN account) and the RDP daemon. This is unlikely to be a deterrent to targetted attacks but will guard against the random attacks that would result from worm activity.

Allowing access from within the University network alone is far better than allowing from anywhere, but is still inherently risky and is not an alternative to keeping systems patched. Such systems are likely to be attacked as soon as another host within the University network is infected, or a pre-infected host is brought within the University network. We have seen this in the past and will inevitably see it again.

A flexible approach is required to patch management. Some may be happy for patches to be applied as soon as they are released; others may be more conservative and wish to test things before deployment to production systems. This is perfectly understandable; updates certainly have been known to cause unexpected problems, as indeed happened last week on a major University service when another vendor’s updates were applied. If policy is that patches are not immediately applied but must await testing and/or a scheduled maintenance window, then it is useful to have plans for addressing emergencies. Can procedures be fast-tracked in the event of a “zero day” vulnerability, or a significant probability of exploitation in the very near future, where the risks of inaction outweigh the risks of an untested update or of inconveniently-timed disruption to service?

No doubt there will be further lessons to be learned when (we still consider it a “when”) the attacks come. No doubt some systems will have been missed. With so many threats to University IT systems around we cannot go overboard on one particular risk at the expense of neglecting the others. For now, we feel we’ve done what we can given our resources, and must move on.

Posted in General Security | 1 Comment

Oxmail versus Hotmail

As most people within the University will be aware, as well as those who have seen external press articles, over the past week or so we have been experiencing severe difficulties in delivering email to Hotmail. With these problems now largely resolved, we now have an opportunity to reflect upon what happened.

How did it start?

Firstly, a brief explanation of our setup for the unfamiliar. OUCS provide a central SMTP relay service (“Oxmail”) that acts as our primary interface for the exchange of email to and from the outside world. As well as handling messages for centrally-provided services, it handles mail from servers within the University’s many constituent departments and colleges.

Painting by Annika Ström - taken from http://www.flickr.com/photos/seeminglee/4424977043/

There are good and bad ways to request removal from mailing lists


The problems appear to have originated with a mailing list set up by another University department in order to communicate announcements with a group several thousand external people. An announcement was sent to that list on Friday 23 September and all was well until Sunday evening when one of the recipients replied. Accidentally hitting “reply all”.

Now, here the trouble started. For many mailing lists, it’s perfectly reasonable to allow subscribers to post without the need for their messages to be moderated. Unfortunately it wasn’t appropriate in this case. It was clearly a mistake, and should have been avoidable. Lessons have been learned in the department that created the mailing list and better procedures put in place for managing mailing lists. Nevertheless this is the sort of error which no doubt has hitherto occurred frequently, generally with minimal negative consequences.

So the result was that the user’s reply went out to the several thousand list members. Unfortunate, but not yet a huge issue. But then other list members started to reply, again sending their messages to the list: “Why am I getting these messages?” “Unsubscribe!” “Please stop this!” “Someone is getting fired in the morning!”

All this happened on a Sunday evening, and, not being funded for 24×7 operations, those who could potentially stem the tide of emails were blissfully unaware of the problems until they arrived in the office after the weekend. When the complaints were noticed on Monday morning over 300 separate messages had been posted to the list. That resulted in several million separate messages either already delivered or floating around on mail queues somewhere. We put a temporary block on the source and started removing messages from the offending mailing list from our queues. By mid-morning the problem was apparently resolved and we move on to other work.

The Microsoft block

Unfortunately, all was not well. The list membership included a large number of recipients with Hotmail or Microsoft Live addresses, who had (not unreasonably) been marking the messages as junk. The usually strong reputation of our mail relays’ IP addresses started falling, and during Monday evening enough messages had been tagged as junk for Microsoft to stop accepting email from us. This was approximately eight hours after the problem was fixed at our end.

Taken from http://www.flickr.com/photos/loop_oh/4558105863/
We had been expecting something like this to occur for some time, but not for this reason. Over the past few months we have seen a huge number of accounts compromised as a result of successful phishing attacks; despite our efforts at user education we are apparently failing to get through to all our users. Compromised accounts are typically abused to send spam, and we have been warning of the potential for disruption to legitimate messages that may result; there has been one known instance of this (with another provider) over the summer.

On Tuesday morning it was rapidly evident that this case was much worse than we have seen with previous incidents. The Hotmail servers were rejecting messages with permanent errors, resulting in immediate bounce notifications to the senders. Messages to recipients at institutions using Live@edu were similarly affected. A large number of users were complaining. As if we needed any more bad news, it turned out that if a user of one particular mail system within the University forwards all their mail to Hotmail in a particular way, the resulting bounces caused a mail loop (which really shouldn’t happen, but that’s another story). To mitigate this issue and to avoid all mail being rejected outright, we took the decision for Oxmail to “hold” all mail for a limited number of domains, rather than attempting delivery and causing a bounce to be generated. This applied to hotmail.com, hotmail.co.uk, live.com, live.co.uk, and msn.com; mail to other domains hosted on Hotmail infrastructure continued to be rejected.

Resolving the issue

Our postmaster team raised a ticket with Microsoft’s email support, while within the security team we mailed our security contacts within Microsoft (messages to Microsoft’s own corporate email system are unaffected) explaining the situation. In particular we stressed that the original cause had been dealt with, and asked whether anything could be done to expedite lifting of the block. Others within OUCS communicated with our sales and more senior contacts. We continued exchanging emails long into Tuesday evening (many of our contacts being on Pacific Time), being encouraged to make use of Microsoft’s reporting mechanisms. We continued to press the matter on Wednesday, particularly as to how long the disruption could be expected to continue, but without much more information being forthcoming. We were particularly unimpressed with the following response via the official support channels:

We are unable to take action for the IP’s … because of their poor reputation within our system. I do apologize if I am unable to provide any details about this situation since we are not a liberty to discuss the nature of the block.

Despite some encouraging signs we remained blocked throughout Friday and into Saturday. Indeed at one point we were told that we should have been unblocked on Thursday, but Oxmail logs proved to the contrary.

It was not until Saturday evening, some time around 8pm BST, that mail once again started to flow. We continued to hold mail until we could monitor its release during working hours on Monday; owing to the large volumes of queued mail and rate-limiting on Hotmail, it took the best part of 24 hours for it all to be delivered. Even several days later, message delivery to Hotmail may be significantly delayed owing to rate-limiting.

Why didn’t you just change IP addresses?

We did consider whether we could switch IP addresses away from the blacklisted ones, or somehow have a special-case diversion for traffic to Hotmail. Such things are not necessarily trivial to change in a complex environment, and we must be wary of unforeseen issues making problems worse. Microsoft’s reputation-based system may not take kindly to a sudden deluge of email from a previously unseen IP address, and attempts to work around the blocks may not have gone down well with those persons we were asking to fix the problem. Furthermore there is the issue of precedent – will we be expected to take such action the next time there is similar disruption to email, even if considerably smaller in scale?

What can be done to avoid such problems in future?

The short answer is that we don’t know. We remain unsure as to whether the blocks were cleared through manual action by Microsoft staff, or simply because after five days of no email traffic, our “bad reputation” in their system expired sufficiently for mail to flow again.

We are happy to admit that the original problem was of the University’s making: as an institution, we screwed up; the fact the blame lies outside OUCS is neither our customers’ nor Microsoft’s concern. But we did what we could to fix the problem as soon as it was identified. The subsequent Microsoft blocks did nothing to prevent further mails from the offending list being delivered to their users, and merely caused major disruption to legitimate emails and business relying on those emails getting through. There has been resulting reputation damage both to Microsoft and to the University. These days admissions, alumni relations and recruitment are all heavily dependent on external email systems including Hotmail.

We cannot help but wonder what would have happened if we had followed the approach of some universities and outsourced student email to Live@edu while keeping staff email inhouse. We could easily have ended up with our staff unable to contact students by email at all for over five days – had it happened in the middle of term the consequences would have been extremely severe.

As a final note, we’d like to thank all of those who have assisted in dealing with this frustrating and time-consuming problem, and to all who have had to bear with us while the disruption has continued.

Posted in Email | 2 Comments

The Price of Phish

Over the last two months OxCERT have been inundated with compromised University accounts being used to send spam.  So how are so many accounts being abused, what are the risks, and what is the cost to the University?

Phishing Form

Let’s deal with the how first.  It appears that the vast majority of the accounts we see abused for spamming purposes, are compromised due to users replying to phishing emails or filling in phishing forms.  What is rather depressing is the fact that in about 99% of these cases, the attackers have gone to little effort to masquerade as genuine University services.  Sadly it seems that users are more than willing to enter their credentials into any old random web form, or reply to any email that asks them for their password. The other prime suspects are key-logging malware such as Zeus or, perhaps, compromised websites where University members have used the same password and email address to register for some third party service.  It certainly seems pretty plausible that users who enter the Oxford SSO password via some Internet cafe, hotel room or other untrusted device around the globe, may fall victim to having their account accessed by the bad guys.

http://www.flickr.com/photos/kalexanderson/5526509587/sizes/m/in/photostream/What about the risks associated with phishing? For the University as a whole, compromised accounts – at best – don’t look good and spam runs could have an adverse effect on the reputation of the University.  However, more importantly, they could lead to University mail servers being blacklisted which would have a knock-on effect to all other users of the service.  In fact, University users were unable to email users of a major service provider for several days as a result of one recent incident. Additionally, for the users in question, they can (and do) have their important emails read and deleted, they can be inundated with bounces and replies (causing a denial of service) and, of course, they will have their account disabled until the incident has been dealt with.  This can sometimes be a considerable period of time, particularly if users are in the far corners of the globe.

Perhaps the most immediate impact to the University however, is the cost associated with dealing with phish. Last year, following a spamming incident OxCERT calculated that the time spent dealing with spammers in OUCS amounted to approximately 24 hours per incident.  That is, at least, 3 working days of one person and doesn’t take into account costs incurred by local IT support staff (ITSS), costs to the user, or other potential costs which are harder to put a figure on.  Placing a monetary value on staff time is difficult, especially if the staff are being employed anyway – there is no direct additional cost to the University.  However, working on the basis of a mid-point, grade 7 employee that probably works out in the region of around £360 per phish.  Not much then?  But given that OxCERT have dealt with around 20 spammers in last 2 months that is  approximately £3600 a month.  Admittedly that is more spammers than we’ve dealt with than the entire previous 10 months put together, but that may be because we  have started looking more closely recently.  Either way, if this trend continues that is about £43,000 per year.  That’s easily one full time employee dealing purely with Oxford accounts being used for spam.  It’s perhaps also worth bearing in mind that these costs (while only a very rough estimate) don’t take overheads into consideration, or (as mentioned) the time of local ITSS.  Neither do they take into consideration incidents where compromised accounts are used to access journals or databases worth thousands, and sometimes tens of thousands, of pounds.

If you were looking at this from the point of view of risk, where Risk = Asset Value/Cost * Vulnerability * Threat then it would certainly seem something worth trying to mitigate.  Given the costs involved, the vulnerability of users to hand out their SSO credentials and the fact that we get phishing attempts reported to us every week, it seems like a high risk to the University.  But what can be done?  OxCERT, of course, already take mitigating action where possible.  We can block links to certain phishing forms, prevent e-mail from getting to certain email addresses etc.  But that only works when users are on the University network. There are also some forms which are more difficult to block ( I don’t think we’d be too popular if we sink-holed spreadsheets.google.com for example) and service providers can sometimes be slow in taking down phishing sites.  We do have detective and reactive controls in place in order to spot spammers quickly and significantly lessen the impact on the University, and we blocked in the region of 700 machines, which were infected with key-logging malware, over the last 12 months. Whilst I’m sure not all of these would have been used to send spam, I have no doubt that some of them would.

Yet despite all of these measures, this is a problem which is still costing the University time and money. Surely the real question has to be how we can prevent this from happening in the first place and how we can get users to question the legitimacy of emails they receive and sites they visit.  These are not new questions and are not easy to answer.  I know the vast majority of ITSS do their bit to educate users but is there more we can do?  I don’t have all of the answers, but would love to hear the thoughts and views of Oxford ITSS on this issue.  Can more be done to educate our users and make them aware of the impact of their actions – both on themselves and on the University as a whole?

Posted in General Security | 1 Comment

URL shortening – a threat to privacy?

OxCERT recently received a request for guidance on URL shortening with regards to using such services to re-direct users to sites that may ask for sensitive personal information.  The concern was that the third-party offering the service may be able to track who visits the sites behind the links.  If those sites collect sensitive information is this a risk worth considering?

Privacy is a balance

As with all questions related to privacy then there is a balance between the invasion of privacy and the benefit of what you are doing.  The first question to ask may therefore be “why use URL shortening?”  Of course URL shortening can be useful, for example where there is lots of embedded data in the URL.  However, if there is no need then why bother?  Additionally, have you considered using your own internal redirection scheme if your current URL isn’t memorable enough or sufficiently aesthetically pleasing?

However, this seemed an interesting example of how risks to information can be misunderstood and/or overlooked.  Yes there may be privacy concerns in that the service provider may be able to trace certain activity and obtain information on (for example) the web browser being used, the OS, the referring URL and the IP address of the visitor.  Is this “personal information”? Possibly, if the service provider can trace the IP back to a living individual.  It’s also possible they could associate that information with past sites you visited via the same service (i.e. from the same IP address).  Is this all so bad?  Maybe. If simply having visited a site is considered sensitive then this could be a reason to avoid URL shortening but the risk seems low if there is a good reason for using it.

What would be of greater concern to me is the risk of shortened URLs being used to “phish” the information by directing users to malicious sites.  The problem with URL shortening (in this respect) is that users don’t know where they are being redirected to.  And how many users check the validity of the site behind the link?  OxCERT still witness users filling in phishing forms on third party sites that have made little or no effort to masquerade as a genuine University site so how many would fall victim to a legitimate looking site asking for sensitive personal information if it appeared to come from a legitimate source?  Users can also be re-directed to sites hosting malware or containing other unwanted content so it is important to bear in mind that, to an extent, trust is being place in the URL shortening service provider.  Therefore it is worth considering which service provider you use.  Are they a well-known, reputable provider who are likely to take swift action for example?

Of course there are also preview tools available in order to allow users to check where the shortened URL takes them, but I think that this is only part of a greater problem.  That is one of teaching users to question who is asking for their information and educating them as to how to verify their identify.  When it comes to a breach of user privacy then, checking the authenticity of the actual site being used to harvest the information, seems a much more important message to get across.

Posted in General Security | Leave a comment

2011 FIRST Conference: Friday

Karlskirche, ViennaThe final day of the conference began with Mikko Hypponen from F-Secure exploring the history of PC-based malware, beginning with the Brain virus in 1986, and moving towards the threats posed by ransomware, social network-based attacks. He also looked at some of the newest developments in malware, including the ways criminals online are able to circumvent new security devices introduced for example to protect online banking. Some of these techniques are quite sophisticated, for instance SpyEye addons for mobile phones to intercept SMS authentication messages from online banking services. The talk concluded with a look at some of the particular issues organisations face when dealing with targeted attacks, and the risk that some of the processes we may follow in general may do more harm than good with a targeted attack.

This was quickly followed by a talk from Mwende Njiraini on Kenyan cybercrime. This explored the growth of telecommunications in Kenya from a highly regulated monopoly to a much more open playing field, including heavy use of Mobile telecommunications. Particular threats involving mobile phone banking, pensions, insurance and healthcare were explored, alongside some of the scams that were commonplace.

The first of the afternoon’s talks was entitled “The road to hell is paved with best practices”, taking an informal look at various maters widely taken to be best practice but whose merits may be dubious. The two presenters took opposing stances on each topic, and it was noticeable for some items that it was hard for them to find much to say in favour. The range of topics covered included that old OUCS stalwart of password complexity and expiry, use of cryptography, limiting the number of administrators of a system, forbidding remote access, blocking pings, removing version banners on software, and removing “unnecessary” software
from systems.

The general conclusion was that while a few “best practices” achieve little or nothing, most have a grounding in common sense. However, everybody’s situation is different, and it is necessary to consider the benefits and risks in each case, summed up as “use your brain”. Blindly imposing restrictions for compliance or to tick boxes with auditors is the way to breed dissatisfaction and resentment, and may well worsen security and productivity.

The second talk was by Stefan Frei of Secunia, discussing the challenges of keeping software up-to-date, in particular the scale of the problem in tackling third-party applications. With 50% of Windows users having more than 66 programs installed from more than 22 vendors, keeping everything up-to-date is a daunting challenge. Even among the top fifty Windows programs, half are not from Microsoft but from thirteen other vendors, and have several times the number of announced vulnerabilities as the Microsoft software. All too frequently, the availability of a patch does not mean that it has been applied. The sheer effectiveness of patching as a security measure cannot be underestimated and for some may be more cost-effective than much more “advanced” security countermeasures.

While this was not a marketing talk, Secunia do make available, at no cost for personal use, their Personal Security Inspector, a means for users to keep third-party applications up-to-date without having to go through each vendors’ update mechanisms. While not something which we have investigated in detail, we feel that this is something worth promoting to end users as a means of staying safe, in addition to application of the regular Microsoft updates.

At an enterprise level the situation is more challenging: updates need to be managed centrally, and there is the possibility of updates breaking interoperability with critical business systems. Where resources are limited, efforts need to be directed towards addressing the most critical updates. The exposure to risk can be greatly reduced by diverting resources towards the most critical updates as opposed to those for the most common software within the organisation. The difficulty here is in determining which updates are most critical, although where CVSS scores are published, these will greatly assist.

The other afternoon session began with Mikko Karikyto of Ericsson PSIRT talking about threats to mobile telecommunications networks and life in a PSIRT. His team is a Product Security and Incident Response Team, dealing with the security of products, rather than any local incidents at Ericsson. As such he is frequently dealing with incidents coming in through their technical support channels and has to deal with customers who may be unaware that they have a security issue (a problem we are all too aware of). He also observed that the vast majority of incidents related to customers who had failed to secure equipment by for instance changing default passwords (again a problem with which we are familiar), although occasionally they do see vulnerabilities in their software.

This session continued with a talk from Andreas Schuster of Deutsche Telekom looking at how they make use of timelines to aid analysis of incidents. It was observed that using a timeline can often help to identify what was the root cause of a compromise and what was merely an effect of the compromise. The speaker also demonstrated some of the tools that can be used to produce such timelines. This sort of data may be useful both for us in improving our incident handling, but also as a method for visualising some of the output from our forthcoming malware analysis laboratory.

As the conference ended, there were reports of mysterious disappearances from the Hilton's bathrooms, rumoured to be the work of ninja figures given out by one of the exhibitors

We all gathered for a brief final session, consisting of the award of prizes (nothing for us this year, unfortunately), some brief farewell speeches, and, as has become traditional, a summary from Masato of his success in trying to meet every conference attendee, once again a new record. After a reminder to book for next year’s conference in Malta, we were given the instruction to “go home!”, although many participants were staying a little longer to appreciate the many delights of this fascinating city. In all, an excellent conference, and we look forward to attending again the future.

Posted in FIRST Conference | Leave a comment

2011 FIRST Conference: Thursday


Thursday morning commenced with a presentation from Marc Feidt of the European Commission, talking about the EU’s vision for a more IT enabled society in order to enable economic growth, a reduction in poverty and their climate change goals. They see the need to increase trust in Network and Information security, and are committed to the creation of both an EU institutional security team (iCERT), and to mandate all countries to have a country-wide CERT.

Michael Moran from INTERPOL gave a talk entitled “Online exploitation of children – What’s it got to do with me?”. He began the presentation on this difficult subject with a fairly light-hearted consideration of pornography in general, its increased availability in recent decades, especially since the rise of the Internet, and, to the amusement of the audience, played the song “The Internet is for Porn” from the musical “Avenue Q”. He then turned to the more serious side of pornography, where participants are not all consenting adults but involve those below the age of consent, primarily under ten years of age and often much younger. He illustrated this part of the talk with some shocking statistics and some disturbing images. While far less graphic than many in this area that law enforcement will encounter, this was one presentation where the slides would most definitely not be available for download.

Once again the need for international co-operation was made: even a small amount of material gathered may provide a vital clue in identifying and prosecuting the persons involved. There is a very real human cost when evidence is uncovered but then swept under the carpet, for instance employees being quietly dismissed rather than bringing in law enforcement; some employers have been sued for negligence as a result. In any investigation is is essential to work within the law; IT staff within the University should ensure they are familiar with University guidelines on the handling of illegal materials.

Thursday’s afternoon sessions commenced with the Blitzableiter team exploring mechanisms to counter flash exploits. We are well aware that Adobe Flash is a very common infection vector on desktop systems and as such any ways to protect users are of interest. Their technique is interesting as it presents a normalisation mechanism to strictly validate flash files. Apparently this turns out to be effective at mitigating almost all of the exploits that have been actively exploited over the past 18 months. This technique can be used either as a plugin to Firefox’s NoScript plugin, as a Squid Addon or in various other manners.

This was followed by a talk from John Kristoff of Team Cymru looking at the question of BGP and routing security. This was a talk aimed at a general audience who were not necessarily running BGP based routing on a daily basis. His talk was very informative, explaining in detail how various attacks could be performed (both intentionally and accidentally), and looked at some useful BGP features for dealing with some forms of attacks, including BGP Flowspec, and Blackhole routing. Finally he looked at the question of what future technologies were on the horizon to improve the future security of routing tables.

In the other afternoon stream we followed, the first presentation covered the Stuxnet incident, an incident targeted at Siemens programmable logic controllers used for control of physical devices such as motors and pumps. While the target was probably Iranian nuclear enrichment, and certain governments widely suspected in the initiation of the attack, the presenters concentrated on the anatomy of the attacks and on the detailed analysis conducted by Siemens. While not directly relevant to the University, the talk showed what is possible through infection of standard desktop systems, propagating to highly-specialised systems, where there is not complete segregation between systems. The consequences of a similar attack against critical national infrastructure cannot be underestimated.

The next talk covered the security of the domain name system (DNS). As an old, light, protocol, DNS was originally lacking what many now would see as essential security requirements, and some are only now being added on to the underlying protocol, for instance DNSSEC. DNS increasingly has a critical role everywhere, and a successful attack against DNS could be successful against critical infrastructure nationally or internationally. The speaker felt that insufficient attention is given to the security and stability of the DNS and is keen for greater monitoring of the “health” of DNS and for the establishment of a dedicated CERT for DNS.

Presentations finished mid-afternoon in order to make time for the Annual General Meeting of FIRST. This was immediately preceded by a brief PGP keysigning party, something of a rushed affair for those attending the AGM, but essential in building up the “web of trust” mentioned in one of the previous day’s lightning talks. Compared to some past AGMs, this year’s was a relatively straightforward affair, and the main item on the agenda was the election, with five of the ten places on the steering committee to be filled. We were pleased that for the first time in some years, a representative of a university CERT was elected to a committee that has often been heavily dominated by representatives of major corporations; our congratulations go to Margrete Raaum of the University of Oslo.

Posted in FIRST Conference | Leave a comment

2011 FIRST Conference: Wednesday

Rathaus, Vienna

Wednesday’s sessions began with a very engaging, entertaining and informative talk from Rob Thomas, founder of Team Cymru, a group with which we co-operate closely. The talk commenced with a brief overview of his history, how he became involved in the process of hunting botnets and the importance, as he sees it, of working in an area that you are enthused about. His talk continued looking at important things for security professionals to remember, including that of working well together and avoiding the risk of groups of criminals being better at collaborative working than the people trying to tackle them. This talk was particularly noteworthy as Rob informed us that he does not intend to present to large audiences such as this in the future.

This was quickly followed by a talk from Rajeev Kumar working as a Commissioner of Police in the Government of West Bengal. He was looking at his experiences working as an investigator and dealing with some of the many threats in India today, including terrorism and conflicts with neighbouring countries. As others this week have described, co-operation with other organisations and with commercial ISPs is vital for effectively dealing with the threats.

As in previous days the morning more general sessions were followed by a choice of more specialised and technical talks. we were faced with the luxury of all three of the parallel sessions appearing to be interesting to us. This faced us with a difficult choice as to what talks we wished to see.

One feature of security conferences, especially when a well vetted audience is present is that some talks may be confidential, in which case we are not able to write about them. One such talk took place looking at the Bredolab takedown, mentioned tangentially in several other presentations but now described in detail.

This was followed by a talk entitled “Listening to the Network: Leveraging Network Flow Telemetry for Security Applications”. On the face of if this looked like a very useful talk as we make heavy use of flows in our incident detection. Unfortunately, this talk turned out to be fairly basic and did not cover anything we do not already do, and covered a selection of tools we already were well aware of and have previously considered and rejected.

Finally in this session, was a talk from Denis Maslennikov of Kaspersky Lab. He was looking at some of the issues relating to SMS based cybercrime in Russia. This included cases of extortion involving encrypting infected system’s data and requiring a premium rate SMS to unlock the files. We were also shown cases of mobile malware sending premium rate SMSes via encouraging users to install an application that does not make it clear that it is generating such traffic. These processes are made possible by a lack of good controls over the issuing of such numbers. In many cases we were told it was possible to obtain these without revealing any personal information.

Within the other track we followed was another talk from Kaspersky Lab, this time by the founder himself, Eugene Kaspersky. He gave a talk describing the evolution of cybercrime and the current problems in tackling it, before moving on to describe a possible new approach: essentially cloud-based tracking of malicious activity as an add-on to traditional antivirus. If a large number of systems report back on their activities, it should be possible to detect patterns which lead to malicious activity, for instance determining that hosts get compromised after visiting a particular URL. The information can then be fed back to other PCs in order to warn them that the location is bad and should be avoided. He fully accepts that it would not defeat determined experts, but might make the difference to many cybercriminals as to whether or not their activities can be profitable. We feel that some caution is needed in order to avoid leaking unnecessary amounts of information on personal browsing habits and suchlike, and that in time exploit kits may be available that can defeat this method, but it is certainly worth a try.

Jeff Williams of Microsoft gave an excellent presentation on Microsoft’s involvement in responding to major threats over the years. He started off describing Microsoft’s response to Blaster and Slammer, which struck back in 2003, and even today have not been eliminated entirely. Responses to threats have evolved rapidly, and often now involve co-operation between many players, for instance high-profile depeerings of ISPs hosting large amounts of malicious activity, the unprecedented co-operation with DNS registries around the world in order to neutralise Conficker, the increasing use of the legal system to take down systems causing Microsoft (and others) real financial harm. Microsoft are working to improve facilities available to customers, for instance in the form of the Malicious Software Removal Tool for end-users and through Smart Network Data Services for ISPs. Copying Apple, he ended his presentation with a “one more thing…”: calling for a concerted effort over the next year to clean an entire country of malware. Several interesting questions followed, and we were pleased to be able to follow up some points further through a very useful conversation with Jeff later in the day.

Finally we heard about the European Grid Infrastructure and the measures taken to maintain its security. The highly devolved nature of the Grid is not so far removed from that we face ourselves with IT in Oxford. Some things are easier: for instance there is at least scope for some centralised monitoring of end nodes, but other problems arise: language barriers, differences in local legislation, problems with timezones. Ensuring full co-operation is a challenge, and much comes down to good working relationships with sites, kept up through occasional attack simulations. Policy frameworks allow for the ultimate sanction of disconnection from the Grid, but just as with us disconnecting a unit completely, this is very much as last resort. To date, the vast majority of security incidents have involved compromised accounts, typically through brute-forcing our stolen credentials; no central power exists to force sites to deploy stronger technologies.

The afternoon’s presentations were rounded off by a series of lightning talks, each run to a strict time limit of no more than five minutes. These are always popular, and as usual covered a wide range of topics, including:

  • Volunteers around the world, OxCERT included, operate a series of network monitoring “pods” for the Dragon Research Group, an offshoot of Team Cymru; they are now looking for pods to gain IPv6 connectivity in addition to IPv4.
  • Fortune 2000 companies leak a considerable amount of information through public materials on their websites. These included information on internal usernames, network information, printers, software versions in use (the majority were found to be using potentially vulnerable Microsoft and Adobe products), and use of robots.txt files on webservers as access controls.
  • OxCERT’s very own David Ford described the Darknet Mesh Project, allowing sites to share information gathered by their darknets relating to incidents at other participating sites.
  • Data visualisation, including using HTML5 web tools to visualise network flow data in three dimensions
  • Passive DNS data: gathering information sent to DNS resolvers in order to be able to track how DNS records change over time, something we find very useful for our own investigations.
  • Methodologies for handling different types of incident and the production of a series of checklists – something that we are increasingly feeling a need for in handling an increasing variety of ever more complex incidents
  • FIRST members were encouraged to sign each other’s PGP keys at Thursday’s keysigning party in order to build up a “web of trust”
  • Masato from Hitachi covered no fewer than three separate topics: SQL injection attacks, what to do in the event of an earthquake, and his annual project of meeting every attendee of the FIRST conference, always with a novel difference.


In the evening, after a drinks reception at the hotel, we moved by coach to the magnificent Vienna Rathaus (town hall) for the conference banquet. Unusually for an event at an IT-related conference, formal attire was encouraged, and we certainly seemed to smarten up well. Musical entertainment was provided in the form of an excellent classical crossover group, whose repertoire ranged from Mozart to Michael Jackson. Upon returning to the hotel, some were prepared to continue partying well into the night, but after a chat with our friends from Team Cymru, decided to call it a night.

Posted in FIRST Conference | Leave a comment

2011 FIRST Conference: Tuesday

Tuesday’s sessions commenced with a look at rogue pharmacy sites by Brian Krebs, well known for his column Krebs On Security. He explored the history of some of the major players in this black-market industry, including some of the other industries that some of them have been involved. He then went on to explore what efforts are being taken by law enforcement to track and take action against these gangs, the steps that have been taken to prevent people from purchasing from them, including the futility of attempting to influence  some of the less well respected banking institutions when it comes to the question of who they permit as customers and what questions they ask. One surprising observation was that these sellers often had very high quality customer service, and were at least as likely to take steps to deal with missing goods/incorrect shipments as more reputable vendors.

The second session of the morning was given by John Stewart, Chief Security Officer of Cisco, a frequent presenter at FIRST conferences. His interesting and entertaining presentation looking at incidents of the past, and trying to explore the question of whether we are winning in the fight against those who wish to cause disruption and harm.

After lunch, we were again given a choice of presentation tracks to follow and again we made the conscious decision to separate and focus on two different areas.

In one stream, we were presented first with a talk from ECSC (an educational security research team) in Korea examining their process of collating data from a wide variety of sensors based in different universities across Korea. Their concept had a lot of interesting features, and their deployment towards separate networks is both something that could be explored at a cross University level, or at the level of the University. One thing the speakers were not sure of (because of the way that the system was devised) was to what extend to collaboration has allowed rules to be created that would not have been possible as a single site. Of course within the UK any system collecting data centrally like this is liable to require careful thought from a legal perspective.

Second to come to the stage was a hastily prepared talk from the US based ICS-CERT, the speaker was standing in at the last minute after a presenter was unavailable. Nonetheless the talk gave a fascinating insight into the world of industrial computer systems, reminding us that the vast majority of these were designed with the view that they would never be connected to anything but an isolated network, but despite this inevitably almost always are. The vast majority of these systems are still developed using practices that were dropped more than 10 years ago in other sectors because of the risks they pose. It was also pointed out that the challenges in patching many industrial systems cannot be underestimated.

Finally within this track, we had a talk looking at the risks of the “extended enterprise”. This was not a term with which we were previously familiar, but turns out to encompass a wide variety of people with whom an organisation has some form of business relationship which might involve transmitting data to/from. This can include business partners, auditors, legal advisors and the ever present customer.

Would you share your personal information with this person?

Within the other streams, one of the talks was again by a Cisco employee, Patrick Gray, who comes from a background in law enforcement with the police and the FBI.  As with the morning’s talks this wasn’t overly technical in nature, concentrating on the human threats to organisations and stressing the need for constant user education in order to create a “human firewall”.  Social networks are increasingly used both by businesses and individuals, although a show of hands revealed a substantial proportion of the audience have yet to join any of Facebook, YouTube or Twitter.  Many Facebook users are far too keen to friend anyone who asks, citing a study which revealed that 46% of randomly-selected users were willing to grant full access to their profile from a plastic duck and from a cat.  The information gathered can be invaluable in social engineering attacks: he cited a case of a senior banker being stalked by criminals who, having found her high school yearbook, picked a classmate without a Facebook account and create a fake presence; a friend request was duly acknowledged and subsequently they invited her to identify further classmates in a photo.  Only thing is, clicking on it resulted in a malware infection and a vector into the bank’s network.  Targetted intrusions are far too common and not limited only to major corporations: he cited several pages of examples from public and private sector, all from the past two weeks.

A representative of JPCERT spoke about their Cyber Clean Center Project, started in 2005 with the aim of reducing the number of infected broadband users within Japan.  Using honeypots to identify infected systems, and with the co-operation of Japanese ISPs, users were contacted informing them of the infections, inviting them to download custom software for disinfection.  Initially this saw a relatively low success rate, but with improved publicity and education, things improved.  Nevertheless there were doubts as to the overall effectiveness of such an approach – similar tactics can be (and indeed are) used by scammers to persuade users to install malicious software.  Additionally there were some instances of the disinfection tool rendering systems completely inoperable, to the dissatisfaction of the users.

On a different track was a talk on preparations for the 2010 Winter Olympics in Vancouver, with the aim of minimising the risks of physical or information security breaches. This requiring a huge amount of planning over many months and co-operation among a large number of authorities and organisation across Canada, and several exercises staging multiple attack scenarios. The event passed off without major incident: a few targetted emails, standard malware such as Conficker and ZeuS, and an instance of criminals taking advantage of a fatal accident in training to attract people into viewing video of the incident, actually serving malware in the form of a fake video codec. The lessons learned are being carried forward for those planning for the London Olympics next year.

Rounding off the presentations was a special session on the response from security teams to the Japanese earthquake, tsunami and subsequent radiation leaks in March.  After a moment’s silence in respect of the many victims, the speakers, mostly based in the Tokyo area, each had their own tales from the day itself in the face of disrupted tranportation and communications.  In the following days, initially there was little specifically for CSIRTs to do: the immediate priority was business continuity and restoring communications as far as possible.  Many organisations were forced to use unorthodox methods for a time – institutional bans on sites such as GMail and Facebook can suddenly backfire when the usual channels are down, while remote access channels suddenly became essential.  Inevitably various rumours, hoaxes and scams were observed in the following days and weeks and considerable effort was expended in protection of those affected or genuinely trying to offer assistance.  It was acknowledged that sooner or later there will be other disasters on a similar scale or worse, and, while no-one can prepare totally for such incidents, much more can be done to increase readiness.

Official events for the day were wrapped up with an opportunity to speak to exhibiting vendors over drinks.  These ranged from large organisations such as BT and RSA to niche players offering specialist products: we were particularly interested in speaking to a couple offering malware analysis systems operating along similar lines to our own malware analysis system currently under development.

Posted in FIRST Conference | Leave a comment