OnTheHub renewals…

When the University first signed up for the OnTheHub service plenty of University members went there to obtain free or discounted software. When asked to register, they used their official email address – after all, why wouldn’t you?

Some time later, as Nexus’ migration to Office 365 project kicked off, it became apparent that this couldn’t continue: we needed to reserve all of the University’s email domains/suffixes for use in our official Office 365 tenancy. Users who were using their University email addresses for any other interaction with Microsoft were creating a ‘viral tenancy’ with no administrative control, while simultaneously preventing us from adding that domain name to the official tenancy. And without the ability to add that domain name to ‘Nexus365’ that college or department would be stuck for all eternity on a tiny in-house Nexus mailbox.

It was around a year ago we identified this problem and asked Kivutu, who administer OnTheHub, to stop accepting registrations from University email addresses. Now users of OnTheHub are asked to register with an email address in the format first.last@oxforduni.onmicrosoft.com. Our help text was also updated to reflect that change.

A year on from that we now have a new problem. OnTheHub users are asked to periodically renew their licence, to prove that they are still members of the University and thus entitled to continue using that software.  It’s been a year, so University OnTheHub users are starting to relicence their software, as requested.

To verify entitlement, users are asked to log in…
The problem is that after all this time many users have forgotten about that oxforduni.onmicrosoft.com address – many of them have been trying to log in with their official University email address. Because we’ve been adding all of those email domains to the official Nexus365 tenancy, and we’ve federated it via Shibboleth to use single-sign-on addresses, Microsoft’s logon page sees an address that belongs to us and ‘helpfully’ sends the request in our direction.

Your end user sees a standard SSO logon, knows what to do with that, so logs in. However what they get is not OnTheHub. What they’re seeing is, in effect, a sneak preview of Nexus365. But it’s not usable in any meaningful sense as what they’re seeing is a not-yet-migrated and not-yet-licensed example of Nexus365. All of the new exciting toys are visible but none of them are usable until we are able to start migrating folk across.

It would therefore be a great help to us, and a useful reduction of the support burden on the helpdesk, if you could advise your OnTheHub users to look up their oxforduni.onmicrosoft.com alias and use that to re-licence their Microsoft software downloads. It will have been sent to their official University email address so should be searchable within their mailbox. Finding and using that address will  ensure that a greater number of people can continue using their software without delay or (additional) confusion by re-licensing at their first attempt.

 

 

 

Posted in Uncategorized | Leave a comment

Nexus365

The Nexus team are trying things out. My mailbox is now just a tad larger than it was yesterday.

Isn’t this number a lovely sight? 🙂

Posted in Uncategorized | Leave a comment

BlackBerry decline continues

The Nexus Blackberry Enterprise server is licensed for 378 BlackBerry devices. And, back in 2010, we needed every last one of those licences. Today things are very different. After a lengthy process contacting users and removing those who had given up their BlackBerries, there are now only 27 people still registered on the server. Of those, only 25 have made contact during the last month. This means that active usage has more than halved just since July 2016, when we counted sixty active users.

Nexus’ BlackBerry server software does not support the current range of BlackBerry handsets. In order to support these newer devices our server software would have to be upgraded. More importantly the version change requires users’ devices to be re-licensed (at cost). The expense and effort required to do this does not make good financial sense for a system hosting so few users. The service requires two BlackBerry Enterprise Servers, for redundancy, and a back-end SQL database. All of these need monitoring, updating, backup, and general fettling. For 25 people this routine upkeep doesn’t represent a good return on the effort required. The department would struggle to justify provision of any new service from which fewer than thirty University members would benefit and which only supports obsolete devices.

The intention is that Nexus’ BES service will be retired ahead of the migration to Office 365. All current BlackBerry server users (i.e. anyone who bought a licence and has gone through the server activation process) should plan to replace those devices as soon as possible. New Blackberry handsets can still be used to connect to Nexus but should be configured to connect only via the ActiveSync protocol. If you are a BlackBerry user who use BIS, or ActiveSync, to connect to Nexus you will be unaffected regardless of whether we maintain a Nexus BlackBerry Enterprise Service.

 

  • 81% are using a device that’s over five years old.
  • The other 19% have 9720 devices (which were first introduced in the summer of 2013).
  • The oldest devices in use – an 8310 and an 8800 model – date from 2007.
Posted in Uncategorized | Leave a comment

Nexus’ BlackBerry Enterprise Server

Early last week an email was sent to all Nexus BlackBerry users whose devices had not made contact for at least three months. The responses (or non-delivery reports!) from those people have allowed a further clean-up of inactive users.

Following this work the number of users remaining on Nexus’ BlackBerry Enterprise Server is now down to 33. A further four names look eligible for removal, since those users’ BlackBerries haven’t made any connection to our servers during February. For now, however, they remain listed as active users.

Maintaining two BlackBerry Enterprise Servers, and a SQL database,  for such a small number of users does not represent a good return on the effort required. The department would struggle to justify provision of a new service from which fewer than thirty University members would benefit.

Nexus’ BES service will therefore be retired ahead of the migration to Office 365. Current BlackBerry users should not replace old devices with new Blackberry handsets unless they propose to connect them to Nexus only via the ActiveSync protocol.

https://help.it.ox.ac.uk/nexus/mobile/blackberry

Posted in Uncategorized | Leave a comment

Office 365, viral tenancies, and side effects

All of the University’s domain names have to be added to Office 365 in order to use them with the service. This process has been hampered both by the sheer number of domains in use here and Microsoft’s decision to allow users to self-register for the service.

Self-registration is described in the documentation as making life easier for the administrator. And if all you’re worried about is ensuring that your users can access Office software as easily as possible then that aim is achieved. However domains with self-registered users become, in Microsoft’s parlance, a ‘viral tenancy’ – anarchic and uncontrolled. To bring them back under central control one must first prove ownership of the domain. This requires:

  • A valid email address within that domain
  • Following a link in a confirmatory email sent to you.
  • Filling in a registration form.
  • Skipping the ‘invite other users’ message.
  • Answering the ‘become the admin’ invitation and collecting a verification text string.
  • Adding the text string into DNS.
  • Cancelling the ‘provide admin contact details’ message.
  • Verifying that the string found in a DNS query matches the one they provided.

Only once all of that has been done can you login to Azure via Powershell and disable any further self-registrations from taking place:

Set-MsolCompanySettings -AllowAdHocSubscriptions $false

In an organisation with, say, two domain names this isn’t too onerous a task. But when you have several hundred domains the lack of an automated way to do this becomes a trifle wearing. Each of these reclaimed domains can then be added into the official tenancy but this too requires a DNS text string to be generated, added, and verified for each one. It is the manual and repetitive aspects of this process which are taking time but we hope to have all University domain names under the central Office 365 tenancy by the end of October 2016.

Unintended Consequences

This process has also produced some other unexpected side-effects – many Nexus users will have used their University email address for things like XBox Live or Visual Studio accounts. Once the domain name has been formally ‘claimed’ for use in Office 365 it is no longer available for ad-hoc registrations for these other services.

"You can't sign up here with a work or school email address. User a personal email..."

Sign-up denied

 

We therefore recommend that all Nexus users ensure that they always use a personal email address when registering for a non-University service.

 

Posted in Uncategorized | Leave a comment

RDP: “The connection has been lost.”

This message was appearing a little too often for my liking:

download

“The connection has been lost. Attempting to reconnect to your session… “

In many cases this problem is caused by a feature known as Auto-Tuning. This is supposed to continually adjust TCP/IP receive window size based on the network conditions at any given moment. But on a less-than-perfect network this can cause time-out issues.

The TCP receive window size is the maximum amount of incoming data that can be buffered at once on the receiving side of a connection. The sending host can send only that amount of data before waiting for an acknowledgement and then a “receive window” update from the receiving host.

The TCP/IP stack nowadays tunes itself with larger default window sizes than it used to. Instead of using a hard-coded default value, TCP automatically adjusts the window size – beneficial during bulk data transmission by reducing the number of segments sent with large amounts of data.

Auto-Tuning continually determines the optimal receive window size by measuring bandwidth delays and the application’s retrieve rate. In an ideal world this means excellent performance even with changing network conditions. In the real world this means occasional timeout errors…

This doesn’t just affect Remote Desktop – some older version of Outlook used to get error 0x800CCC0F (“The connection to the server was interrupted. If this problem continues, contact your server administrator or Internet service provider (ISP).”) for the same reason.

What can be done?

To tweak your auto-tune settings:

Run CMD as an administrator and enter this command:

netsh interface tcp set global autotuninglevel=restricted

If you later wish to reverse this setting:

netsh interface tcp set global autotuninglevel=normal

To see what you’ve got right now:

netsh interface tcp show global

The permited values for the AutoTuningLevel parameter are:

disabled
Sets the receive window at the default value.
highlyrestricted
Lets the receive window grow beyond the default value, but does so very conservatively.
restricted
Lets the receive window grow beyond the default value, but limits such growth in some scenarios.
normal
Lets the receive window grow to accommodate most scenarios.
experimental
Lets the receive window grow to accommodate extreme scenarios.

N.B.  The experimental value may decrease performance and should only be used for testing.

 

Posted in Uncategorized | Leave a comment

RIP Smartscreen

Microsoft have included spam-filtering in Exchange for many years, under the SmartScreen name. But from 1st November that will change: there’ll be no more updates and the feature won’t be included in new versions of Exchange Server.

Updates for SmartScreen client-side spam-filtering for Outlook on Windows will also be ending, although SmartScreen will remain current in Microsoft’s web browsers. Despite sharing the name the in-browser feature protects against malicious websites.

If you’re using SmartScreen now the existing definitions you have will still be there, and will still work, but there won’t be any more updates to them after 1st November.

The motivation for this seems to be twofold – the blunt instrument of applying a spam-confidence level to an email isn’t very effective, and Microsoft’s online protection offering does a far better job. Realtime filters can react faster to spoofed email, hijacked accounts, and user feedback. Coincidentally Exchange Online Protection is chargeable for on-premises users, perhaps as a further incentive to move to Office 365.

There is no net effect on Nexus from this change (Oxmail does the job of spam-scoring for the University) but, as many of you know, messages categorised as legitimate can still end up in ‘junk’ because of Outlook applying filters – it’s why we’ve always advised that users set spam filtering via the SCL and not via Outlook’s SmartScreen. The absence of future updates mean that client-side spam filtering will only become less effective over time.

In the longer term Nexus’ move to Office 365 should ensure that Exchange Online Protection has an opportunity to take over spam-filtering responsibilities.

Posted in Uncategorized | Leave a comment

The New York Times on email chains

This is some excellent advice on the subject of “When I’m Mistakenly Put on an Email Chain Should I Hit ‘Reply All’ Asking to Be Removed?”
https://twitter.com/bydanielvictor/status/771698039908CrWfRgTXgAEQHJE[1]

Posted in Uncategorized | Leave a comment

“Why does Nexus send me spam?”

We have had a spate of spam making it into Nexus mailboxes recently, provoking a flurry of discourse on the ITSS-Discuss mailing list. To be able to stop spam more effectively there are some checks that IT Support Staff can do with their users prior to logging support tickets.

Spam detection isn’t as simple as virus/malware detection: there isn’t the same definitive certainty one gets when a virus’ signature matches a message’s attachment. To conclusively determine if a message is spam requires an understanding of context and language that is currently still far in excess of a humble computer’s capabilities. There is also a personal aspect – one person’s spam message is another’s crucial information. So spam isn’t deleted, no matter how certain we can be that that’s what it is. Instead we allow you to personally set your own tolerance to spam. If you do this correctly it will determine what goes into ‘Junk’ and what goes into ‘Inbox’.

Firstly though, here’s what is going on with incoming email:

ON ARRIVAL AT THE UNIVERSITY
New messages arrive at OXMAIL and are scanned. Details of Oxmail’s processes can be found here: http://help.it.ox.ac.uk/email/scanning/index and here: https://help.it.ox.ac.uk/network/smtp/relay/index

OXMAIL then sends messages on to their destination, which might be Nexus, forwarded to a personal email address outside the University, or to a departmental server.

ON ARRIVAL AT NEXUS
The spam score assigned by OXMAIL is translated directly into a Spam Confidence Level, or SCL. Each asterisk in the X-Oxmail-Spam-Level header is counted: the total becomes that message’s SCL. So this example ‘X-Oxmail-Spam-Level: *********’ would have an SCL of 9.

Nexus is able to automatically move spam messages into the Junk folder of your mailbox before you see them, but only if you tell it to do so. This is a two-stage process:

  1. Turn on the spam filter in OWA. This step often gets overlooked. Without doing this step any setting you apply won’t take effect. Instructions for doing this can be found here: https://help.it.ox.ac.uk/email/filter/index.
  2. Select your level of tolerance to spam. This is done via the SelfReg tool (https://register.it.ox.ac.uk/self/nexus).
    The higher the SCL is that you can accept, the fewer messages will be moved into Junk. The options equate to the following:
    OFF: All messages will go into your Inbox.
    LOW: Messages with an SCL above 7 will be moved to Junk.
    MEDIUM: Messages with an SCL above 5 will be moved to Junk.
    HIGH: Messages with an SCL above 3 will be moved to Junk.

WHY DO YOU TELL US TO TURN OFF JUNK FILTERS IN OUTLOOK? (or other email clients?)

The assessment of spamminess done by OXMAIL is logged, consistent, and repeatable. If two people are sent the same message, and have the same spam-filtering preference set, then the same thing should happen to that message for both people.

End-users’ applications aren’t so clear-cut. Each program can use widely differing technologies to assess spam and the software which scans it isn’t always the same version, even when the application is the same. Furthermore, messages that are moved to Junk by the email client application can’t be differentiated between a user choosing to move a message between folders or their software doing it for them. All that Nexus’ servers see is an end-user request for a message to be moved.

In other words Outlook, and others, can do a pretty good job of spotting and filtering spam after it’s delivered. But we can’t guarantee that your program won’t falsely-identify a genuine message as spam. We can’t guarantee that your experience will match that of the person next to you (do they update as often as you do?). We can’t see in our logs what process decided to move that message between folders.

By deactivating client-side spam filtering we can be sure that the only spam processing that does happen is recorded in message headers.  This gives us a fighting chance of diagnosing any issues with the process. If you do spam filtering in your application you are muddying the waters for us if things misbehave and you want us to help you to troubleshoot.

WHAT CAN I DO BEFORE LOGGING A TICKET?

  1. Check that the user has set spam processing in SelfReg AND has turned on spam processing in OWA.
  2. Review the headers of an offending message to see what Spam Score it has been given.
  3. Verify if the user’s application software, or rules processing, or macros, or any other such process might be doing additional spam processing / filtering.
  4. Ensure that the end-user hasn’t configured a whitelist within their email application that is over-riding the spam-processing values they have set.
  5. Record the outcome of these tests in the details when you log a ticket for us to investigate why you are receiving spam to your inbox.

nexus_itss_spam-processing[1]

 

EDIT 1st August 2016:

In the case of a shared mailbox SelfReg does not currently allow you to alter the spam preference – it is limited to the currently-logged-on user. However the Nexus team can manually apply spam preference values for you if spam becomes an issue within a shared mailbox.

Posted in Uncategorized | Leave a comment

Autodiscover: no longer optional

Executive summary: Outlook 2016 users may find themselves disconnected when Autodiscover is not present for your  unit’s domain name.

Over the last few months we have noticed a rise in the number of Outlook connectivity incidents reported to the helpdesk.  In many cases we are finding that the issue is linked to a combination of two things: the use of Outlook 2016 and the absence of a resolvable autodiscover entry for that individual’s domain name. We know that some units have delayed getting those records created, relying on the fact that you could still get basic Outlook-Exchange connectivity via legacy (unsupported) Exchange 2003 RPC over HTTP. This method no longer works in Outlook 2016 because this version has dropped support for Exchange Server 2003.

This has not been a significant support issue until now. It was around the time that email was being migrated from Herald into Nexus that we first sent a message to IT Support Staff requesting the creation of autodiscover DNS entries.  At that time we were suggesting it to primarily help brave souls with Outlook 2007. Today that advice still stands, but in order to support current, and future, versions of Outlook these DNS records have now become a necessity, not a ‘nice to have’.

The University strongly recommends that each unit/college/department who manage their own DNS should create an autodiscover entry.  This is the recommended solution for both on on-premises Exchange and when mailboxes are in the Office 365 cloud. It therefore takes care of Nexus now and into the future. You should ensure that your DNS includes an autodiscover CNAME or A record for your domain name (such as autodiscover.it.ox.ac.uk) which can be used to resolve all related email addresses. These should point to autodiscoverredirect.nexus.ox.ac.uk.

Once all of that is done:

To improve performance for Outlook 2016 you may also wish to use the following Registry settings which will force Outlook to bypass root domain discovery and redirect to the autodiscover CNAME or A record to resolve the address. This saves a little lookup time by bypassing initial root domain queries.

In the key HKEY_CURRENT_USER\SOFTWARE\Microsoft\Office\16.0\Outlook\AutoDiscover create/modify a DWORD value of ‘ExcludeHttpsRootDomain’ with a value of ‘00000001’.

Group Policy can do this too via ‘Exclude the root domain query based on your primary SMTP address’ setting.

Posted in Uncategorized | Leave a comment