Desktop Services Team

A previous post highlights the way in which Google Chrome operates and the memory it uses. It’s not alone and amongst other things most modern browsers gobble up memory quicker than turkey at Christmas.

We’re in a world where a web browsers are out to get one-up on each other and with more and more services running through them its no surprise that they are becocming resource heavy. A quick test highlights both the amount of memory that the browsers are using and the different ways they run processes. Opening the same four tabs in Internet Explorer, Firefox and Chrome show a noticeable difference in the way they are running:

Chrome has opened a zillion processes, Firefox runs everything within a single process and Internet Explorer opens a single 64bit process when you first open it, followed by 32 bit processes for any additional tabs where it needs them. The tabs I had open in all browsers were our Service Desk, Nexus SharePoint, this blog post and a Microsoft TechNet article. None of these are particular strenuous pages but all three browsers were using a noticeable amount of memory.

By the time other applications and processes are running this memory usage highlights the potential for a machine with 4GB of RAM installed to quickly start to run low on resources. Starting on the 15^th February we’ll be upgrading desktops to 8GB of RAM to help alleviate this issue. Further details are on this blog post.

Late yesterday we agreed and released a Microsoft patch (KB3050265) to all of our managed machines. This resolves a known issue whereby in some circumstances the machines resources get eaten up by a rogue svchost process whilst it is analysing which updates are installed and/or needed. Our testing this week has proved successful and we’ll be monitoring the progress of the updates and any noticeable changes closely.

For a while we’ve been taking a close look at the processes that run on desktop machines relating to Windows Updates. We use a Microsoft WSUS (Windows Server Update Services) server to manage and deploy Windows updates to all CONNECT managed machines. As noted in the previous blog post we’d spotted a specific issue that was occurring on some machines relating to the svchost processes taking a large amount of RAM. This was tracked back to the scheduled checks machines do every day to review their update status against the WSUS server.  We’d been deploying a script to specific machines where we have seen this issue for a while however were also aware that this was very much a short-term fix, with the problems and symptoms often re-appearing on machines.

The Microsoft blurb describes the svchost as “a generic host process name for services that run from dynamic-link libraries”.  In slightly more understandable terms Windows lumps together related system services and runs them as a svchost processes, so it’s not a one-one link between a service and a process but a group of services running together as one process. If you look at the list of processes running on a machine you’ll find a number of these running, some within the user instance and some running at system level behind the scenes. Here’s a screenshot from my PC showing this:

It’s good news for anyone using our desktops with only 4GB of RAM. Hot off the back of our annual PC replacement cycle, over the next three weeks (15th February – 4th March) we’ll upgrading all of our desktop machines so that they have a minimum of 8GB of RAM. This is over 1000 machines spread across at least 34 buildings, totalling over 30kg of additional RAM modules to be installed!

We’re still finalising the plan but we will be attacking large buildings such as University Offices and Dartington House after work, between 5 and 7pm. This is to help minimize the interruption to staff and allow us to quickly complete as many as possible. With the smaller buildings we’ll be asking you for help in confirming access arrangements and the timings. We’re part of the way through contacting individuals that are using machines due to be upgraded so if you think you have a machine with 4GB we should be emailing you within the next week or so.

It should be noted that by no means are we treating this as the resolution for all of the issues that have been reported. However having reviewed the information we’ve gathered so far we can see that a number of machines with 4GB of RAM running at or near their memory capacity. As a result a decision was agreed to get all existing machines up to our current standard of 8GB.

To quickly check the memory your machine has Start button and open Computer from the right hand menu. Along the bottom of the new window that has opened will be listed your PC name and the amount of memory that is installed. In the screenshot below this is OUIT-D004462UST and 8GB respectively.

Further to our previous testing we have continued to monitor changes we have made to the automatic printer deployments. This is currently being tested on a small number of machines to ensure we know the full impact of the changes and we’re set to review this later this week to agree the next steps. So far the testing has been extremely positive with a minimal impact on usability.

Alongside looking at reviewing and filtering Event Logs we are also working on a set of scripts that can pull back useful information from machines. This can give us information such as the CPU and memory usage, the status of services and a few other useful bits of information. Being able to instantly pull this information back from machines will allow us to capture a real-time snapshot of what it is doing to help us build up the whole picture.

Further to our initial investigations we are applying a pair of updates to test machine to address an issue in which system performance can be decreased during scans. This issue has the greatest effect on computers that have a small amount of physical memory. We’ll be monitoring the machines for the next few days to see if they continue to exhibit the same symptoms. We’re concentrating on reviewng a couple of fixes that Microsoft have produced to resolve issues around machine performance relating to the update services.

Further infomation on them is at https://support.microsoft.com/en-gb/kb/3050265 and https://support.microsoft.com/en-us/kb/3102810.

Chrome runs each tab as a separate process running them in complete isolation. This means it can be extremely quick however it does mean that more memory is being used. Even tabs running the background are kept running to allow pages to refresh and be ready if you refer back to them. Other web browsers are also adopting this method of separate processes however we’ve seen a number of instances where Chrome is eating a huge amount of memory. As an example, below is a screenshot of all the Chrome processes and their memory use from a machine which had just 4 tabs open at the time.

There are a number of 3rd party add-ons which can reduce the amount of memory that Chrome uses. However by far the simplest way of controlling it is not to have lots of tabs open at once as well as changing settings such as not reloading all previous running tabs when you start it.

Event logs give a huge amount on information on what is going on within the Windows OS. However filtering out the interesting bits and comparing them across machines can be extremely difficult and time consuming.  We’re in the process of reviewing a handful of 3rd party  tools to helps with this and highlight any common issues we have between machines.

We are spending further time looking at the processes and methods used for deploying Windows Updates to our machines, including using the Microsoft Best Practice Analyser to review the server configuration. In particular there is a process that runs locally to scan the machine and compare what updates it has installed against what are available via our WSUS server which controls and deploys them. This process runs periodically in the background and has been noticed by a number of us that it can consume a huge amount of memory whilst it is running.

Our desktops machines are almost exclusively Dell OptiPlex machines that are up to 5 years old.  This generally puts us in a very good position with relatively few hardware failures however we can’t rule out problematic hardware causing some of the issues that people are experiencing.

We brought a machine used in a meeting room back to our offices to take a further look at it. This was a machine which we’d had multiple reports as running slowly and generally being problematic so it was felt to be a good sample machine for us to review.  One of the tests we’ve been running is a  tool that stress-tests the machine to ensure reliability of the system. This includes running components like the CPU and RAM at 100%, playing high detailed graphics and thrashing the hard disks.  Basically absolutely hammering the machine and seeing if it survives. Having left these tests running for 10 hours the good news is that it behaved perfectly with no scorch marks or fires, much to the relief of my colleague whose desk I had borrowed for the experiment. Gibson Building survived another day.

So, we haven’t found any immediate issues but we’ll continue testing in the background and widen the net to include some performance benchmarking tools. This will allow us to check the hardware performance, comparing the results with other computers with the same components.