Network Development Team

Mathematical Institute

The initial switched /64 connection to the Mathematical Institute was switched to a routed firewall connection this week and as a result I’ve routed the entire /56 that was set aside to them. I think we’ll have to require units to have a routing firewall for the /56 and limit non routed units to a maximum of 3 /64’s until they have a firewall in place, otherwise we’ll have too much configuration to do for so many individual /64’s to be routed on the backbone. With the work complete my contact in the Institute has gone on holiday for a fortnight, which is perfect as I’ve some catching up to do in order to improve their service. The Institute is the first unit in the university to receive a /56 IPv6 connection.

OUCS offices

We deployed IPv6 without router advertisements, setup a single host that was used for normal purposes (generating network traffic) and then completed a security response to a pseudo report of the host being compromised. The test highlighted some behavioural differences between our response tools for IPv4 and those for IPv6 which caused minor confusion (the differences were already documented but we could improve the situation). We also completed our migration of Netdisco to the latest CVS source which means IPv6 tracking to a local switchport is working without issue, that is, we can track a compromised host to an office wallport inside our department building. Outside OUCS we can only track a host to the final connection to the college/department, then the tracking must be internal and is handled by the independent IT staff at the unit – Netdisco is a perfect solution for this if you’re not using it already.

I’m going to talk to the security team again and then should be able to give out some addresses to internal teams in order to encourage some interest in IPv6 enabling other services.

Webcache/Centos Kernel

In order to add IPv6 connection tracking support I’ve built a new kernel for our Centos hosts, using the latest stable version at www.kernel.org. I believe I mentioned in a previous post that IPv6 connection tracking wasn’t present on the stock 2.6.18 kernel (any kernel prior to 2.6.20 I believe) and I’m somewhat reluctant to use more complex rules or wait for RHEL/Centos 6 so a new kernel seems a reasonable solution for us as long as it’s only in place until Centos 6 is available.

The main changes I made over the stock kernel config are

• General setup ->
  • “enable depreciate sysfs features to support old userspace tools” -> I understand that you must enable this in order for the new kernel to boot on the current Centos
• Networking Support ->
  • Network Options
    • enable “the ipv6 protocol” and enable “source addressed based routing” in the submenu (I would make all the others in the submenu available at least as modules)
    • enable “Network packet filtering framework (netfilter)”
      • “IPv6 Netfilter configuration”
        • enable “IPv6 connection tracking support”
        • enable “IP6 tables support”
        • I specifically selected “packet filtering” “reject” and “log”, and made all others available as modules

There’s quite a few other changes made (if you’re going to the trouble of packaging your own kernel you might as well tailor it to your needs) but they aren’t relevant to the IPv6 connection tracking.

I didn’t quite adhere to the official centos kernel rebuilding guide as I found it rather awkward to follow. From a documentation and testing point of view I think it’s always a good idea to test your instructions on a clean machine and a person that knows little about the subject area (loved ones or a coworker from another section that happens to be passing through…) in case you’re making subconscious omissions of steps that are obvious to yourself. You may need to bribe the person for their time if you find your subject area is utterly dull to the majority of the population.

The other oddity is that the newer kernel uses more bits for the storage of MCE logs so the Centos mcelog package needs updating otherwise it will complain each hour. I used the spec file from the existing .src.rpm and the more recent source to build a new package. Notice that stock 64 bit hosts will have mcelog but not 32bit hosts. I believe the new kernel requires mcelog for both architectures.

I’ve tested it on our development host, I’ll be making formal plans to deploy on the university webcache on Tuesday which will make the service dual stacked. Testing the service in advance is problematic so I’ll need to formalise and scrutinise the deployment steps.

IPv6 University Firewall

We want to replace our current development IPv6 firewall with a production service. Of interest to us with regard to this is that Cisco released the 5585-X ASA’s this last week. These run the same software and commands as the ASA 5510 and upwards (the 5505 is different in some aspects from the rest of the range I believe) but outperform our FWSM modules for IPv6. So we could deploy a couple of ASA 5500 series devices, develop any in house scripts/software needed to work with them and then move to the 5585-X when the main university IPv4 firewall service is due to be replaced in 2 years time.

My concern that would be that we deploy a lower capacity ASA, enable a service such as the HFS backup service for IPv6, then watch as hordes of workstations without a native IPv6 connection use the JANET 6to4 service, sending IPv4 traffic out onto JANET resulting in high IPv6 traffic in through the new firewall, making an internet connection that was barely used swamped overnight, resulting in user complaints. I think as long as we’re careful and monitor the volumes of traffic we should be able to predict and respond without issue but it’s one reason why better performance monitoring tools for the firewall might be of interest.

I’ve had an initial discussion with Cisco about the above and have spent some time in the past week reading through the ASA manuals and listening to the Cisco product podcasts (whilst doing other tasks). I don’t use iTunes much but I believe from memory it was simple to select the podcasts in the store section and search for ‘Cisco’, subscribe to the channels shown and then ‘show previous episodes’ for each channel and ‘get all’.

Next week

Next week I’ll concentrate on the Webcache IPv6 deployment, which will be early Tuesday during the JANET at risk period (e.g. 7am).

You may be aware that we’ve this week lost a prized team member (Oliver Gorwits) to another employer, have also recently re-employed for the wireless position and are seeing the retirement of our senior manager, so I don’t want to make too many other predictions about next week – I enjoy working on the IPv6 deployment issues but we have many other projects and support calls.

If I’m lucky I might have a deployment plan for an ASA solution to the development firewall at the end of next week.