To locate a node (workstation, printer, etc) on the network, you use a combination of the ARP logs from a router, and the MAC-port mappings from a switch. The ARP log translates IP to MAC, and the MAC-port log translates the MAC address to a switch port. The standard mechanism used to gather all this data is SNMP, from a central network management host.
On the University backbone network we gather this information once an hour from all our devices. In the past year, we implemented a new database backed system to replace the older flat-file data store (which had understandably grown large with historical data and was slow to query). The new system uses a piece of software called Netdisco, which is written in Perl and has a reasonable web front-end to perform basic querying and reporting.
So with running both the old and the new systems, our switches and routers were being queried twice an hour, and this was beginning to put an unwelcome load on their CPUs. Normally the CPUs on these devices do very little except for what’s called “management plane and control plane” activities – SSH, routing protocols, spanning tree, and so on. SNMP can be very intensive for a device, so we want to minimize it if at all possible.
Last week I rewrote three of our web tools for IT Staff, to migrate them from the flat file data source to the new database. In fact, I wasn’t able to completely migrate them, as we’ve not yet installed our new IP Address Management system, but I was at least able to drop one of our major hourly SNMP processes. So now we’re much happier about the load on our devices’ CPUs, but you may notice a couple of side effects in these web tools.
The main ARP Query tool doesn’t yet have much historical DNS data. The old flat file system logged the DNS entry of the IP at the time it was seen in an ARP log. The new system now does that too, but we’ve not imported the historical data from the old to the new. I looked at this, and it was a daunting data munging task, which I’ll leave for another day.
The Network Audit and Status tools similarly now speak to our new database backend. Unfortunately without our new IP Address Management system they also still load up some large flat files, so are slow to run – a typical query taking about 10 seconds. I hope this will be acceptable for the next nine months or so until we are able to move to full dynamic data lookups.