port. Since this is a cabling document, the device names should be in
terms of people looking at the outside of the device - how the
physical socket is labelled, or some other obvious designation; a
server may refer to `eth0` internally, but unless you stick a label
saying `eth0` on the outside of your server (which is a good idea!), a
name like `left` might be more useful to the people who go to the
datacenter to re-cable your server.

If you don't know what port it's plugged into, as you have incomplete
information, just use `?` as the port name. This will suppress errors
about multiple cables going into the same port, as many cables will
probably go to the `?` port on a device.

Device names can be of the form `<owner>:<device>`. If so, the owning
organisation can be declared with an `organisation` block, to attach
metadata, as described below.

    vlan <vlan>
    

specialist declarations:

    type {server|switch|router|manager|power|external|virtual|workstation|printer|telephone|firewall|other}

This declares the type of device, which affects how it is displayed in
diagrams, and how it is categorised in the HTML output.


    ip4 <port> [<vlan>] <ip> [shared|stub]
    
This declares that a given port of the device, optionally specifying a
vlan for tagged ports, has a given IPv4 address. There can be more
than one ip4 declaration for a given vlan on a given port, and more
than one vlan on a port, as you would expect. An IP address that
appears on more than one port (eg, HSRP / CARP) should be marked
`shared` to suppress warnings about the duplication. An IP address
that does not need to match a subnet (eg, a /32 address set up on a
Cisco router's loopback interface) can be declared 'stub' to suppress
warnings about that, too.

    mac <port> [<vlan>] <mac> [shared]

This declares that a given port of the device (again, optionally
specifying a vlan for tagged ports), has a given MAC address. The
address should be in `xx:xx:xx:xx:xx:xx` form, but omitting the colons
or using other characters (spaces, hypens) is also
acceptable. Re-using the same MAC will cause warnings, unless `shared`
is specified.

Some devices are virtual - VMs, for example. We can
express this fact by declaring that the physical server hosts the
virtual server like so:

    hosts <device>

And general notes:

    note "<text>"

And details of authorised and emergency contacts:

    contact [authorised|emergency] "<text>"

# History

## Version 0.3

Added MAC address support

## Version 0.2

Added the office-lan example, and improved the rendering logic

## Version 0.1

Initial public release



# TODOs

## Generalise VLANs

Having multiple virtual circuits over one cable can be useful for:

* Polyphase power cables
* Serial/KVM "splitter" cables
* ...and maybe other uses in future.

We can currently create VLANs on non-Ethernet cables, just it looks
odd; they should be renamed from "VLANs" to "virtual circuits" or some
shorter phrasing.

Suggestion: s/vlan/circuit/ but keep "vlan" as a parser-level synonym.

And add a "type" declaration within a circuit declaration, with values
matching the same types allowed for cables.

## Generalise device/cable types

The core engine has a fixed list of device and cable types (and vlan
types?), even though it doesn't really care what they are. Front-end
scripts tend to care more, as they often invoke special rendering for
particular types of object.

Adding to this list involves extending the parser regexps, extending
the documentation, and then extending the various rendering scripts.

It might be nice to have a configuration file that lists all the types
of typable things, and then listing key=value properties for them. The
parser can generate its regexps from the lists, and then frontends can
look up keys to find styling information for specific types.

Make better use of more node shapes:

http://www.graphviz.org/doc/info/shapes.html

Also, make use of colour to better distinguish node and link types in
the graphviz outputs.

## Minor issues

* Make ip4 reserved ranges appear as a single row in the IP4 displays
  in the HTML.

* Make it a proper installable Python thing where you do `python
  setup.py install`

## Core Network

device core-switch {
       type switch
       # This IP address is attached to the "management" port
       # of the switch, via a virtual cable (see below)
       ip4 management 192.168.100.10
       mac management 00:26:9e:25:70:27
}

virtual core-switch-mgmt {
	# Here we show how the management interface of a switch
	# works. It's a virtual cable with only one "end",
	# thereby assigning a VLAN to the port "management" on the switch.
	device core-switch/management
	vlan core.400
}

device hades {
       type router
       note "Cisco 2800"
       note "Border router"
       ip4 loopback 123.123.123.8 stub
       ip4 internal core.210 123.123.123.17
       mac internal core.210 f2:0b:a4:c9:c1:0d
       ip4 internal core.220 123.123.123.33
       mac internal core.220 f2:0b:a4:c9:c1:0d
       ip4 external 123.123.123.2
       mac internal core.220 f2:0b:a4:c9:c1:0e
}

virtual hades-loopback {
	device hades/loopback
}

utp 001 {

device bovril {
       type firewall
       note "Cisco ASA"
       note "Firewall"
       note "Provides DHCP to 192.168.2.1 on core.310"
       ip4 external 123.123.123.18
       mac external 00:c4:12:3e:e3:01
       ip4 internal core.300 192.168.1.254
       mac internal core.300 00:c4:12:3e:e3:01
       ip4 internal core.310 192.168.2.1
       mac internal core.310 00:c4:12:3e:e3:01
       ip4 internal core.320 192.168.3.1
       mac internal core.320 00:c4:12:3e:e3:01
       ip4 internal core.400 192.168.100.1
       mac internal core.400 00:c4:12:3e:e3:01
}

serial s2 {
       device bovril/console
       device boris/com1
}

}

## Internal Servers

device internal-switch {
       type switch
       ip4 management 192.168.100.11
       mac management 00:c4:12:4f:08:12
}

virtual internal-switch-mgmt {
	device internal-switch/management
	vlan core.400
}

	device keith/vif0
	vlan core.320
}

device laserjet {
       type printer
       ip4 ethernet 192.168.3.20
       mac ethernet 00:00:24:c3:8a:1c
}

utp i004 {
    device internal-switch/4
    device laserjet/ethernet
    vlan core.320
}

## Power management

device pdu1 {
       type manager
       ip4 ethernet 192.168.100.5
       mac ethernet 00:00:24:c2:44:9f
}

utp i007 {
    device internal-switch/7
    device pdu1/ethernet
    vlan core.400
}

}

# Main Office

device office-switch {
       type switch
       ip4 management 192.168.100.12
       mac management 00:e0:81:2c:4d:43
}

virtual office-switch-mgmt {
	device office-switch/management
	vlan core.400
}

   print "<div id=\"nav1\">"
   print "<a href=\"#nav-devs\">Devices</a>"
   print "<a href=\"#nav-vlans\">VLANs</a>"
   print "<a href=\"#nav-cables\">Cables</a><br>"
   print "<a href=\"#nav-unused\">Unused Ports</a>"
   print "<a href=\"#nav-ip4subnets\">IP4</a>"
   print "<a href=\"#nav-macs\">MACs</a>"
   print "<a href=\"#nav-orgs\">Organisations</a>"
   print "</div>"

   print "<div id=\"nav2\">"

   deviceNames = cabling.devices.keys()
   deviceNames.sort()

   vlanNames = cabling.vlans.keys()
   vlanNames.sort()

   cableNames = cabling.cables.keys()
   cableNames.sort()

   subnetNames = cabling.ip4subnets.keys()
   subnetNames.sort()

   macNames = cabling.macs.keys()
   macNames.sort()

   orgNames = cabling.organisations.keys()
   orgNames.sort()

   print "<h2 id=\"nav-devs\">Devices</h2>"

   for devType in ["server","switch","router","manager","power","external","virtual","workstation","printer","telephone","firewall","other"]:
      found_any = False

      print "</ul>"

   print "<h2 id=\"nav-vlans\">VLANs</h2>"

   print "<ul>"
   for vlanName in vlanNames:
      print "<li><a href=\"#vlan-%s\">%s</a></li>" % (vlanName,cabling.vlans[vlanName])
   print "</ul>"

   print "<h2 id=\"nav-cables\">Cables</h2>"

   print "<ul>"
   for cableName in cableNames:
      if cabling.cables[cableName].cableType != "unused":
         print "<li><a href=\"#cable-%s\">%s</a> %s</li>" % (cableName,cableName,cabling.cables[cableName].cableType)
   print "</ul>"

   print "<h2 id=\"nav-unused\">Unused Ports</h2>"

   print "<ul>"
   for cableName in cableNames:
      if cabling.cables[cableName].cableType == "unused":
         print "<li><a href=\"#cable-%s\">%s</a></li>" % (cableName,cableName)
   print "</ul>"

   print "<h2 id=\"nav-ip4subnets\">IP4 Subnets</h2>"

   print "<ul>"
   for subnetName in subnetNames:
      subnet = cabling.ip4subnets[subnetName]
      print "<li><a href=\"#subnet-%s\">%s</a></li>" % (subnet.id,subnet.nice_title())
   print "</ul>"

   print "<h2 id=\"nav-macs\">MAC Addresses</h2>"

   print "<ul>"
   for mac in cabling.macs:
      print "<li><a href=\"#mac-%s\"><tt>%s</tt></a></li>" % (mac, mac)
   print "</ul>"

   print "<h2 id=\"nav-orgs\">Organisations</h2>"

   print "<ul>"
   for orgName in orgNames:
      print "<li><a href=\"#org-%s\">%s</a></li>" % (orgName,cabling.organisations[orgName])
   print "</ul>"

   print "</div>"

   numberRE = re.compile(r"[0-9]+")
   def portcmp(a,b):
      # Sort two port names, but noticing if they're all numeric and doing a numeric
      # comparison if so
      if numberRE.match(a[0]) and numberRE.match(b[0]):
         return cmp(int(a[0]),int(b[0]))
      else:
         return cmp(a[0],b[0])

   print "<div id=\"details\">"

   print "<h1>Devices</h1>"

   for deviceName in deviceNames:
      device = cabling.devices[deviceName]
      print "<h2 id=\"%s\">%s (%s)</h1>" % (deviceName,device, device.deviceType)

      print "<p>Owner: <a href=\"#org-%s\">%s</a></p>" % (device.owner.code,device.owner.name)

      if len(device.hostedDevices) > 0:
         print "<p>Hosted virtual devices</p>"
         print "<ul>"
         for hosteeName in device.hostedDevices:
            print "<li><a href=\"#%s\">%s</a></li>" % (hosteeName, hosteeName)
         print "</ul>"

      if len(device.hostedBy) > 0:
         for hostName in device.hostedBy:
            print "<p>This is a virtual device hosted by <a href=\"#%s\">%s</a>" % (hostName,hostName)

      for note in device.notes:
         print "<p>%s</p>" % note

      ports = device.ports
      ip4s = device.ip4s
      macs = device.macs

      ports.sort(portcmp)

      print "<table class=\"device\">"
      for portData in ports:
         (port,cable) = portData
         print "<tr class=\"dev-hdr\" id=\"%s-%s\"><td>port <b>%s</b></td><td>%s <a href=\"#cable-%s\">%s</a></td></th>" % (deviceName,port,port,cable.cableType, cable.name,cable.name)
         if cable.tagged == True:
            first = True
            for vlan in cable.vlans:
               if first:
                  print "<tr class=\"dev-vlans\"><td>Tagged VLANs</td>"
                  first = False
               else:
                  print "<tr class=\"dev-vlans\"><td></td>"

               print "<td><a href=\"#vlan-%s\">%s</a></td></tr>" % (vlan,cabling.vlans[vlan])
         elif cable.tagged == False:
            print "<tr class=\"dev-vlans\"><td>VLAN</td><td>"
            print "<a href=\"#vlan-%s\">%s</a>" % (cable.vlan,cabling.vlans[cable.vlan])
            print "</td></tr>"
         else:
            # Not Ethernet
            pass

         if len(cable.devices)>1 and cable.cableType != "unused":
            first = True
            for oDevice in cable.devices:
               (oDevName,oDevPort) = oDevice
               if oDevName == deviceName:
                  continue # Don't list ourselves

               if first:
                  print "<tr class=\"dev-links\"><td>Links to</td>"
                  first = False
               else:
                  print "<tr class=\"dev-links\"><td></td>"
               print "<td><a href=\"#%s-%s\">%s/%s</a></td></tr>" % (oDevName, oDevPort, oDevName, oDevPort)

         # Let's map vlan names to pairs ([macs],[ip4s])
         port_vlans = {}

#         print >> sys.stderr, "%s MACS: %r IP4s: %r" % (device,macs,ip4s)

         if macs.has_key(port):
            for mac in macs[port]:
               if mac.vlan == None:
                  vlan = False
               else:
                  vlan = mac.vlan

               if not port_vlans.has_key(vlan):
                  port_vlans[vlan] = ([],[])
               port_vlans[vlan][0].append(mac)

         if ip4s.has_key(port):
            for ip4 in ip4s[port]:
               if ip4.vlan == None:
                  vlan = False
               else:
                  vlan = ip4.vlan

               if not port_vlans.has_key(vlan):
                  port_vlans[vlan] = ([],[])
               port_vlans[vlan][1].append(ip4)


         for vlan in port_vlans:
            (port_macs,port_ip4s) = port_vlans[vlan]
            first = True

            for mac in port_macs:
               if first:
                  if vlan == False:
                     print "<tr class=\"dev-ip4\"><td><i>unspecified vlan</i></td>"
                  else:
                     print "<tr class=\"dev-ip4\"><td>vlan <a href=\"#vlan-%s\">%s</a></td>" % (vlan,cabling.vlans[vlan])
                  first = False
               else:
                  print "<tr class=\"dev-mac\"><td></td>"

               print "<td>MAC <tt>%s</tt>" % mac.mac
               if mac.shared:
                  print "<i>(shared)</i>"
               print "</td></tr>"

            for ip4 in port_ip4s:
               if first:
                  if vlan == False:
                     print "<tr class=\"dev-ip4\"><td><i>unspecified vlan</i></td>"
                  else:
                     print "<tr class=\"dev-ip4\"><td>vlan <a href=\"#vlan-%s\">%s</a></td>" % (vlan,cabling.vlans[vlan])

                  first = False
               else:
                  print "<tr class=\"dev-ip4\"><td></td>"

               print "<td>IP4 "

               if ip4.vlan == None:
                  print "%s" % (network.unparse_ip(ip4.ip))
               else:
                  subnetFound = False

                  for snName in cabling.ip4subnets:
                     sn = cabling.ip4subnets[snName]
                     if ip4.vlan == sn.vlan:
                        for Oip4 in sn.ip4s:
                           for Oport in Oip4.ports:
                              if Oport[0] == deviceName and Oport[1] == port:
                                 subnetFound = True
                                 print "<a href=\"#subnet-%s-%d.%d.%d.%d\">" % (sn.id, ip4.ip[0], ip4.ip[1], ip4.ip[2], ip4.ip[3])

                  if subnetFound:
                     print "%s</a>" % network.unparse_ip(ip4.ip)
                  else:
                     print network.unparse_ip(ip4.ip)

               if ip4.shared:
                  print "<i>(shared)</i>"

               print "</td></tr>"

      print "</table>"

   print "<h1>MAC Addresses</h1>"
   print "<table class=\"macs\">"

   for mac in macNames:
      first = True
      for macport in cabling.macs[mac]:
         if first:
            print "<tr><th id=\"mac-%s\"><tt>%s</tt></th>" % (mac,mac)
            first = False
         else:
            print "<tr><td></td>"
         
         print "<td><a href=\"#%s-%s\">%s/%s</a>" % (macport.device, macport.port, macport.device, macport.port)

         if macport.vlan != None:
            print "(<a href=\"#vlan-%s\">vlan %s</a>)" % (macport.vlan, macport.vlan)

         print "</td></tr>"

   print "</table>"

   print "<h1>VLANs</h1>"

   for vlanName in vlanNames:
      vlanLinks = cabling.vlans[vlanName].ports

      print "<h2 id=\"vlan-%s\">VLAN %s</h2>" % (vlanName, cabling.vlans[vlanName])

      for note in cabling.vlans[vlanName].notes:
         print "<p>%s</p>" % note

      print "<ul>"

      for sn in cabling.vlans[vlanName].ip4subnets:
         print "<li>IPv4: <a href=\"#subnet-%s\">%s</a></li>" % (sn.id, sn.nice_title())

      print "</ul>"

      print "<table class=\"vlan\">"

      # Sort by cable then device then port
      vlanLinks.sort(lambda a,b: cmp("%s/%s/%s" % (a[2],a[0],a[1]), "%s/%s/%s" % (b[2],b[0],b[1])))

      lastCable = None

      for vlanLink in vlanLinks:
         (dev,port,cable) = vlanLink

         if cable != lastCable:
            print "<tr class=\"vlan-cable-hdr\">"
            print "<th>%s <a href=\"#cable-%s\">%s</a></th>" % (cable.cableType, cable.name,cable.name)
            lastCable = cable
         else:
            print "<tr>"
            print "<td></td>"
         print "<td><a href=\"#%s-%s\">%s/%s</a></td>" % (dev,port,dev,port)

         print "</tr>"

      print "</table>"

   def documentCable(cableName,cable,showVLANs):
      print "<tr id=\"cable-%s\"><th>%s</th>" % (cableName, cable)

      print "<td>"
      for device in cable.devices:
         (dev,port) = device
         print "<a href=\"#%s-%s\">%s/%s</a><br>" % (dev,port,dev,port)

      print "</td>"

      if showVLANs:
         if cable.tagged == None:
            print "<td></td>"
         elif cable.tagged:
            print "<td>"
            for vlan in cable.vlans:
               print "<a href=\"#vlan-%s\">%s</a><br>" % (vlan,vlan)

            print "</td>"
         else:
            print "<td>untagged <a href=\"#vlan-%s\">%s</a></td>" % (cable.vlan,cable.vlan)
         print "</tr>"

      if len(cable.notes)>0:
         if showVLANs:
            print "<tr class=\"cable-notes\"><td colspan=\"3\">"
         else:
            print "<tr class=\"cable-notes\"><td colspan=\"2\">"
         for note in cable.notes:
            print "<p>%s</p>" % note
         print "</td></tr>"


   print "<h1>Cables</h1>"

   print "<table class=\"cables\">"
   print "<tr><th>Cable</th><th>Ports</th><th>VLANs</th></tr>"
   for cableName in cableNames:
      cable = cabling.cables[cableName]
      if cable.cableType != "unused":
         documentCable(cableName, cable, True)

   print "</table>"

   print "<h1>Unused Ports</h1>"

   print "<table class=\"cables\">"
   print "<tr><th>Group</th><th>Ports</th></tr>"
   for cableName in cableNames:
      cable = cabling.cables[cableName]
      if cable.cableType == "unused":
         documentCable(cableName, cable, False)

   print "</table>"

   print "<h1>IP4 Subnets</h1>"

   for subnetName in subnetNames:
      subnet = cabling.ip4subnets[subnetName]

      print "<h2 id=\"subnet-%s\">%s on vlan <a href=\"#vlan-%s\">%s</a>" % (subnet.id,network.unparse_ip(subnet.net), subnet.vlan, cabling.vlans[subnet.vlan])
      if subnet.title != None:
         print "(%s)" % subnet.title

      print "</h2>"

      print "<table class=\"subnet\">"
      subnet.ip4s.sort(lambda a,b: cmp(a.ip,b.ip))
      lastIP4 = None
      for ip4 in subnet.ip4s:
         first = True

         if lastIP4 != None and network.next_ip(lastIP4) != ip4.ip:
            print "<tr><td colspan=\"2\">...</td></tr>"

         lastIP4 = ip4.ip

         for port in ip4.ports:
            if first:

               #print "<a href=\"#subnet-%s-%d.%d.%d.%d\">" % (sn.id, ip4.ip[0], ip4.ip[1], ip4.ip[2], ip4.ip[3])

               print "<tr class=\"subnet-hdr\" id=\"subnet-%s-%d.%d.%d.%d\"><th>%s</th>" % (subnet.id, ip4.ip[0], ip4.ip[1], ip4.ip[2], ip4.ip[3], network.unparse_ip(ip4.ip))
               first = False
            else:
               print "<tr><td></td>"

            if port[0] == None: # reserved IP
               print "<td>(%s)</td>" % port[1]
            else:
               print "<td><a href=\"#%s-%s\">%s/%s</a></td></tr>" % (port[0],port[1],port[0],port[1])
      print "</table>"

   print "<h1>Organisations</h1>"

   for orgName in orgNames:
      org = cabling.organisations[orgName]
      if org.name != org.code:
         print "<h2 id=\"org-%s\">%s (%s)</h2>" % (org.code, org.name, org.code)
      else:
         print "<h2 id=\"org-%s\">%s</h2>" % (org.code, org.name)

      for note in org.notes:
         print "<p>%s</p>" % note

      if len(org.emergencyContacts) > 0:
         print "<p>In emergency, contact:"
         first = True
         for contact in org.emergencyContacts:
            if not first:
               print ", "
            first = False
            print contact

      print "<h3>Owned Devices</h3>"
      print "<ul>"
      for device in org.devices:
         print "<li><a href=\"#%s\">%s</a></li>" % (device.name, device.name)
      print "</ul>"

   print "</div>"

body {
	font-family: sans-serif;
	font-size: small;
}

div#nav1 {
	position: fixed;
	top: 0em;
	left: 0em;
	width: 15em;
	height: 4em;
}

div#nav2 {
	position: fixed;
	top: 4em;
	left: 0em;
	width: 20em;
	bottom: 0em;
	overflow-y: scroll
}

div#details {

      self.devices = []
      self.tagged = None
      self.notes = []

      # Ethernet cables only
      self.vlan = None
      self.vlans = []

   def __repr__(self):
      return "%s %s" % (self.cableType, self.name)
   pass

class VLAN:
   def __init__(self):
      self.name = None
      self.ports = []
      self.notes = []
      self.ip4subnets = []
      self.title = None
      self.ip4reservations = []
      self.macs = {} # MAC -> list of MacPort objects

   def __repr__(self):
      if self.title == None:
         return "%s" % (self.name)
      else:
         return "%s (%s)" % (self.name, self.title)

class Device:
   def __init__(self):
      self.name = None
      self.ports = []  # List of (portname,cable) pairs
      self.notes = []
      self.ip4s = {} # indexed on port name
      self.macs = {} # indexed on port name
      self.deviceType = "server" # default
      self.owner = None
      self.hostedDevices = [] # List of names of devices we 'host' as VMs
      self.hostedBy = [] # Name of device(s) that host(s) this one, if it's virtual

   def __repr__(self):
      return "%s" % self.name

   def has_port(self,port_name):
      for port in self.ports:
         if port[0] == port_name:
            return True
      return False

class Organisation:
   def __init__(self,code):
      self.code = code
      self.name = code
      self.emergencyContacts = []
      self.authorisedContacts = []
      self.notes = []
      self.devices = []

   def __repr__(self):
      return self.name

class IP4Port:
   # An IP address on a port
   # Optionally specific to a VLAN on that port
   def __init__(self,vlan,ip,shared,stub):
      self.vlan = vlan
      self.ip = ip
      self.shared = shared
      self.stub = stub

   def __repr__(self):
      return "%s on vlan %s" % (unparse_ip(self.ip),self.vlan)

class MacPort:
   # A MAC address on a port
   # Optionally specific to a VLAN on that port

   def __init__(self,vlan,device,port,mac,shared):
      self.vlan = vlan
      self.device = device
      self.port = port
      self.mac = mac
      self.shared = shared

   def __repr__(self):
      return "%s on vlan %s" % (self.mac,self.vlan)

class IP4Subnet:
   # A subnet with the same IP address but different vlans is a different subnet
   def __init__(self,net,title,multiple):
      global IP4SubnetCounter
      self.net = net
      self.vlan = None
      self.ip4s = []

      self.ports = []
      self.shared = False

   def __repr__(self):
      return "%s on %s" % (unparse_ip(self.ip),",".join(map(lambda(port): "%s/%s" % port,self.ports)))

class CablingScheme:
   def __init__(self,cables,vlans,devices,macs,ip4subnets,organisations):
      self.cables = cables
      self.vlans = vlans
      self.devices = devices
      self.macs = macs # Dict mapping a mac address to a list of MacPort objects with that mac
      self.ip4subnets = ip4subnets
      self.organisations = organisations

# IPs are represented as 5-tuples: 127.0.0.1 = (127,0,0,1,32)
# 10.0.0.0/8 = (10,0,0,0,8)
def parse_ip(ip_str):
   parts = ip_str.split(".")
   if len(parts) != 4:
      raise ValueError, "Invalid IP address %s" % ip_str
   result = (int(parts[0]),int(parts[1]),int(parts[2]),int(parts[3]),32)
   return result

def parse_net(net_str):
   (ip_str,bits) = net_str.split("/")
   parts = ip_str.split(".")
   if len(parts) != 4:
      raise ValueError, "Invalid IP address"
   result = (int(parts[0]),int(parts[1]),int(parts[2]),int(parts[3]),int(bits))
   return result

macCleanRE = re.compile("[^a-fA-F0-9]+")
macParseRE = re.compile("(..)")

def parse_mac(mac_str):
   mac_str_clean = macCleanRE.sub("",mac_str).lower()
   result = ""
   for part in macParseRE.finditer(mac_str_clean):
      if result == "":
         result = part.group(0)
      else:
         result = "%s:%s" % (result,part.group(0))

#   return "'%s/%s/%s'" % (mac_str, mac_str_clean, result)
   return result

# Trim off all bits to the right of the prefix
def trim_net (parsed_ip):
   bits = parsed_ip[4]
   orig_bits = bits
   bytes = parsed_ip[0:4]

   result = []

   while bits>=8:
      bits = bits - 8
      result.append (bytes[0])
      bytes = bytes[1:]

   if bits != 0:
      result.append (bytes[0] & (0xff00 >> bits))

   while len(result)<4:
      result.append (0)

   result.append(orig_bits)

   result = tuple(result)

   return result

if trim_net((1,2,3,4,32)) != (1,2,3,4,32):

# returns 1 if a contains b
# returns -1 if a is contained by b
# returns None if a and b don't overlap
def compare_nets(a,b):
   # Normalise
   a = trim_net(a)
   b = trim_net(b)

   # Equal?
   if a == b:
      return 0

   # superset?
   if a[4] > b[4]:
      # a is smaller than b
      a_as_b = trim_net ((a[0],a[1],a[2],a[3],b[4]))
      if a_as_b == b:
         return -1

   if a[4] < b[4]:
      # a is larger than b
      b_as_a = trim_net ((b[0],b[1],b[2],b[3],a[4]))
      if b_as_a == a:
         return 1

   return None

if compare_nets((1,2,3,0,24),(2,2,3,0,24)) != None:
   raise RuntimeError

if compare_nets((1,2,3,0,24),(1,2,3,0,24)) != 0:
   raise RuntimeError

class IPRange:
   def __init__(self,fromIP,toIP):
      self.fromIP = fromIP
      self.toIP = toIP
      self.currentIP = fromIP
      self.finished = False

   def __iter__(self):
      return self

   def next(self):
      if self.finished:
         raise StopIteration
      else:
         self.currentIP = next_ip(self.currentIP)
         self.finished = self.currentIP == self.toIP
         return self.currentIP

   noteRE = re.compile(r"note +\"([^\"]*)\"")

   # title "<text>"
   titleRE = re.compile(r"title +\"([^\"]*)\"")

   # device <name> {
   deviceStartRE = re.compile(r"device +([^ /]+) *{")

   # type <type>
   deviceTypeRE = re.compile(r"type +((server|switch|router|manager|power|external|virtual|workstation|printer|telephone|firewall|other))")

   deviceHostsRE = re.compile(r"hosts +([^ /]+)")

   # ip4 <port> <vlan> <ip> [shared|stub]
   ip4vlanRE = re.compile(r"ip4 +([^ ]+) +([^ ]+) +([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) *((shared|stub)?)")

   # ip4 <port> <ip> [shared|stub]
   ip4RE = re.compile(r"ip4 +([^ ]+) +([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) *((shared|stub)?)")

   # mac <port> <vlan> <mac> [shared]
   macVlanRE = re.compile(r"mac +([^ ]+) +([^ ]+) +([^ ]+) *((shared)?)")

   # mac <port> <mac> [shared]
   macRE = re.compile(r"mac +([^ ]+) +([^ ]+) *((shared)?)")

   # vlan <name> {
   vlanStartRE = re.compile(r"vlan +([^ /]+) *{")

   # ip4 subnet <ip>/<prefix> "<title>" [multiple]
   ip4subnetRE = re.compile(r"ip4 +subnet +([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/[0-9]+) +\"([^\"]+)\" *((multiple)?)")

   # ip4 reserved <ip>-<ip> "<description>"
   ip4reservedRE = re.compile(r"ip4 +reserved +([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)-([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+) +\"([^\"]+)\"")

   # organisation <code> {
   organisationStartRE = re.compile(r"organisation +([^ :]+) *{")

   # contact emergency "<text>"
   orgEmergContactRE = re.compile(r"contact +emergency +\"([^\"]+)\"")

   # contact authorised "<text>"
   orgAuthContactRE = re.compile(r"contact +authorised +\"([^\"]+)\"")

   endRE = re.compile(r"}")

   cables = {}
   cable = None

   vlans = {}
   vlan = None

   devices = {}
   device = None

   ip4subnets = {}
   ip4subnet = None

   organisations = {}
   organisation = None

   lineNumber = 0
   for line in infile:
      m = commentRE.match (line)
      if m:
         line = m.group(1)

      line = line.strip()
      lineNumber = lineNumber+1

      # print >> sys.stderr, "%d:(%s/%s/%s) %s" % (lineNumber, cable, device, vlan, line)

      if line == "":
         continue

      if cable == None and vlan == None and device == None and organisation == None:
         m = cableStartRE.match(line)
         if m:
            cable = Cable()
            cable.cableType = m.group(1)
            cable.name = m.group(2)

            if cables.has_key(cable.name):
               print >> sys.stderr, "line %d: The cable %s is declared twice" % (lineNumber,cable.name)
               sys.exit(1)

            continue
         else:
            m = deviceStartRE.match(line)
            if m:
               device = Device()
               device.name = m.group(1)

               if devices.has_key(device.name):
                  print >> sys.stderr, "line %d: The device %s is declared twice" % (lineNumber,device.name)
                  sys.exit(1)

               nameparts = device.name.split(":")
               if len(nameparts) == 1:
                  orgname = "unknown"

               device.owner.devices.append(device)
               continue
            else:
               m = vlanStartRE.match(line)
               if m:
                  vlan = VLAN()
                  vlan.name = m.group(1)

                  if vlans.has_key(vlan.name):
                     print >> sys.stderr, "line %d: The vlan %s is declared twice" % (lineNumber,vlan.name)
                     sys.exit(1)
                  continue
               else:
                  m = organisationStartRE.match(line)
                  if m:
                     organisation = Organisation(m.group(1))

                     if organisations.has_key(organisation.code):
                        print >> sys.stderr, "line %d: The organisation %s is declared twice" % (lineNumber, organisation.code)
                     continue
                  else:
                     print >> sys.stderr, "line %d: Syntax error in global scope:" % lineNumber
                     print >> sys.stderr, line
                     sys.exit(1)

      if organisation != None:
         m = noteRE.match(line)
         if m:
            organisation.notes.append(m.group(1))
         else:
            m = titleRE.match(line)
            if m:

                     if m:
                        organisations[organisation.code] = organisation
                        organisation = None
                     else:
                        print >> sys.stderr, "line %d: Syntax error in organisation scope:" % lineNumber
                        print >> sys.stderr, line
                        sys.exit(1)


      if device != None:
         m = noteRE.match(line)
         if m:
            device.notes.append(m.group(1))
         else:
            m = endRE.match(line)
            if m:
               devices[device.name] = device
               device = None
            else:
               m = ip4vlanRE.match(line)
               if m:
                  if not device.ip4s.has_key(m.group(1)):
                     device.ip4s[m.group(1)] = []

                  shared = m.group(4) == "shared"
                  stub = m.group(4) == "stub"
                  device.ip4s[m.group(1)].append(IP4Port(m.group(2),parse_ip(m.group(3)),shared,stub))
               else:
                  m = ip4RE.match(line)
                  if m:
                     if not device.ip4s.has_key(m.group(1)):

                     if m:
                        device.deviceType = m.group(1)
                     else:
                        m = deviceHostsRE.match(line)
                        if m:
                           device.hostedDevices.append(m.group(1))
                        else:
                           m = macVlanRE.match(line)
                           if m:
                              if not device.macs.has_key(m.group(1)):
                                 device.macs[m.group(1)] = []

                              shared = m.group(4) == "shared"
                              device.macs[m.group(1)].append(MacPort(m.group(2),device.name,m.group(1),parse_mac(m.group(3)),shared))

                           else:
                              m = macRE.match(line)
                              if m:
                                 if not device.macs.has_key(m.group(1)):
                                    device.macs[m.group(1)] = []

                                 shared = m.group(3) == "shared"
                                 device.macs[m.group(1)].append(MacPort(None,device.name,m.group(1),parse_mac(m.group(2)),shared))
                              else:
                                 print >> sys.stderr, "line %d: Syntax error in device scope:" % lineNumber
                                 print >> sys.stderr, line
                                 sys.exit(1)

      if vlan != None:
         m = noteRE.match(line)
         if m:
            vlan.notes.append(m.group(1))
         else:
            m = endRE.match(line)
            if m:
               vlans[vlan.name] = vlan
               vlan = None
            else:
               m = ip4subnetRE.match(line)
               if m:
                  net = parse_net(m.group(1))
                  trimmed = trim_net(net)
                  if trimmed != net:
                     print >> sys.stderr, "line %d: Subnet %s is not canonical (should be %s)" % (lineNumber, unparse_ip(net), unparse_ip(trimmed))

                  multiple = m.group(3) == "multiple"
                  vlan.ip4subnets.append((trimmed,m.group(2),multiple))
               else:
                  # ip4 reserved <ip>-<ip> "<description>"
                  m = ip4reservedRE.match(line)
                  if m:
                     fromIP = parse_ip(m.group(1))
                     toIP = parse_ip(m.group(2))
                     descr = m.group(3)

                     # FIXME: Check for overlap
                     vlan.ip4reservations.append((fromIP,toIP,descr))

                  else:
                     m = titleRE.match(line)
                     if m:
                        vlan.title = m.group(1)
                     else:
                        print >> sys.stderr, "line %d: Syntax error in vlan scope:" % lineNumber
                        print >> sys.stderr, line
                        sys.exit(1)

      if cable != None:
         m = endRE.match(line)
         if m:
            cables[cable.name] = cable
            cable = None
         else:
            m = deviceRE.match(line)
            if m:
               cable.devices.append ((m.group(1),m.group(2)))
            else:
               m = vlanRE.match(line)
               if m:
                  if cable.cableType == "unused":
                     print >> sys.stderr, "line %d: Syntax error in unused port pool scope:" % lineNumber
                     print >> sys.stderr, line
                     sys.exit(1)

                  cable.tagged = False
                  cable.vlan = m.group(1)
               else:
                  m = taggedVlansRE.match(line)
                  if m:
                     if cable.cableType == "unused":
                        print >> sys.stderr, "line %d: Syntax error in unused port pool scope:" % lineNumber
                        print >> sys.stderr, line
                        sys.exit(1)

                     cable.tagged = True
                     for _vlan in m.group(1).split(","):
                        if _vlan != "":
                           cable.vlans.append (_vlan.strip())
                  else:
                     m = noteRE.match(line)
                     if m:
                        cable.notes.append(m.group(1))
                     else:
                        print >> sys.stderr, "line %d: Syntax error in cable scope:" % lineNumber
                        print >> sys.stderr, line
                        sys.exit(1)

   return CablingScheme(cables, vlans, devices, {}, ip4subnets, organisations)

def crossRefHosts(cabling):
   for deviceName in cabling.devices:
      device = cabling.devices[deviceName]
      for hosteeName in device.hostedDevices:
         if cabling.devices.has_key(hosteeName):
            cabling.devices[hosteeName].hostedBy.append(deviceName)
         else:
            print >> sys.stderr, "Device %s claims to host nonexistant device %s" % (deviceName, hosteeName)
            sys.exit(1)

def crossRefPorts(cabling):
   for cableName in cabling.cables:
      cable = cabling.cables[cableName]
      for device in cable.devices:
         (dev,port) = device

         # Create devices that were not declared explicitly in the
         # file, but implied by having a cable going to them
         if not cabling.devices.has_key(dev):
            device = Device()
            cabling.devices[dev] = device
            device.name = dev

            nameparts = device.name.split(":")
            if len(nameparts) == 1:
               orgname = "unknown"
            else:
               orgname = nameparts[0]

            if cabling.organisations.has_key(orgname):
               device.owner = cabling.organisations[orgname]
            else:
               org = Organisation(orgname)
               device.owner = org
               cabling.organisations[orgname] = org

            device.owner.devices.append(device)

         # Is something already plugged into that port?
         # Disable warning if port name is '?' since that means we don't know what's what
         if port != "?":
            for devPort in cabling.devices[dev].ports:
               if devPort[0] == port:
                  print >> sys.stderr, "The port %s/%s is already attached to %s, and %s is also attempting to plug into it" % (dev, port, devPort[1], cable)
         cabling.devices[dev].ports.append ((port,cable))

         if cable.tagged == False:
            # Fix up any ip4s/macs without VLAN specified
            for portName in cabling.devices[dev].ip4s:
               if portName == port:
                  ip4s = cabling.devices[dev].ip4s[portName]
                  for ip4 in ip4s:
                     if ip4.vlan == None:
                        # print >>sys.stderr, "Assigning vlan %s from cable %s to ip %s of %s/%s" % (cable.vlan, cable, ip4, portName, dev)
                        # print >>sys.stderr, "We shared this cable with %s" % cable.devices
                        ip4.vlan = cable.vlan

            for portName in cabling.devices[dev].macs:
               if portName == port:
                  macs = cabling.devices[dev].macs[portName]
                  for mac in macs:
                     if mac.vlan == None:
                        # print >>sys.stderr, "Assigning vlan %s from cable %s to ip %s of %s/%s" % (cable.vlan, cable, ip4, portName, dev)
                        # print >>sys.stderr, "We shared this cable with %s" % cable.devices
                        mac.vlan = cable.vlan

def crossRefVLANs(cabling):
   for cableName in cabling.cables:
      cable = cabling.cables[cableName]

      if cable.tagged == None:
         pass # Not Ethernet
      elif cable.tagged: # Lots of VLANs
         for vlan in cable.vlans:
            # Create any vlans that were not explicitly declared
            # in the configuration, but implied by being present on a cable
            if not cabling.vlans.has_key(vlan):
               cabling.vlans[vlan] = VLAN()
               cabling.vlans[vlan].name = vlan

            # Build list of ports on each vlan
            for device in cable.devices:
               (dev,port) = device
               cabling.vlans[vlan].ports.append((dev,port,cable))

      else: # Just one VLAN
         # Create any vlans that were not explicitly declared
         # in the configuration, but implied by being present on a cable
         if not cabling.vlans.has_key(cable.vlan):
            cabling.vlans[cable.vlan] = VLAN()
            cabling.vlans[cable.vlan].name = cable.vlan

         # Build list of ports on each vlan
         for device in cable.devices:
            (dev,port) = device
            cabling.vlans[cable.vlan].ports.append((dev,port,cable))

def crossRefMacs(cabling):
   # Examine the MACs declared on vlans
   for deviceName in cabling.devices:
      device = cabling.devices[deviceName]
      for port in device.macs:
         macs = device.macs[port]

         for mac in macs:
            # Add to global list
            if not cabling.macs.has_key(mac.mac):
               cabling.macs[mac.mac] = []

            cabling.macs[mac.mac].append(mac)

            if mac.vlan != None:
               # Add to VLAN's MAC list

               # Create VLAN if it does not already exist
               if not cabling.vlans.has_key(mac.vlan):
                  cabling.vlans[mac.vlan] = VLAN()
               cabling.vlans[mac.vlan].name = mac.vlan

               if cabling.vlans[mac.vlan].macs.has_key(mac.mac) and not mac.shared:
                  print >> sys.stderr, "Warning: mac %s appears on VLAN %s in more than one place" % (mac.mac, vlan.name)
               else:
                  cabling.vlans[mac.vlan].macs[mac.mac] = []

               cabling.vlans[mac.vlan].macs[mac.mac].append(mac)

def crossRefIP4Subnets(cabling):
   # Examine ip4 subnets delcared on vlans
   for vlanName in cabling.vlans:
      vlan = cabling.vlans[vlanName]

      processedSubnets = []

      for subnetDetails in vlan.ip4subnets:
         (subnet,title,multiple) = subnetDetails
         for existingSubnetName in cabling.ip4subnets:
            existingSubnet = cabling.ip4subnets[existingSubnetName]
            comparison = compare_nets(existingSubnet.net,subnet)

            if comparison != None:
               # Same subnet on two VLANS is fine, it can be bridged
               # Or it might be being used for BGP anycast
               # Best warn anyway, though.
               # Overlapping subnets are a bit odd, though.
               if comparison == 0:
                  if not (multiple and existingSubnet.multiple):
                     print >>sys.stderr, "The subnet %s (from vlan %s) also exists on vlan %s" % (unparse_ip(subnet), vlan,  existingSubnet.vlan)
               else:
                  print >>sys.stderr, "The subnet %s (from vlan %s) overlaps the existing subnet %s" % (unparse_ip(subnet), vlan, existingSubnet)
               break

         newSubnet = IP4Subnet(subnet,title,multiple)
         newSubnet.vlan = vlanName
         cabling.ip4subnets[repr(newSubnet)] = newSubnet
         processedSubnets.append (newSubnet)

      vlan.ip4subnets = processedSubnets

   # Assign an ID to each subnet

   for subnetName in cabling.ip4subnets:
      subnet = cabling.ip4subnets[subnetName]

      subnet.id = "%d.%d.%d.%d.%d-%s" % (subnet.net[0],subnet.net[1],subnet.net[2],subnet.net[3],subnet.net[4],subnet.vlan)

   # Examine ip reservations

   for vlanName in cabling.vlans:
      vlan = cabling.vlans[vlanName]
      for resv in vlan.ip4reservations:
         (fromIP,toIP,descr) = resv
         for subnetName in cabling.ip4subnets:
            subnet = cabling.ip4subnets[subnetName]
            if vlanName == subnet.vlan and compare_nets(fromIP, subnet.net) == -1:
               ipRange = IPRange(fromIP,toIP)
               for ip in ipRange:
                  newIp4 = IP4(ip)
                  newIp4.ports.append((None,descr))
                  newIp4.shared = False
                  subnet.ip4s.append(newIp4)

   # Examine ip addresses on ports on devices
   for deviceName in cabling.devices:
      device = cabling.devices[deviceName]

      for ip4portName in device.ip4s:
         if not device.has_port(ip4portName):
            print >>sys.stderr, "An IP is declared on port %s/%s, but no cable graces that port" % (deviceName, ip4portName)
         ip4ports = device.ip4s[ip4portName]
         for ip4port in ip4ports:
            ip4 = ip4port.ip

            if ip4port.vlan == None and not ip4port.stub:
               print >>sys.stderr, "Port %s/%s declares IP %s, but the vlan is not declared and we cannot derive one automatically" % (deviceName,ip4portName, unparse_ip(ip4))
               continue

            #print >>sys.stderr, "Examining port %s" % ip4port

            matchedSubnets = []
            for subnetName in cabling.ip4subnets:
               subnet = cabling.ip4subnets[subnetName]
               #print >>sys.stderr, "considering %s vs %s: %s" % (ip4, subnet.net, compare_nets(ip4, subnet.net))
               #print >>sys.stderr, "vlan is %s vs. %s" % (ip4port.vlan, subnet.vlan)
               if (ip4port.vlan == subnet.vlan) and compare_nets(ip4, subnet.net) == -1:
                  # We have found our subnet
                  matchedSubnets.append(subnet)
                  #print >>sys.stderr, "ip address %s from %s/%s (vlan %s)" % (unparse_ip(ip4), deviceName, ip4portName, ip4port.vlan)
                  #print >>sys.stderr, "matches subnet %s" % subnet

                  existing = False
                  for existingIp4 in subnet.ip4s:
                     if existingIp4.ip == ip4:
                        if not (ip4port.shared and existingIp4.shared):
                           #print >>sys.stderr, "%s / %s" % (ip4port.shared, existingIp4.shared)
                           print >>sys.stderr, "The IP %s on %s/%s collides with %r" % (ip4port, deviceName, ip4portName, existingIp4)

                        existing = True
                        existingIp4.ports.append((deviceName,ip4portName))

                  if not existing:
                     newIp4 = IP4(ip4)
                     newIp4.ports.append((deviceName,ip4portName))
                     newIp4.shared = ip4port.shared
                     subnet.ip4s.append(newIp4)

            if ip4port.stub:
               if len(matchedSubnets) != 0:
                  print >>sys.stderr, "The IP %s is marked as a stub, but matches some subnets (%s)" % (ip4port,matchedSubnets)
            else:
               if len(matchedSubnets) == 0:
                  print >>sys.stderr, "The IP %s does not match any subnets" % ip4port
               elif len(matchedSubnets) != 1:
                  print >>sys.stderr, "The IP %s matched more than one subnet (%s)" % (ip4port,matchedSubnets)

def loadCabling (infile):
   cabling = parseCabling (infile)
   crossRefHosts (cabling)
   crossRefPorts (cabling)
   crossRefVLANs (cabling)
   crossRefMacs (cabling)
   crossRefIP4Subnets (cabling)

   return cabling