# Introduction
Tangle is a tool to help you document networks.
It takes a text file that lists:
* Endpoint devices (eg, servers, workstations) * Switches, routers, firewalls * Management devices (eg, console/KVM servers, managed PDUs) * Network cables * Power cables * Serial/KVM/USB/parallel cables * Telephone cables (for analogue phones that don't go over UTP) * Structured cabling systems (eg, patch panels, wall sockets, and the trunk lines between them) * Telephones * Logical circuits (eg, VLANs, power phases) * IPv4 subnets * External networks (ISP networks, the phone network, etc) * Locations (where equipment is) * Organisations (that own equipment and locations)
...and how they interconnect.
It can then convert that file into a heavily cross-referenced HTML document listing all the components of your network. Every device will list every cable that connects to it (specifying the logical circuit(s), IPv4 addresses and so on), and clicking on a cable will lead to a list of all devices and circuits on that cable; clicking on a circuit will lead to a list of all devices and IPv4 subnets on the circuit; and clicking on an IPv4 address or subnet will go to a list of all the IPv4 addresses on that subnet, and so on.
It also outputs several "dot" files for GraphViz. The default makefile produces three diagrams: one for physical network cabling, one for logical network connectivity (showing what devices are on which circuits), and one showing management cabling (power, serial, and KVM).
It is written in Python, and consists of two layers: a network parser library which reads the network description file, checks it for errors (duplicate IP addresses, etc), and produces an in-memory cross-referenced object graph; and then a set of scripts that use that library to generate the HTML and "dot" output formats. You can write your own scripts that do other things with the data (such as generating reverse DNS zonefiles).
# Getting it
Pick up the latest release (0.5) or the unstable development version
# Using it
A simple (all too simple) driver shell script is supplied, called `document-network.sh`, that runs parts of Tangle to make a veritable soup of network documentation.
Try it out on the supplied examples. From this directory, run:
./document-network.sh examples/home-lan.tangle ./document-network.sh examples/office-lan.tangle
...then look at the HTML and PNG files that appear in the examples directory.
# The tangle file format
## Cables
Each cable starts with a line of the form:
<type> <name> { ... }
Valid cable types, with their common uses, are:
* `utp`, `fibre`, `wifi`, or `virtual` for Ethernet cables. `virtual` cables are used for logical network connections that join virtual machines to the network, or the internal ports by which switches connect their management processor into the switching fabric. `wifi` cables, in particular, tend to have lots and lots of devices on them, but any cable can have any number of devices.
* `serial` and `parallel` for RS232 and parallel links. `serial` cables are often used for management consoles on devices.
* `usb` for USB connections (often found joining printers to print servers, or for attaching low-end UPSes to management servers; we don't suggest you document your keyboard and mouse in Tangle).
* `kvm` for combined keyboard/video/mouse connections used as part of a remote management solution.
* `telephone` for dedicated telephone cables. If your phone system uses UTP cabling, call it `utp`.
* `power`, or if you want to be specific, `power240vAC`, `power110vAC`, `power48vDC` or `power12vDC`, for power cables.
Note that what matters is the *physical* type of cable; if you're running RS232 or telephones or KVM or even power over UTP, then call it `utp`.
Cables all need unique names. Ideally, they should be numbered, with the number written on the cable (http://www.partex.com/ sell nice clip-on cable numbers). If you have a multi-location system you can reuse cable numbers within a location, and write the cable names in your Tangle file as "<location>.<number>".
Within the cable declaration are lines defining the details of the cable:
device <name>/<port>
Declares that this cable goes to the given device on the given port. Since this is a cabling document, the device names should be in terms of people looking at the outside of the device - how the physical socket is labelled, or some other obvious designation; a server may refer to `eth0` internally, but unless you stick a label saying `eth0` on the outside of your server (which is a good idea!), a name like `left` might be more useful to the people who go to the datacenter to re-cable your server.
If you don't know what port it's plugged into, as you have incomplete information, just use `?` as the port name. This will suppress errors about multiple cables going into the same port, as many cables will probably go to the `?` port on a device.
panel <panel>/<port>
This declares that the cable plugs into the specified port of a patch panel or wall socket.
A cable can have any number of `device` and `panel` declarations.
circuit <circuit> vlan <circuit>
The logical circuit carried on this cable. For an Ethernet cable, that's an untagged VLAN. Circuit names must be unique; if you reuse VLAN names or numbers in different contexts, name them `<location>.<id>`. It is suggested that you use a physical identifier such as the actual VLAN ID, phase name as from the supplier, etc as the ID (eg, the number you configure into the switch), but you can use any alphanumeric string as the ID. The synonym `vlan` for `circuit` is accepted. This declaration is only allowed once.
circuits <circuit>,<circuit>,... tagged vlans <circuit>,<circuit>,...
For cables that carry multiple circuits (such as tagged VLANs, polyphase power cables, KVM octopus cables, etc), the list of circuits carried on this cable. `tagged vlans` is a synonym for `circuits`. This declaration is only allowed once.
note "<text>"
General notes to sysadmins. There can be more than one.
## Panels
Patch panels and wall sockets are all modelled as "panels" in Tangle. Each panel must be declared like so:
panel <panel> { ... }
Within the panel declaration, the following are legal:
title ""
Gives the panel a long name.
note "<note>"
Attaches a note to the panel.
location <location>
Assigns a location to the panel.
Panels may be assigned owners, by giving them a name of the form `<owner>:<panel>`.
However, the ports on a panel are not declared in the panel itself; they are declared implicitly, by having cables plugged into them and trunks declared between them
## Trunks
The fixed cabling that joins panels is known as a "trunk" in Tangle. As trunks are often large groups of cables, they are always declared as part of a "trunk group", like so:
<type> trunk <name> { ... }
The `<type>` is from the same set of types used for cables, and indicates the type of cabling in this trunk group.
Within the trunk group, the following declarations are allowed:
title "<name>"
Assigns a long name to the trunk group.
note "<note>"
Attaches a note to the trunk group.
trunk <label> <panel>/<port> <panel>/<port>
Declares a single trunk between the given panel/port pairs, with the given label. The label need only be unique within the trunk group.
trunk <panel>/<port> <panel>/<port>
Declares a single trunk between the given panel/port pairs, with a label made by adding 1 to the label of the previous trunk declared. This is only legal when the label of the previous trunk was a number (at the start of a trunk group, the previous label is considered to be `0`).
group <panel>/<port>-<port> <panel>/<port>-<port>
Declares a number of trunks between the specified range of ports on the specified panels. This is only useful when panels have numeric port names. It also implies automatic trunk number, so it is only legal when the previous trunk label was numeric, or if this is the first trunk declaration within a trunk group.
## Links
When a single cable joins ports on devices, it makes a link. However, when a cable connects to panels and trunks, then multiple cables - at least one patch cable at each end, and a trunk in the middle - join together to join ports on devices. That group of interconnected cables is known as a "link". Cables and panels and trunks are the physical things you tell Tangle about; it works out what the resulting links are itself.
A single-cable link will be named after the cable; but a multi-cable link will end up with a name composed of its component cables.
Do not confuse a link - which is a single physical path joining some devices - with a circuit, which is a more abstract notion like a VLAN.
## Devices
We can also optionally provide extra data about devices with a device declaration:
device <name> { ... }
If a cable refers to a device with no device declaration, that's fine - we just don't know anything about the device other than what cables plug into it. It's a black box.
Within a device declaration, we allow `note` declarations, and these specialist declarations:
type {server|switch|router|manager|power|external|virtual|workstation|printer|telephone|firewall|other}
This declares the type of device, which affects how it is displayed in diagrams, and how it is categorised in the HTML output.
ip4 <port> <circuit> <ip> stub
This declares that a given port of the device, optionally specifying a circuit for tagged-VLAN ports, has a given IPv4 address. There can be more than one ip4 declaration for a given circuit on a given port, and more than one circuit on a port, as you would expect. An IP address that appears on more than one port (eg, HSRP / CARP) should be marked `shared` to suppress warnings about the duplication. An IP address that does not need to match a subnet (eg, a /32 address set up on a Cisco router's loopback interface) can be declared 'stub' to suppress warnings about that, too.
mac <port> <circuit> <mac> shared
This declares that a given port of the device (again, optionally specifying a circuit for tagged VLAN ports), has a given MAC address. The address should be in `xx:xx:xx:xx:xx:xx` form, but omitting the colons or using other characters (spaces, hypens) is also acceptable. Re-using the same MAC will cause warnings, unless `shared` is specified.
Some devices are virtual - VMs, for example. We can express this fact by declaring that the physical server hosts the virtual server like so:
hosts <device>
Note that the system makes no real distinction between physical and virtual devices. They can nest arbitrarily; you can have VMs within VMs.
VM host servers should declare a virtual cable that carries appropriate vlan circuits to ports on virtual machines, and which plugs into an internal virtual port in the VM server, to explain how the VMs get network connectivity.
We can also give devices physical locations, like so:
location <location>
## Unused ports
It can be nice to document unused ports on devices, too, so you know what's spare. This is done with an `unused` declaration:
unused <name> { ... }
The block of unused ports needs to be given a name. How you structure these is up to you - if a bank of switches are all functionally identical, then you might group all their unused ports into one block; or you might have separate blocks for different kinds of ports on the one device.
Within an unused ports declaration, you can use the `device` declaration to name a device/port pair that is unused, exactly like in a cable declaration. You can also use `note` to attach notes.
## Locations
Locations must be declared with a `location` declaration:
location <location> { ... }
Locations accept `note` declarations, like most objects in Tangle, and an optional `title` declaration that assigns a long title that may contain spaces.
Locations may, in turn, be within a parent location:
location <location>
And they can have zero or more addresses:
address <type> ""
The address type can be any single word; I recommend using `postal` for postal addresses, `latlong` for latitude/longitude, etc.
Also, like devices, locations can be given a name of the form `<organisation>:<name>`, declaring that they belong to a specific organisation.
## Circuits / VLANs
Likewise, we can provide extra information about a circuit with a `circuit` declaration:
circuit <name> { ... }
Again, we can use `note` declarations within a circuit.
title ""
This gives the circuit a meaningful title, since circuit 'names' are usually just physical identifiers such as VLAN numbers.
Each circuit has a type, which can be specified with:
type {ethernet|serial|parallel|power|kvm|usb|telephone}
Ethernet circuits can carry IP4 subnets. These are declared like so:
ip4 subnet <ip>/<prefixlen> "" multiple
This declares that a given IPv4 subnet exists on the circuit. There can be more than one. The same ip range can exist on different circuits, too, but this will cause a warning unless the `multiple` flag is given, as this state of affairs is sometimes legitimate (eg, when doing BGP anycast routing, or an IPv4 subnet appears on different circuits in different locations)
ip4 reserved <ip1>-<ip2> "<description>"
This declares that a range of ip4 addresses are reserved for something. This allocates the IP4s as being in use, as if a port on the circuit were assigned an IP, but without a port being required. It's perfect for IP ranges set aside for future use, or for DHCP / VPN / dialup address pools.
## Organisations
Finally, we can declare information about organisations.
organisation {
...
}
The code of the organisation is the short name used before the colon in the name of a device or location.
Within an organisation, we can specify a longer name using a title declaration:
title "<name>"
And general notes:
note "<text>"
And details of authorised and emergency contacts:
contact emergency "<text>"
# History
## Version 0.6
Fixed bug in IP ranges, which was missing the first IP of the range.
## Version 0.5
Added support for locations.
## Version 0.4
Generalised VLANs into "circuits", which can apply to more than just Ethernet (eg, power; a polyphase power system can carry multiple power circuits in a cable, which split out into single-phase cables, for instance).
"vlan" is now a parser synonym for "circuit".
## Version 0.3
Added MAC address support
## Version 0.2
Added the office-lan example, and improved the rendering logic
## Version 0.1
Initial public release
# TODOs
## More flexible mappings from cable types to circuit types
Right now, we have a fixed mapping, that each cable type (eg, `utp`) has a corresponding circuit type (eg, `ethernet`) that it can carry. However, UTP in particular can carry ethernet, serial, KVM, and others. This, currently, will cause warnings as the system will complain that a UTP cable is meant for Ethernet! It'd be best to make each cable type map to a list of circuit types so the warnings can be handled better. It makes it harder to guess the type of a circuit, though - we'd need to try to find, for undeclared-type circuits that don't have IP4 subnets or MAC addresses (which would make them definitely Ethernets) a common type, or just leave them as of unknown type.
## Structured cabling
Right now cables are rather simple beasts. To better support structured cabling systems, it would be nice to explain how a physical link gets from some switch port to another, via a structured cabling system.
Proposal:
Allow the creation of "panels", which have a series of named ports, just like a switch. But they are not a "device" to tangle. Ports have a type, from the same set as cables.
"trunk group" objects join pairs of panel ports together. Provide an easy way to make a series of trunks between contiguous ranges of numbered ports. Eg, "Ports 1-4 on patch A are trunked to ports 5-8 on patch B". Trunks, like any other cables, may join any number of panel ports (consider busses, such as power cabling).
Cables may now join device ports or panel ports.
Inside tangle, what is currently called a "cable" becomes a "link". A link is just like a cable, except it has a different name per attachment point, and it has an internal structure composed of one or more cables, with trunks between two panel ports joining cables.
So, we might have:
panel patch-a { title "Second floor patch panel" # Patch ports are created implicitly by trunking them }
panel patch-b { title "Basement patch panel" }
utp trunk st { title "Stairwell trunk" group patch-a/1-4 patch-b/5-8
# Could also be written out as:
# trunk patch-a/1 patch-b/5 # trunk patch-a/2 patch-b/6 # trunk patch-a/3 patch-b/7 # trunk patch-a/4 patch-b/8
# Each trunk line lists *one or more* endpoints, although # there's little use for a one-end trunk; perhaps unfinished # cabling!
# Each connection in the trunk is assigned a label, by default # str(int(previous_label)+1), so both of the above would # label them '1' to '4'.
# But we can also name or number them explicitly, eg if we have # identification by pair colour in a big cable:
# trunk red patch-a/1 patch-b/5 # trunk blue patch-a/2 patch-b/6 # trunk green patch-a/3 patch-b/7 # trunk yellow patch-a/4 patch-b/8 }
utp A01 { # Patch two ports together panel patch-a/1 panel patch-a/2 }
utp B01 { panel patch-b/5 device server/eth }
utp B02 { panel patch-b/6 device switch/6 }
Now, we have a link from server/eth to switch/6, which tangle will treat as it currently treats cables. But that link knows it's called "B01" where it connects to server/eth, and "B02" where it connects to switch/6, and it knows it has this structure:
utp B01 -> panel patch-b/5 -> trunk st/1 -> panel patch-a/1 -> utp A01 -> panel patch-a/2 -> trunk st/2 -> panel patch-b/6 -> utp B02
...which can be explored in the HTML view of the cable. # Each panel would have its own table of what things plug into its ports, and what trunks link it to where, too.
### TASKS:
* Generalise the notion of "cable" to "link" in the existing codebase, but with the current cable-parsing code as input; just modify the data model, rename classes, update the output renderers to handle links having different names at different attachments, and make everything work again. - DONE
* Modify the input parser to create new panel, trunk group, and cable objects. A "trunk" is always identified as trunk group + label of the trunk within it, for easier management. - DONE
* Add a phase, after parsing, to create links from the panels/trunks/cables it finds. - DONE
* Extend the HTML renderer to display the detailed structure of links. - DONE
* Extend the HTML renderer to display panels. - DONE
* Add a third dot renderer that shows panels (rectangles) and trunk groups (thick lines), and devices (ovals) and cables (thin lines).
* Document it - DONE
## Generalise device/cable types
PROGRESS: Cable+circuit types now come from a `CABLE_TYPES` dict in network.py; provide an option to load more.
The core engine has a fixed list of device and cable types (and circuit types?), even though it doesn't really care what they are. Front-end scripts tend to care more, as they often invoke special rendering for particular types of object.
Adding to this list involves extending the parser regexps, extending the documentation, and then extending the various rendering scripts.
It might be nice to have a configuration file that lists all the types of typable things, and then listing key=value properties for them. The parser can generate its regexps from the lists, and then frontends can look up keys to find styling information for specific types.
Make better use of more node shapes:
http://www.graphviz.org/doc/info/shapes.html
Also, make use of colour to better distinguish node and link types in the graphviz outputs.
## Better rendering
Add an option to the dot generators, passing in an optional HTML filename; if present, then an image map is created, linking objects in the diagrams to the corresponding objects in the HTML report.
Add an option to generate a graph of the cables and trunks forming a link which, when image-mapped, can be included in the HTML documentation, for each link. Or find some other way to graph the potentially complex interconnections of trunks and cables in a link.
## Minor issues
* Add support for multi-drop trunks. The core data structures support it, but the parser needs to be a little more sophisticated.
* Note the filename+line number of definition of all the objects, for ease of back-referencing. (How to display nicely in the HTML view?)
* Make ip4 reserved ranges appear as a single row in the IP4 displays in the HTML, rather than one per IP.
* Make it a proper installable Python thing where you do `python setup.py install`
* Make a list of stub addresses, and put a table of them in the ip4 subnets section of the HTML report, as a special "others" subnet
* If an ip4 is declared on a device but there isn't a cable 'creating' that port on the device, create a port without any cables (and warn about it).
* Refactor the naming of `code`, `name`, and `title` fields. Standardise on `name` and `title`, most likely, for code name and full title.
* Normalise how owners and locations are handled; make `location` valid in any object (even trunks can be confined to a building), and have a generalised owner/location cross-referencing function that can be called on any array of objects with `name` and `location` properties, given the name of an array in the `organisation` and `location` objects to add objects to.
* Improve the CSS, to make table rows clearer, and better separation of top-level headings between objects in the main pane.