Learn Development Tools
In this section, you'll bring up the development environment. In the process, you'll be introduced to tools that will later prove useful for turning the provided hub into a learning switch. You'll cover both general and OpenFlow-specific debugging tools.
Let's define some terminology, starting with terminal types:
- VirtualBox console terminal: connects to OpenFlowTutorial. This is the one created when you started up the VM. You can't copy and paste from this tutorial page to the console terminal, so it's a bit of a pain. Minimize this NOW, if you haven't already done so. Once you've used it to set up networking, it won't be needed.
- SSH terminal: connects to OpenFlowTutorial. Created by using putty on Windows or SSH on OS X / Linux, as described in the previous section. Copy and paste should work on this terminal.
- xterm terminal: connects to a host in the virtual network. Created in the next section when you start up the virtual network. Will be labeled at the top with the name of the host.
- OpenFlow Controller: sits above the OpenFlow interface. The OpenFlow reference distribution includes a controller that acts as an Ethernet learning switch in combination with an OpenFlow switch. You'll run it and look at messages being sent. Then, in the next section, you'll write our own controller on top of NOX or Beacon (platforms for writing controller applications).
- OpenFlow Switch: sits below the OpenFlow interface. The OpenFlow reference distribution includes a user-space software switch. Open vSwitch is another software but kernel-based switch, while there is a number of hardware switches available from Broadcom (Stanford Indigo release), HP, NEC, and others.
- ovs-ofctl: command-line utility that sends quick OpenFlow messages, useful for viewing switch port and flow stats or manually inserting flow entries.
- Wireshark: general (non-OF-specific) graphical utility for viewing packets. The OpenFlow reference distribution includes a Wireshark dissector, which parses OpenFlow messages sent to the OpenFlow default port (6633) in a conveniently readable way.
- iperf: general command-line utility for testing the speed of a single TCP connection.
- Mininet: network emulation platform. Mininet creates a virtual OpenFlow network - controller, switches, hosts, and links - on a single real or virtual machine. More Mininet details can be found at the Mininet web page.
- cbench: utility for testing the flow setup rate of OpenFlow controllers.
Let's get started...
The network you'll use for the first exercise includes 3 hosts and a switch (and, eventually, an OpenFlow controller, but we'll get to that later):
To create this network in the VM, in an SSH terminal, enter:
$ sudo mn --topo single,3 --mac --switch ovsk --controller remote
This tells Mininet to start up a 3-host, single-(openvSwitch-based)switch topology, set the MAC address of each host equal to its IP, and point to a remote controller which defaults to the localhost.
Here's what Mininet just did:
- Created 3 virtual hosts, each with a separate IP address.
- Created a single OpenFlow software switch in the kernel with 3 ports.
- Connected each virtual host to the switch with a virtual ethernet cable.
- Set the MAC address of each host equal to its IP.
- Configure the OpenFlow switch to connect to a remote controller.
Since you'll be working in Mininet for the whole tutorial, it's worth learning a few Mininet-specific commands:
To see the list of nodes available, in the Mininet console, run:
mininet> nodes
To see a list of available commands, in the Mininet console, run:
mininet> help
To run a single command on a node, prepend the command with the name of the node. For example, to check the IP of a virtual host, in the Mininet console, run:
mininet> h1 ifconfig
The alternative - better for running interactive commands and watching debug output - is to spawn an xterm for one or more virtual hosts. In the Mininet console, run:
mininet> xterm h1 h2
If you get message "X11 connection rejected because of wrong authentication" you should run:
$ export XAUTHORITY=$HOME/.Xauthority
and try again.
If you get no error and the xterm window doesn't launch, you might not have xterm installed (such as on a clean install of Ubuntu 18.04 LTS or later). You can check if xterm is installed by running xterm in a non-Mininet console. If the command is not found, you can install xterm using sudo apt-get install xterm or other method of your choice.
You can close xterm windows now, as we'll run through most commands in the Mininet console.
If Mininet is not working correctly (or has crashed and needs to be restarted), first quit Mininet if necessary (using the exit command, or control-D), and then try clearing any residual state or processes using:
$ sudo mn -c
and running Mininet again.
NB: The prompt mininet> is for Mininet console, $ is for SSH terminal (normal user) and # is for SSH terminal (root user) (See Command Prompt Notes). Hereafter we follow with this rule.
Mininet has loads of other commands and startup options to help with debugging, and this brief starter should be sufficient for the tutorial. If you're curious about other options, follow the Mininet Walkthrough after the main tutorial.
(Just to remind you, we are assuming that you have already started up Mininet in another window using sudo mn --topo single,3 --mac --switch ovsk --controller remote, as directed above.)
ovs-ofctl is a utility that comes with Open vSwitch and enables visibility and control over a single switch's flow table. It is especially useful for debugging, by viewing flow state and flow counters. Most OpenFlow switches can start up with a passive listening port, from which you can poll the switch, without having to add debugging code to the controller.
Create a second SSH window if you don't already have one, and run:
$ ovs-ofctl show s1
One user reports the need to prepend with sudo, otherwise a "permission denied" comes back:
$ sudo ovs-ofctl show s1
The show command connects to the switch and dumps out its port state and capabilities.
Here's a more useful command:
$ sudo ovs-ofctl dump-flows s1
Since we haven't started any controller yet, the flow-table should be empty.
Note: In the above example, ovs-ofctl is talking to a local instance of Open vSwitch via a Unix domain socket which it is looking up by name. The actual socket is probably something like /var/run/openvswitch/s1.mgmt . If you were running another switch such as the Stanford reference switch or even a hardware OpenFlow switch, you would need to connect to a passive TCP port using a command like
$ ovs-ofctl dump-flows tcp:{ip address}:{port}
where {ip address} is the IP address of the switch's management interface and {port} is the passive OpenFlow listening/management port. If Mininet is invoked with --switch user, it will open up a passive listening port for each switch at port (6633+n) where n is the number of the switch. This will enable ovs-ofctl to be used in commands like
$ ovs-ofctl dump-flows tcp:127.0.0.1:6634
For the purposes of this tutorial, you will usually be using Open vSwitch so you can just use the simple form of the ovs-ofctl command.
Note that there are also dpctl and ovs-dpctl commands; usually these commands should not be used and ovs-ofctl should be used instead! With Open vSwitch, those commands can allow you to examine OVS's kernel flow cache, which is usually a subset of the full OpenFlow flow table which you or the controller have programmed. Flows in OVS's flow cache will usually time out quickly, for example within 5 seconds. This allows OVS to support large flow tables reasonably efficiently without a huge kernel flow cache, but it can also introduce slowdowns when flows miss in the kernel flow cache and have to be reloaded from user space.
Now, go back to the mininet console and try to ping h2 from h1. In the Mininet console:
mininet> h1 ping -c3 h2
Note that the name of host h2 is automatically replaced when running commands in the Mininet console with its IP address (10.0.0.2).
Do you get any replies? Why? Why not?
As you saw before, switch flow table is empty. Besides that, there is no controller connected to the switch and therefore the switch doesn't know what to do with incoming traffic, leading to ping failure.
You'll use ovs-ofctl to manually install the necessary flows. In your SSH terminal:
# ovs-ofctl add-flow s1 in_port=1,actions=output:2 # ovs-ofctl add-flow s1 in_port=2,actions=output:1
This will forward packets coming at port 1 to port 2 and vice-verca. Verify by checking the flow-table
# ovs-ofctl dump-flows s1
Run the ping command again. In your mininet console:
mininet> h1 ping -c3 h2
Do you get replies now? Check the flow-table again and look the statistics for each flow entry. Is this what you expected to see based on the ping traffic?
The VM image includes the OpenFlow Wireshark dissector pre-installed. Wireshark is extremely useful for watching OpenFlow protocol messages, as well as general debugging.
Start a new SSH terminal and connect to the VM with X11 forwarding.
(Reminder: here are the command-line commands to do so:
If you're using MAC OS X or Linux, enter:
$ ssh -X mininet@[guest ip address]
If you're using putty.exe from the Windows command-lineshell, enter:
C:> putty.exe -X mininet@[guest ip address]
If you're using putty's GUI, make sure X11 forwarding is enabled.)
Now open Wireshark:
$ sudo wireshark &
You'll probably get a warning message for using wireshark with root access. Press OK.
If Wireshark doesn't start, try the command
$ sudo -E wireshark &
Click on Capture->Options... in the menu bar. Select the 'Loopback: lo' interface, then click the Start button in the lower right of the window. You may see some packets going by.
Now, set up a filter for OpenFlow control traffic, by typing 'of' in Filter box near the top:
of
Press the apply button to apply the filter to all recorded traffic.
First stop mininet and clean
mininet> exit $> mn -c
Now, with the Wireshark dissector listening, start the OpenFlow reference controller. In your SSH terminal:
$sudo controller ptcp:6633
This starts a simple controller that acts as a learning switch without installing any flow-entries.
Now restart the topology:
$>sudo mn --topo single,3 --mac --switch ovsk --controller remote
You should see a bunch of messages displayed in Wireshark, from the Hello exchange onwards. As an example, click on the Features Reply message. Click on the triangle by the 'OpenFlow Protocol' line in the center section to expand the message fields. Click the triangle by Switch Features to display datapath capabilities - feel free to explore.
These messages include:
| Message | Type | Description |
| Hello | Controller->Switch | following the TCP handshake, the controller sends its version number to the switch. |
| Hello | Switch->Controller | the switch replies with its supported version number. |
| Features Request | Controller->Switch | the controller asks to see which ports are available. |
| Set Config | Controller->Switch | in this case, the controller asks the switch to send flow expirations. |
| Features Reply | Switch->Controller | the switch replies with a list of ports, port speeds, and supported tables and actions. |
| Port Status | Switch->Controller | enables the switch to inform that controller of changes to port speeds or connectivity. Ignore this one, it appears to be a bug. |
Since all messages are sent over localhost when using Mininet, determining the sender of a message can get confusing when there are lots of emulated switches. However, this won't be an issue, since we only have one switch. The controller is at the standard OpenFlow port (6633), while the switch is at some other user-level port.
You can optionally add columns for the source and destination ports of each packet by right-clicking on the column headers and selecting 'Column Preferences...'. Use the green plus button to add a column then enter a name, double click on the type field to change it, and drag and drop to the order you desire. Click OK when done updating your preferences.
Now, we'll view messages generated in response to packets.
Before that update your wireshark filter to ignore the echo-request/reply messages (these are used to keep the connection between the switch and controller alive): Type the following in your wireshark filter, then press apply:
of and not (of10.echo_request.type or of10.echo_reply.type)
Note that there are a variety of other ways that you could express that filter.
If you have an old version of the wireshark plugin, you may need to use a slightly different syntax:
of && (of.type != 3) && (of.type != 2)
Run a ping to view the OpenFlow messages being used. You will need to run this without having any flows between h1 and h2 already installed, e.g. from the "add-flows" command above:
sudo ovs-ofctl del-flows s1
It's also recommended to clean up ARP cache on both hosts, otherwise you might not see some ARP requests/replies as the cache will be used instead:
mininet> h1 ip -s -s neigh flush all mininet> h2 ip -s -s neigh flush all
Do the ping in the Mininet console:
mininet> h1 ping -c1 h2
In the Wireshark window, you should see a number of new message types:
| Message | Type | Description |
| Packet-In | Switch->Controller | a packet was received and it didn't match any entry in the switch's flow table, causing the packet to be sent to the controller. |
| Packet-Out | Controller->Switch | controller send a packet out one or more switch ports. |
| Flow-Mod | Controller->Switch | instructs a switch to add a particular flow to its flow table. |
| Flow-Expired | Switch->Controller | a flow timed out after a period of inactivity. |
First, you see an ARP request miss the flow table, which generates a broadcast Packet-Out message. Next, the ARP response comes back; with both MAC addresses now known to the controller, it can push down a flow to the switch with a Flow-Mod message. The switch does then pushes flows for the ICMP packets. Subsequent ping requests go straight through the datapath, and should incur no extra messages; with the flows connecting h1 and h2 already pushed to the switch, there was no controller involvement.
Re-run the ping, again from the Mininet console (hitting up is sufficient - the Mininet console has a history buffer):
mininet> h1 ping -c1 h2
If the ping takes the same amount of time, run the ping once more; the flow entries may have timed out while reading the above text.
This is an example of using OpenFlow in a reactive mode, when flows are pushed down in response to individual packets.
Alternately, flows can be pushed down before packets, in a proactive mode, to avoid the round-trip times and flow insertion delays.
iperf is a command-line tool for checking speeds between two computers.
Here, you'll benchmark the reference controller; later, you'll compare this with the provided hub controller, and your flow-based switch (when you've implemented it).
In the mininet console run :
mininet> iperf
This Mininet command runs an iperf TCP server on one virtual host, then runs an iperf client on a second virtual host. Once connected, they blast packets between each other and report the results.
Now compare with the user-space switch. In the mininet console:
mininet> exit
Start the same Mininet with the user-space switch:
$ sudo mn --topo single,3 --controller remote --switch user
- Note: The reference switch (`--switch user`) does not handle MAC address 00:00:00:00:00:01 correctly, so it should not be used with `--mac`**
mininet> iperf
See a difference? With the user-space switch, packets must cross from user-space to kernel-space and back on every hop, rather than staying in the kernel as they go through the switch. The user-space switch is easier to modify (no kernel oops'es to deal with), but slower for simulation.
Exit Mininet:
mininet> exit
Continue on to your first assignment, Create a Learning Switch