Hello again, since last updates I worked hard to finish my project and to reach the final milestone for this project.

As I explained in my previous post[1], due to some issues, we’ve decided to change the topic of the project to Implementing Pop-Routing in OLSRd instead of OSPF.

In this last month I completed the code for the OLSRd plugin[2], which I hope will be merged soon [3].In order to allow PRINCE to interact with OLSRd I had to modify the PRINCE source code[4] and create a new plugin[5].

The last part of my GSOC was testing the functionalities of my project.
To perform this tests I used a tool developed by the University of Trento, called “NePA TesT”[6]. NePA allowed me to simulate a mesh network in my laptop and to perform tests on it. The network topology was defined using NetJSON, but for my purpose modified it to use graph generators[7]

To ensure that PRINCE was working correctly on this virtual network I measured the centrality and the tuned timer for each node. Then I compared these values to the ones calculated by the original algorithm. Since the simulated network was real, and it needed a bit of time to converge, I took the last 10 values to avoid to measure errors. This are the maximum errors for each size and each kind of graph:

I also measured the “hello” messages’ rate to check if it was being calculated correctly by PRINCE. As I did for the centrality I took the mean of the last 10 values for each node and I compared them against the ones calculated using the python Pop-Routing algorithm.

Hence, as we can see from these tables, PRINCE is calculating the Centrality, and the timer’s value, with a really small error. This test also highlighted a bug (*) in the c_graph_parser library with very that particular kind of graph [8].

The last test I performed was to check whether the message to update the timers’ emission rate was actually modifying the emission rate of the messages.
I used a simple graph: 2 nodes connected by one link. And I captured the traffic with tcpdump, before and after the update message.
After 30 seconds I sent a message to the OLSRd poprouting plugin to update the hello timer to 5s. As you can see from the graph below it is working correctly!

I can conclude that PRINCE is working correctly with OLSRd and now it can be used to enhance the Wireless Community Networks that are still using it.
I would like to thank Freifunk, Ninux and Google for giving me the opportunity to participate in GSoC.

Cheers, Gabriele Gemmi

[1]: https://blog.freifunk.net/2017/07/27/implementing-pop-routing-ospf-july-updates/
[2]: https://github.com/AdvancedNetworkingSystems/olsrd/tree/poprouting
[3]: https://github.com/OLSR/olsrd/pull/38
[4]: https://github.com/AdvancedNetworkingSystems/poprouting/tree/refactor_ospf
[5]: https://github.com/AdvancedNetworkingSystems/poprouting/tree/refactor_ospf/prince/lib/olsrd
[6]: https://ans.disi.unitn.it/redmine/projects/community-newtork-emulator/wiki
[7]: https://github.com/AdvancedNetworkingSystems/wcn_emulator
[8]: https://github.com/AdvancedNetworkingSystems/poprouting/issues/23

Hello everyone!
First things first so here is the code that I’ve written. You can find it in these repositories:
OpenWifiCore (core server application)
OpenWifiFeed (LEDE/OpenWRT feed with boot flasher and boot notifier)
OpenWifiWeb (web frontend)
OpenWifiTemplates (old templating system)
OpenWifiLocation (plugin that detects the location of a node via google location api and nearby wifi aps)

I also worked together with Arne to interact with his SDWN controller and agent. We created a small website that informs you on how to use the two tools together. (As I’m writing this it is still somewhat under construction. But I hope everything will be there soon 🙂 ) Furthermore there is also specific documentation (WIP) for OpenWifi here.

To be a little more precise here is the list of commits that have been done:

May

OpenWifiCore
OpenWifiFeed
OpenWifiWeb

June

OpenWifiCore
OpenWifiWeb
OpenWifiLocation

July

OpenWifiCore
OpenWifiWeb
OpenWifiFeed

August

OpenWifiCore
OpenWifiFeed
OpenWifiWeb
OpenWifiTemplates

Overview about what has been done

Everything that has been done can be put into 4 categories: authentication/authorization, api, database-model and infrastructure. I want to give you a brief overview of all these categories.

Infrastructure

Docker images are now build automatically via TravisCI and deployed on docker hub. You can find more information in my first evaluation blog post. How to use the docker images is described in the documentation. These images are also used for testing.

database-model

The graph based database model was made a first class citizen. It now automatically converts between the internal representation and the representation needed to sync to the AP. It is also now possible to create new configurations and create links between these. The query format is also used for authentication/authorization.

api

Providing all the new functionality via a REST style API was the focus of this google summer of code (web views are still needed for quite some things). It is used for managing users, services, nodes and changing the graph based configuration model. It is described in the documentation.

The concept of having a service is also something new that I created together with Arne in august. If an external application needs to make changes to configurations of specific nodes it can just subscribe as a service. The service takes a list of database queries, a name and a shell script with a compare string. If the output (stdout) of the shell scripts matches the compare value the node gets the name of the service as it’s capability (also something that was added during this GSoC). When a node has the capability the queries are applied to it’s config in a regular interval by the job-server.

authentication/authorization

This was the biggest part of this GSoC. There are two ways of giving access to a node – either by giving access to a path string of a configuration or by allowing a specific database query. For more details see my second evaluation blog post.

There were quite some challenges as this system allows for some really complex access configurations and there are still some things to improve here. (see below)

What else? Aka the smaller bits

There were also some smaller bits that were done that don’t really fit the other categories. The boot-notifier and boot-flasher scripts were improved, a way to abstract communication with the nodes was started (see documentation), some small UI fixes to the graph view were done. And probably a lot of other smaller fixes I forgot 😉

Future – or what needs to be done

I guess the most important thing is to get all of these new features exposed via the web views. That shouldn’t be to hard but wasn’t the highest priority during GSoC. Everything related to the graph should be made aware of all possible path strings that lead to a configuration. Currently just one string is used – this should be lists! The fine grained authorization needs some more testing and I also want to improve the pattern matching by combining the regular expressions you could use to describe a path string. This could be done with greenery. The authorization right now is also just focused on nodes and configuration – it would be nice to restrict access to some views and actions. For example restrict the access to LUCI, sshkeys or executing commands on a node. Furthermore there should be a per node option where “the truth” of a config lies (like if there is a difference between the actual node configuration and the configuration on the server which one is considered the one to go with) – than it would also be nice to disable the sync for some nodes (if manual changes on the node need to be made for example).make

Acknowledgments

A big thanks goes out to Google for organizing something cool as the Google Summer of Code, to Freifunk for letting me do this project with them and last but not least Julius for being my mentor!

Result

In these three months I was working on the implementation of Luci2 (the graphic interface of LEDE / OpenWrt). The project was to translate the functionalities that Libremesh currently uses in Luci to the new proposal. The new environment consists of a backend based on UBUS that exposes JSON with data and the structure of the view.

As far as Google Summer of Code goes, I was able to make the ubus modules that emit information about bmx6, batman-adv, alignment, spectrum analysis, libremap and finally a series of several utilities. The results can be found in the lime-packages-ui repository.

Documentation

Each module has its documentation on the calls and the expected answers.

• ubus-luci-openairview
  • get_interfaces
  • get_stations
  • spectral_scan
  • get_station_signal
  • get_iface_stations
• ubus-luci-bmx6
  • status
  • originators
  • links
  • topology
  • tunnels
• ubus-luci-batman-adv
  • interfaces
  • originators
  • gateway
• ubus-luci-location
  • get_location
  • set_location
• ubus-lime-metrics
  • get_metrics
  • get_gateway
  • get_path
  • get_last_internet_path
  • get_internet_status
  • get_station_traffic
• ubus-lime-utils
  • get_cloud_nodes

To future

Finally I want to clarify that it is my intention to continue until achieving a complete implementation with the front end and keep the packages made. I will adapt the current luci packages to consume the data from the UBUS modules, so its use can be immediate and not wait for the complete development of luci2.

Thanks to the Freifunk community and the Libremesh team for giving me the opportunity to participate in this GSoC. No doubt I will continue to contribute to free software, to have more community and free networks.

tl;dr:
If you are looking for the work I have done for GSoC 2017, here are the principal repositories containing the source code and documentation:
– [1] ONOS
– [2] SDWN controller
– [3] SDWN agent

Also, check out the website that Johannes and I created to document how our two GSoC projects can be combined.

August Progress

In my last blog post I wrote that during the final coding phase I wanted to focus on applications for my controller platform – mainly an app that integrates my controller with Johannes’ OpenWifi server. While I was programming it, I noticed that I was getting tangled in unstructured dependencies between the different parts of my code. I realized that the controller needed a refactoring to better define its main application-facing interfaces. So I decided to modularize it into 4 separate pieces:

the SDWN core API,

its core implementation,

a command-line interface,

and a REST API.

With the refactoring done, I built two applications: the aforementioned OpenWifi integration and a simple client hearing map. Unfortunately I ran out of time just when adding features was fun again. The basics are all in place and working but I did not finish the more advanced applications such as the client load balancer or client authenticator that I mentioned in my last blog post.

On the more positive side, I really like how the newly structured controller behaves when adding features as well as how the OpenWifi integration turned out. A combined SDWN/OpenWifi setup is shaping up to become a neat management tool to operate a large deployment of LEDE nodes in which most of the initial configuration is automated. Nodes discover the OpenWifi server, download a configuration which makes them initiate a connection to the SDWN controller which gathers information about the nodes’ wireless capabilities and running access points. Throw in the hearing map and the nodes start reporting clients in their vicinity to the controller which makes that information available via its various APIs. Seeing it in action felt rewarding and dampened my initial disappointment of not getting everything done.
Johannes and I have put together a small website using GitHub pages which explains how to set up and use a combined SDWN/OpenWifi system.

Also during the past month, I have reached out to the ONOS developers about my work on the compatibility of ONOS’ fork of Loxigen and its upstream version. ONOS recently moved to a much more recent version of the loxigen-generated Java library for its OpenFlow subsystem which should make it easier for me to merge with it my modified ONOS. I want to explore if I can completely separate my code from the controller and have it provide the same functionality as external modules. This would of course greatly simplify future maintenance of the work I did.

The SDWN System

The following sections document the structure of the system that I have built and list possible applications.

The Controller

The SDWN controller that I have written comes as an OSGi bundle that gets deployed as an ONOS application. It relies on ONOS’ OpenFlow subsystem which manages the connection to the wireless switches in the network. Since I am using a custom OpenFlow protocol extension, I have added a corresponding protocol driver into ONOS’ south-bound interface which is responsible for the handshake with the SDWN agent. The SDWN controller’s core interacts with that driver and ONOS’ OpenFlow controller to create its abstraction of the network elements. It maintains so-called SDWN entities to represent the relevant wireless network resources and their relationships. For example, an access point is running on a network interface card which supports a certain set of transmission frequencies and rates. The controller would represent this state in the form of Java objects for the network interface card, its capabilities as well as the access points and set the references between them appropriately. This information is stored and made accessible through different APIs. If clients associate with the access point, the agent on the node notifies the controller which learns the client’s properties and stores the relationship with the AP. SDWN applications need not communicate with the wireless hardware directly but only work with the abstractions provided by the controller. There are also means for applications to subscribe to certain events such as the association of a new client or the reception of a probe request frame by an access point.

SDWN Applications

Applications can provide their services without having to interact directly with the network, they can leave communication details to the controller. For instance, the simple hearing map that I wrote subscribes to 802.11 management frame events. Every time the controller gets notified about such a frame by one of its APs, it informs the hearing map service. The service in turn stores the information about which AP on which switch received the frame on which frequency at what time and with how strong a signal.

Although I did not get to implement it, all the interfaces necessary to implement a blacklist of banned clients are in place. Simply register with the SDWN controller, hook into the stream of authentication and association request events, and deny unwanted clients based on their MAC address.

Or you could automatically push clients towards certain frequency bands: Query the hearing map for clients in the vicinity of an AP broadcasting at the desired frequency and check if they are currently associated with an access point on an unfavored channel. If yes, instruct that AP to disassociate and ban the clients for some time and grant their association request at the chosen target AP.

The Agent

To manage the network elements from the central controller, each of them needs to run an agent that maintains the connection, responds to queries, and executes commands. The SDWN agent is called wlanflow and it is built with the indigo OpenFow agent framework by the Floodlight project. It is available as a feed to integrate with the LEDE build system and uses UCI for configuration. Please refer to the instructions in the repository at GitHub (REPO LINK) about installation and configuration.
Wlanflow interacts with different other programs and services on the same machine. For example, it learns about most wireless capabilities through Linux’ netlink interface and subscribes to the standard Linux wifi soft-mac daemon hostapd via LEDE’s ubus IPC bus to receive notifications about incoming wifi management frames.

The Protocol

The SDWN controller and agent talk using an extended version of OpenFlow 1.3. The libraries for both the Java controller and the C agent are generated using a modified version of Loxigen.

Deploying the SDWN System

Like I said in the beginning of this blog post, I did not manage to realize all the ideas I had for this system. For this reason, the final product is a bit bare and lacks finish. Nonetheless, I encourage you to build a small installation and try out its features if you have spare wireless LEDE nodes lying around. I would appreciate your feedback and suggestions for improvements or new features.

You can find instructions on how to build and deploy the components of the SDWN system in their respective repositories which are listed at the top of this page.

Acknowledgments

Finally, I would like to thank Freifunk for letting me participate in GSoC again. Thanks to Andreas for organizing everything on Freifunk’s side and to my mentor, Julius.