blog.freifunk.net

PowQuty Live Log introduction

Posted on 21. May 201726. May 2017 by Stefan Venz

Hi everyone,

I’m Stefan and happy to present my GSoC 2017 project to you.
That is extending the PowQuty daemon and luci web-app with an event log and notification system for events violating the EN-50160 norm.
Probably most of you don’t know powquty. Therefore i will give a short introduction to it, before I present my actual project.

PowQuty

PowQuty[1] is an application for measuring power quality parameters of the power grid like voltage and frequency. It receives samples by a USB connected oscilloscope and writes the calculated values to a log file. This log can be read by the powquty luci web-app. The web interface will plot the measurements directly into several graphs. These are power over time, frequency over time and metrics over time.
It was implemented during the last GSoC by Neez El Sayed [2] and its intention is “to strengthen the role of open software in the uprising smart grids by providing some essential functionalities”.

PowQuty Live-log

The goal of this project is to provide an event log of measurements that are outside defined parameters.
Public power distribution networks have to meet certain criteria. These are formulated in the EN50160 norm.
If there is a power drop, or a frequency spike PowQuty will write this to its log like every other value. The user can examine the log file or its graphical representation for critical values, but essential equipment could already be damaged.
If such an event occurred, a user or power network administrator should be notified. This event log will get a graphical interpretation in the luci powquty app as well to enable a better tracking of events.
This will allow users of PowQuty to react quickly to changing conditions in power supply networks.

Therefore we will add a notification system to PowQuty as well as additional logging options.
To collect events at a central server we will extend the mqtt usage to send an mqtt notification on a violating event.
In addition we will add an email and slack notification option to allow quick responses.

[1]: https://github.com/thuehn/PowQuty
[2]: https://blog.freifunk.net/author/Neez-El-Sayed/

Our GSoC 2017 projects

Posted on 14. May 201727. May 2017 by Andreas Bräu

In 2017 we got 15 slots for our projects. That’s great, we’ve never got so many slots for GSoC before. Our students are from 8 countries. They do their work for 6 sub organizations.

Until the end of May the Community Bonding Period takes place, so students can introduce their projects to the community. They also prepare together with their mentors the folling coding period.

This post will give a short overview on the selected projects. In the next weeks some detailed blog posts will follow describing all the projects.

GSoC project overview

Title	Org	Student	Abstract
RetroShare Improvements	retroshare	ange	The aim of this proposal is to improve RetroShare incrementally during the summer in the following work lines: – Semi-automatic RetroShare friendship suggestion based on phone contacts – Semi-automatize JSON API code and documentation generation – Multiple simultaneous heterogeneous network connection for each peer – Improve RetroShare Android usability and performances – Seamless key exchange via Quiet Modem – Chat and messages multiple device synchronization – Directories multiple device synchronization
Implementing Pop-Routing in OSPF	ninux	Gabriele Gemmi	Prince is a network daemon that continuosly monitors the network topology and sets the timers for OLSRv2, it is developed in C and we are currently deploying it in the Ninux community network in Florence. This year we want to implement Pop-Routing for another link-state routing protocol: OSPF. OSPF is the state-of-the-art interior routing protocol for wired networks and it is used also in some wireless community networks. This project consists in realizing a plug-in of the OSPF open source routing daemon (Quagga or Bird) that will: – Expose the network topology to Prince using NetJson – Receive and re-set the network parameters from Prince
Improving nodewatcher data representation capability	wlan slovenija	rasovica	Currently nodewatcher has very limited overview of used IP space without more precise division of existing and used subnets. This could be improved using compact and colorful matrix with IP ranges and links to nodes. I would display data about used ips similar to the [IP Census 2014](http://census2012.sourceforge.net/images/hilbert_icmp_map_lowquality.jpg), better tables and other data representation that would help explain the network structure, it would help in explaining the network and its statistics even to an uninformed observer. I would also update the look and functionality of other graphs and tables representing statistical data from nodewatcher
Extending LoxiGen and ONOS to enable SDN control of wireless switches via OpenFlow	lede	Arne Kappen	LoxiGen is a tool by the Floodlight project to generate OpenFlow libraries for several programming languages including Java and C. The goal of this project is to extend LoxiGen to include messages for controlling wireless switches using OpenFlow. It will include the necessary extensions to LoxiGen as well as a proof-of-concept implementation of the control channel. This encompasses an application for the ONOS SDN controller and an agent for the LEDE router operating system.SDN-enabled wireless switches offer comfortable centralized management of larger deployments and may provide the basis for more complex use-cases in the future such as network slicing and seamless mobility.
OpenWRT/LEDE Configuration Management	lede	Johannes Wegener	Improve the proof-of-conecpt version of a OpenWRT/LEDE configuration software.
Web interface for Retroshare, a secure communication software	retroshare	Stefan Cokovski	I am applying to work on the web interface for RetroShare, secure communication software. RetroShare already has a web interface with limited functionality and questionable look, compared to the main Qt GUI which RetroShare uses. My goal will be to improve the already present functionalities, introduce new functionalities to the web interface and design a new look for the web interface.
Implement NetJSON output in ubus (OpenWRT/LEDE)	lede	Arunkumar Ravichandran	To bring in support for NETJSON object in the LEDE/OpenWRT Linux distributions. The support for NETJSON is brought in at the interconnect system- ubus. To add support for a new ubus API which allows retrieving these two NetJSON object types: DeviceConfiguration and DeviceMonitoring. DeviceConfiguration NetJSON object is filled in using the plugins available in System Configuration Abstraction Layer(SCAL). DeviceMonitoring NetJSON is retrieved by reusing the code parts and adding hooks in the nodewatcher-agent.
LibreMesh Attended Sysupgrade	libremesh	paul0	Performing updates on routers is quite different from full Linux distribution. It’s not sustainable to do release upgrade via a packet manager. Instead it’s usually required to re-flash the system image. Depending on the installed packages an image rebuild may be to complex for regular users. A more convenient way is needed. This project will implement an “image as a service” server side which provides custom build images depending on installed packages. A notification in the web interface will notify about the new release. After image creation a one-click installation is offered within the web interface. The server side implementation will use established tools like LEDE’s ImageBuilder to provide an generic approach for image creating. In this way the entire OpenWrt/LEDE community including several community-mesh firmware projects will benefit from that new update routine.
Spectrum Analyzer	libremesh	Nicolas Andres Pace	I believe that Community Networks’ Operating Systems could use the Spectrum Analysis a lot to make better decisions on how they use the electromagnetic field wisely and efficiently. As of now, no Free OS has been using Spectrum Analysis, so they can’t know whether the radios are being used efficiently or not. This module would allow all routers that use Atheros radios (the main radio manufaturer around) to see how is the spectrum and take better decisions on which channels they use. Deliverables * Create an OpenWRT/LEDE package to get Spectrum info from the radio. * Create a LuCI interface to visualize the Spectrum information retrieved.
HMAC signing of Nodewatcher data & IPv6 support for Tunneldigger	wlan slovenija	red_moo	Currently all monitoring reports by WLAN nodes are unsigned and can be spoofed by anyone. This represents a security problem and a possible solution is to add HMAC-based signing to encrypt node monitoring reports to prevent spoofing. Nodewatcher’s monitoring system will also be improved so it will generate an event and a warning in case of a signature verification failure.The aim of the second part of this GSoC project is to design and implement IPv6 support for Tunneldigger so that it works in IPv6-only and IPv4/IPV6 mixed environment where both server and client have IPv6 connectivity in some form.
Add MPTCP support in LEDE/OpenWRT trunk	lede	SPYFF	Create an MPTCP supported OpenWRT/LEDE trunk and ensure its operation in a multipath Wi-Fi aggregation environment. The trunk should contain the tooling for establish a SOCKS proxy for operating systems without MPTCP support.
geolocator (Software defined GPS)	FF Nordwest	Jan-Tarek Butt	A dynamical and flexible geolocator which should provide a software defined GPS device and receives its position over WIFI hardware with various services like openwifi.su, MLS and etc.
netjsongraph.js: visualization of netjson data	ninux	GeekPlux	NetJSON is a great work attracted some interest from around the world, but there are a lot of defects. And I’m a front-end developer and now focus on Data Visualization, so I want to contribute to NetJSON. In my proposal, I come up with my idea and list some tasks and schedules.
lime-webui: port to LuCI2	libremesh	Marcos Gutierrez	Make an inventory of all LuCI components that LibreMesh uses and that will not be compatible with LuCI2. Analyze which dependencies must be replaced or rewritten and generate the views according to the new framework selected.
Powquty Live-Log	lede	Stefan Venz	PowQuty provides statistics from measurements taken by a USB oscilloscope. The statistics provide information about the power distribution network, but don’t highlight the most important information. Power networks need to comply with EN50160, if this norm is violated, damage to connected devices is possible. To avoid damage, powqutyd will log violations and notify users on occurrence.

GSoC 2017 and Freifunk: Students wanted

Posted on 5. March 2017 by Andreas Bräu

Good news: Freifunk has been accepted again as mentoring organization for GSoC. If you’re a student, please bookmark March 20th in your calendar. That is the date when student application period opens and proposals can be submitted. In the meantime you should visit our ideas page and get in touch with the mentors and our community.

Applications for GSoC

This year we offer much more ideas than in the past but as always we don’t know how many slots (aka projects) we will get from google at the end. So we can’t give you any guarantee for proposals to be chosen. We will select the students and assign them to projects, but the final announcement will be made by the GSoC Program Office.

You aren’t limited to the ideas listed on our ideas page. It is possible to propose your own idea. But keep in mind that you need a mentor from one of our communities, too.

If want to apply please visit our students checklist page before. You can find a lot of information there on how to get in touch with our community. You’ll also find our application template there. It’s necessary to follow that template for a successful application.

During GSoC

There are some things we expect from you:

provide status updates to your mentor and your project, you should specify details with your mentor (how often, what medium to use, …)
commit early and often to the public repositories of your project
sign up our WLANWare mailinglist
join one of our community meetings
write at least 4 blog posts at this blog:
1. During the community bonding period. Deadline is May 30 – introduce your project here, present your ideas, show us your project plan with milestone
2. Before first evaluations. Deadline is June 26 – show your results and explain your status
3. Before second evaluations. Deadline is July 24 – show your results and explain your status
4. Before final evaluations. Deadline is August 21 – finally present your project and show us your result. Maybe it’s already live 🙂

To give you a short prospect of what is planned to get you into the project after successful application:

Timeline

Community Bonding (May)

Use this period to refine your proposal. Add more specific milestones. Try to get into software development, e.g. by fixing small bugs. Talk to your mentor and the developers of your project to introduce yourself and your proposal.

From 26th to the 28th of May (the last weekend before Coding Phase 1 starts) there’s the Wireless Community Weekend (WCW) in Berlin. Here the Freifunk community will meet with their guests to create an unconference and hackathon. You have the chance to learn a lot about free wireless networks. It’s your chance to show your project/idea to a lot of community members. So you can get your first feedback before coding starts.

If you need help with accomodation or travel costs, please don’t hesitate to ask your mentors or organization admins.

Coding phases

There’s something new this year: We have 3 coding phases with 3 evaluations. Both, students and mentors have to complete evaluations to proceed. Maybe you can use these evaluations as milestones of if you want to work agile as sprint dates.

If you can’t make it to the WCW in May there’s another great chance in the first coding period to meet a lot of developers and community members during Battlemesh V10 in Vienna from June 05 to June 10. Focus of this event are routing protocols. You can use that event to get a lot of detailed technical knowledge, e.g. by doing pair programming.

If you have any questions please contact the organization admins.

So let’s have a great summer! 🙂

(Thanks to Clemens who wrote some parts if this text before)

Runderneuerung blog.freifunk.net

Posted on 1. March 20171. March 2017 by Andreas Bräu

Unser Freifunkblog ist auf eine neue Plattform umgezogen. Das alte Drupal 6, das sein Lebensende längst erreicht hat und keine Updates mehr bekam, haben wir abgelöst und die Inhalte nach WordPress migriert. Nun ist das Blog mit neuer Technik unter der Haube und neuem Design für die nächste Zeit gerüstet. Der älteste Artikel im Blog ist im Übrigen über 11 Jahre alt und stammt aus dem Jahr 2005.

Zugang zum Blog

Alle Artikel, Schlagwörter und Autoren sind erhalten geblieben. Frühere Benutzer wurden wieder angelegt, nur die Passwörter haben wir nicht übernommen. Die Anmeldung kann mit der beim alten Blog verwendeten Emailadresse erfolgen. Mit der “Passwort vergessen”-Funktion kann man ein gültiges Passwort erhalten. Bei Problemen mit der Anmeldung schreibt bitte eine Email an das Webteam (web(at)freifunk.net).

Wer noch keinen Account auf diesem Blog hat und einen hier Freifunkartikel veröffentlichen möchte wendet sich bitte auch an das Webteam. Gerade für communityübergreifende Themen eignet sich diese Plattform.

Aufräumarbeiten

Einige Bilder lassen sich leider nicht mehr auffinden, da sie extern eingebunden waren und nicht mehr online sind. Auch der in Drupal integrierte Editor hat manchmal ganz schön viel unsinnigen HTML-Overhead erzeugt und führt auch unter WordPress zu komischen Formatierungen. Diese ließen sich nicht automatisiert aufräumen, das ist wohl mal ein Thema für einen Editathon :).

Implementing Pop-Routing – Final Evaluation

Posted on 22. August 201629. January 2017 by Gabriele Gemmi

Hello again, for two months I have been working on my project and I have achieved all the goals.

An alpha version of my program was released for the mid-term evaluation.Since then I have fixed all the bugs, packaged the program for OpenWRT, tested the code and written the documentation.

Everything is available on the GitHub repository [0]

I structured my work opening by issues for all the bugs I had to fix and for everything I wanted to improve. Then, I organized the issues in milestones. Milestone 0.1 [1] is the one that I had to complete to finish the project. When that milestone was closed I made the branch “v0.1” [2].

Tests

In order to be sure that my program worked correctly. I wrote a simulator in python. It’s made by a small server, with the same interface as OONF’s telnet plugin, and two python libraries written by my mentor: cn_generator [3] and pop-routing[4]. The first one generates synthetic graphs using the Cerda-Alabern[5] algorithm, the second one is a pop-routing implementation in python. In the server I implemented the commands to push the NetJson[6] topology and the ones to receive the timers values.

When prince requests a topology from my test program, cn_generator generates a random graph and pushes it back; meanwhile using the python implementation of pop-routing, the references values are computed. The values received from prince are then compared to ones calculated using python.

The goal of tests is to verify that the difference between the reference and the measured values is always less than 1%.

Measurements

I’m going to use my work to write my Bachelor Thesis, so I wanted to perform some measurements to check how well it worked.

My goal was to implement the algorithm on an embedded device, so I chose to measure the execution time on a “Ubiquiti Picostation M2HP” to see how well it was performing.

I branched Prince [7], modifying the code to measure the time needed to calculate the betweenness centrality and push it back along with the timers.

I used the graph generator to create graphs from 5 to 500 nodes, and I measured the time needed to compute with a sample every 10 nodes. For each sample I ran 5 tests, then calculating the mean and the standard deviation. The results are shown here:

As you can see from the graphic above, the computation time on the embedded device is quite good if we use the heuristic (8s for 100 nodes), it proved to be unusable without it (100s for 100 nodes)

OpenWRT Package

The last objective I gave myself in the previous post was to write a plugin for OLSRd. Since OLSRd isn’t maintained anymore – and since all the developers are working on OONF – I decided to focus on it while avoiding the plugin. Instead, I wrote a makefile and packaged prince for openWRT / LEDE. The makefile hasn’t been published yet in any openWRT feed, but it is hosted in my repository [8]. Instructions are available along with documentation.

Future Work

I’ll keep working on this project, mantaining the code and fixing the future issues. Since the graph-parser library is the last piece of code implemented in C++ and it depends on the Boost libraries, I’m looking forward to re-implement it in pure C.

One of my goals was also to run prince in my WCN (Ninux Firenze), but switching from OLSRd to OONF is taking more time than expected, so I’m hoping to try it in the future.

Gabriel

[0]: https://github.com/gabri94/poprouting/
[1]: https://github.com/gabri94/poprouting/milestone/1?closed=1
[2]: https://github.com/gabri94/poprouting/tree/v0.1
[3]: https://ans.disi.unitn.it/redmine/projects/cn_generator/repository
[4]: https://ans.disi.unitn.it/redmine/projects/pop-routing/repository
[5]: https://pdfs.semanticscholar.org/4ac8/05e7359c6b20c3cdd5da24284d3826b9609c.pdf
[6]: http://netjson.org/
[7]: https://github.com/gabri94/poprouting/tree/exec_time
[8]: https://github.com/gabri94/poprouting/blob/master/Makefile.openwrt

Powquty (PoWer Quality) – GSoC 2016

Posted on 22. August 2016 by Neez El Sayed

Dear Freifunkers,

Now that GSoC 2016 comes to its end, please allow me to update you on the powquty project. Despite some delays regarding the hardware, the goals set for GSoC 2016 have been reached. Here follows a picture that shows a compact demonstration for powquty.

The picture depicts the set-up for powquty, which consits of a LEDE based wireless router, connected to the PC over serial interface, where a Termial session of the router is running. The oscilloscope is attached via USB to the router. On the router’s terminal session we see that powqutyd has been started, with the correspondent messages shown on stdout. These message show the calculated power parameters.

Installation

For installing powquty follow these steps:

add the following line to your feeds.conf in your source directory:

src-git powquty https://github.com/thuehn/powquty.git

update the your feeds, then install powqutyd from feed
Include powqutyd in your config (make menuconfig) by choosing powqutyd from: Utilities –> powqutyd
Save and exit menuconfig
Compile and install powqutyd
scp the ipk file to your router and install it
opkg install powqutyd_0.1-1_<target>.ipk

Note this package depends on the following libraries/packages, that have to be installed before installing powqutyd:

libmosquitto
libconfig
kmod-usb-acm (kernel module)

When successfull the powqutyd package will create:

the binary powqutyd in /usr/sbin
the configuration file in /etc/powqutyd/powqutyd.cfg

Usage
Before running powqutyd you need to configure it.

Powqutyd needs to read the measurement samples from the USB oscilloscope. The USB oscilloscope has to be plugged to the router before running powqutyd. The USB oscilloscope implements the USB Communication Device Class (CDC) device specification. This means that the kernel module kmod-usb-acm will recognize the USB oscilliscope once plugged and will create a tty device probably under /dev/ttyACM0. Depending on your setup this could be different. Check your system logs after plugging the USB oscilloscope to find out the actual path of the tty-device on your setup and adjust the path in the config file of powqutyd (/etc/powqutyd/powqutyd.cfg) accordingly. Note: if the tty-device is not set right the powqutyd will not start!
Powqutyd will send the calculated power quality parameters using MQTT-protocol to an MQTT-broker. This means that powqutyd requires IP connectivity between your router and the MQTT-broker. Of course this is given if you set up an MQTT broker on your router itself, but this is not a requirement, as long as the router has an IP connectivity to an MQTT-broker. For testing purposes we used mosquitto on the router as MQTT-broker. Depending on your setup you need to adjust the mqtt_host config option in your /etc/powqutyd/powqutyd.cfg accordingly. The mqtt_host config option is a string that could contain either the IP-address of the Fully Qualified Domain Name (FQDN) of the MQTT-broker. Note: at the current state, the MQTT-client implemeted by powqutyd uses the port 1883 with no SSL support.
Furthermore the powqutyd’s MQTT-client is an publish only client, thus it will not subscribe and has no will. Nonetheless the topic under which the powqutyd’s MQTT-client publishes needs to be set. This can be done by adjusting the mqtt_topic config option in your /etc/powqutyd/powqutyd.cfg accordingly.
Powqutyd sends three types of messages to the MQTT-broker: i)msg_device_online, ii) msg_device_data, iii) msg_device_offline. These messages are explained below, yet all of them use a common setting which is a (universally) unique id for the devices that communicate with the same MQTT-Broker. This way the MQTT-broker can differenciate between the messages it receives. This device-unique-id is set by the config option dev_uuid
It is possible to print the results messages that powqutyd sends to the MQTT-Broker to stdout. This can be set by the option powqutyd_print. If set to 0 (zero) powqutyd will not print the result to stdout.
Running powqutyd: Once all the configuration above are done powqutyd can be started by typing: “powqutyd &” to your terminal.

Work achieved

For the powquty project, my mentor and I set up a dedicated Github page accessible under the following link:

https://github.com/thuehn/powquty

There the list of my commits can be seen under:

https://github.com/thuehn/powquty/commits/master?author=neez34

Code

The following picture depicts the folder structure of the work done during the porject.

.
├── etc
│ └── powqutyd
│ └── powqutyd.cfg
├── include
│ ├── calculation.h
│ ├── config.h
│ ├── emulator.h
│ ├── helper.h
│ ├── mqtt.h
│ └── retrieval.h
├── lib
│ ├── libpqlib.a
│ └── PQ_App.h
├── Makefile
├── src
│ ├── calculation.c
│ ├── config.c
│ ├── emulator.c
│ ├── helper.c
│ ├── main.c
│ ├── mqtt.c
│ └── retrieval.c
└── test
 ├── mqtt_test.c
 ├── usb_test.c
 └── vserial_test.c

During the project I had to reschedule the tasks planned repeatedly according to the progress and to the other circonstances. The order of the tasks changed significantly from the original plan, yet it was necessary to achieve the Goals of the project. Finally the tasks I did were in the following order:

First I had ported the library to a binary blob that can be used on the router.
Next I ve implemented the MQTT-functionality.
After that, I started working on the emulator, then I started working on the USB communication. For that I followed a trial and error approach, for which I used the folder called test to try out different things. Strictly speaking it is not needed at all for the software to compile and run, yet I kept it for future reference. For the USB comuication I implemented two solutions: One using libusb, and another one using virtual serial port (termios.h), since i wasn’t sure how will the USB oscilloscope will behave on the router and if any driver on the router will provide me with a tty device. luckily the kmod-usb-acm did so, thus i continued the project with the virtual serial port solution.
Once the USB communication was working, the next step was to retrieve the sampled data from the oscilloscope. First trials yielded the first sine signal as shown in the next figure

The picture shows ten sine waves corresponding to 200 ms, with 10kHz sampling rate, formed by 2048 samples. Yet as it can be seen the calibration was missing and the values were between +2500 and -2500. That is because the values (as seen in the background of the figure) where signed short numbers, and uncalibrated. (I used octave for plotting).
Once I was able to get the calibration parameters from the USB oscilloscope, I thereafter applyed them to the samples and got the expected sine – Wave with the Voltage-values expected in EU (RMS = 230 Volts) as shown in the next picture

After successful retrieval of the sampling values, I continued designing the software for powquty. I decided to use a multi-threaded approach with a Thread for sending MQTT-messages, a second for retrieving measurment samples and a third for calculating the power quality parameters. I included a ring-buffer as an interface between the retrieval thread and the calculating-thread.
once all three component were integrated and working together I added the configuration (using libconfig) instead of hard-coded MQTT-Broker address, and hard-coded TTY-device …etc.

Goals and future work

As described in the first blogpost, the goal of the powquty project within GSoC 2016 is to create a LEDE package which ensures the following three functionalities:

Retrieving sampled voltage data from the power quality measurement device via USB
Calculating statistical power quality parameters (such as: avg. voltage, harmonics, ect.) form the retrieved voltage samples
Provisioning of the calculated parameters for retrieval and graphical representation

At this point the powquty project has reached its first big Milestone where the basic functionalities set for the GSoC 2016, have been completely implemented.
Yet the end of one milestone is the begin of the next milestone. This is why I conclude with some propositions for future work on powquty:

Design and implementation of error handling
Migration of the config to uci config
Design and implementation of another log option than stdout
Design and implementation of a Luci App for easy configuration and showing results

Nonetheless, please allow me to express my gratitude to my mentor, Dr. Thomas Hühn, for his outstanding support, and constant availability. Also great thanks are due to Freifunk, whose distinguished voluntary efforts – not only in regard to GSoC – made this possible in the first place. Last not least, great credits are due to Google for encouraging this innovative type of value creation.

Cheers,
Neez

DynaPoint Final

Posted on 22. August 2016 by Tobias Ilte

Hi everyone,

this is the final blog post about DynaPoint. Short recap: I created a daemon which regularily monitors the Internet connectivity and depending on that activates and deactivates the proper access points. That way the handling with APs would become easier as you already could tell the status by the AP’s SSID. I also created a LuCI component for it to make the configuration more easy.

In the past weeks I was able to add some new features, fix bugs and complete the LuCI component. Especially the latter was really interesting and gave me some knowledge about how LuCI works.

In the last post I mentioned that it’s better to verify Internet connectivity by using wget instead of just pinging an IP address.
Consequently I switched from Pingcheck to wget. I also added an option to use curl instead of wget. With curl you also get the option to choose the interface for the connection.
When you provide internet via VPN-interface you can explicitly check the connection of that interface now. The reason why I don’t use curl as default is because of curl’s rather large size. For some routers with only 4 MB of storage it might be too much.

I also added an “offline threshold”, which will delay the switch to offline mode. So for example when you set the interval to 60 seconds and offline_threshold to 5, the switch to offline mode will be made after 5 cycles of checking (=300 seconds).

So how does an example configuration look like?

To use dynapoint just add dynapoint_rule ‘internet’ and dynapoint_rule ‘!internet’ in the desired sections in /etc/config/wireless:

config wifi-iface
    option device ‘radio0’
    option network ‘lan2’
    option mode ‘ap’
    option encryption ‘none’
    option ssid ‘freifunk’
    option dynapoint_rule ‘internet’

config wifi-iface
    option device ‘radio0’
    option network ‘lan2’
    option mode ‘ap’
    option encryption ‘none’
    option ssid ‘freifunk-maintenance’
    option dynapoint_rule ‘!internet’

The configuration of dynapoint takes place in /etc/config/dynapoint:

config rule ‘internet’
    list hosts ‘http://www.example.com’
    list hosts ‘http://www.google.com’
    option interval ’60’
    option timeout ‘5’
    option offline_threshold ‘3’
    option add_hostname_to_ssid ‘0’
    option use_curl ‘1’
    option curl_interface ‘eth0’

All of that can also be configured via LuCI:

If you want to try out DynaPoint for yourself please visit https://github.com/thuehn/dynapoint for more information.

Future work

Currently there is only support for one AP per state. In the next weeks I want to add support for multiple APs per state.
Also I want to add support for more rules. At this time there is only support for one rule “internet”. I want to make this more generic and provide support for custom rules.

Acknowledgements

I want to thank my mentor Thomas Hühn for his excellent mentoring and great ideas during this project.
Also of course thanks to Freifunk for letting me realize this project and thanks to Google for organizing GSoC.

Monitoring and quality assurance of open wifi networks out of client view (final evaluation)

Posted on 22. August 2016 by Jan Tarek Butt

Hi together,

now the time has come to explain the full Google Summer of Code Project. In both blog posts before I explained the work packages and structure of the Project [0][1]. In the first post I declare the three main subjects. Here is a short overview to remind of the project structure:

sub-projects,

Mainline project

and seminars.

The sub-projects are background work for community projects.

The mainline Google Summer of Code project is to develop a new firmware for routers, based on LEDE [2]. The third point are seminars for enlightment of technical aspects of the Freifunk Community.

First I would like to list up all sub-projects and their status.

The first of the sub-projects is the hoodselector. For a final explanation of this construct I would like to explain the following points to give you a good understanding of this concept. On the Nordwest Freifunk community we had one big problem. Due to the batman-adv management traffic, the network setup is not really scalable. This problem also exists on many other communities where they have thousands of mesh routers inside one single network. If there are too many routers inside of one layer 2 network the batman-adv management traffic will flood this network and make it useless.

Therefore in the Nordwest Freifunk Community we decided to develop the concept of hood-networking. This concept consists of two main components: The hoodfile [3] and the hoodselector[4]. The hoodfile is a json file containing all informations necessary for a definition of a geobased hood. One hood is defined by a geostationary fixed quadrants, VPN-peering- and wireless configurations. Inside a hoodfile are multiple hoods defined[5]. Following an example of a definition of a hood:

{

“name”: “rastede”,

“bssid”: “02:00:0A:12:A0:00”,

“defaulthood”: false,

“servers”: [

{

“host”: “ras01.sn.ffnw.de”,

“port”: “10000”,

“publickey”: “ca1b5487ffc20a1f90e0ac14e835d84ab9e52612b5ca62e073d0a13dad98775e”

}

“boxes”: [

[

53.22,

8.09

[

53.36,

8.32

]

A hood has the following definition:

– name

The name describes the region depending on its geo coordinates. For example, if you create a hood over the city Oldenburg(Oldb) (Germany), a good name could be Oldenburg or ol as short. Every name has to be unique inside a hoodfile. Redundant names are not allowed!

– bssid

The bssid will be set for the adhoc wireless interface. This is the main part of splitting the layer 2 network. Inside the bssid there is the IPv4 sub-network encoded which is in use inside the hood. In the above json part the following IPv4 sub-network is encoded “10.18.160.0/21” dec to bin => “0000’1010 0001’0010 1010’0000 0000’0000” bin to hex => 0x0A 0x12 0xA0 0x00 hex to mac => 02:00:0A:12:A0:00. Therefore the bssid should also be unique.

– defaulthood

The defaulthood boolean is only true on the default hood. The default hood doesn’t have coordinates and is the inverted form of all other hoods with geo coordinates.

– servers

Contains an array of VPN connection informations. These informations are:

VPN server address (host)

VPN port

VPN crypto key

One VPN server should be used for one hood only! If two hoods have the same VPN server, batman-adv will loop them over VPN.

– boxes

This 3 dimensional array describes the geographical size of the hood. The surface is rectangular. You just need two points per box to reconstruct it. Here is an example:

53.22,

8.09 #____________

| |

|____________|

# 53.36,

8.32

Each hood can have any number of rectangles inside the boxes array.

To make your life a little easier you can use the hoodgen[6] and source[7] to write your json with the required informations. This is a simple web visualization tool to plan hoods and generate the right json format for the hoodfile. This tool has been written by Eike Baran. Big Thanks to him for this helpful tool!

Now to the hoodselector.

It is a software that creates decentralized, semi automated ISO OSI layer 2 network segmentation for batman-adv layer 2 routing networks. This program reads the geobased sub-networks called hoods from the above mentioned hoodfile. The decision of choosing the right hood is made on following points: first, the hoodselector checks, if the router has a VPN connection. If it has, the hoodselector then checks, if a static geoposition was set on the router. If not, it tries to get a position using wireless based localization with the so called geolocator. The geolocator [8][9][10] is a software which makes it possible to receive a position based on wireless networks “seen” around. These informations will be sent to the openwifi map project [11]. Knowing the position of the router the hoodselector can find the right hood, because each hood is defined with geocoordinates. If the Router doesn’t have a VPN connection e.g. as a mesh only router, the hoodselector triggers a WIFI scan and searches for neighboured mesh routers in other hoods. If there is an other router with a different BSSID but with the same mesh SSID, the router chooses it’s hood based on the neighboured BSSID. I got much positive feedback from many other Freifunk Communities. Someone even created a integration request issue for gluon [12]. Gluon is a framework based on openWRT[13] and is very popular in the Freifunk community [14]. Before I will send this as a patch to gluon there remains one last urgent issue [15]. The current hoodselector is not able to handle mesh on LAN or WAN connections. So there is still a potential point of failure. Because persons who are not familiar enough with the hood-networking concept can accidentally interconnect hoods over the mesh on cable functions. I plan to fix this problem up to mid of september. When this issue is closed I would like sending patches for integration to gluon. Other issues can be found here[16].

On the Nordwest Freifunk community we currently have 10 active hoods including a default hood. That is a special hood where all routers will connect to, if they are not able to choose a hood, including also routers there out of ranges from other real hoods. After the last 3 months we can safely say that the setup works. Commits can be found here[17] All currently active hoods can be see here in this picture.

As next I would like to tell you about the second sub-project as a prework for the mainline project. In that part I work on a proper workaround with the continuous-integration (CI) system of Gitlab[18]. As I explained in the midterm evaluation, on our Nordwest community we started automatically building of nightly testing firmware images for our community firmware. The CI works now with a dynamical multiple core build processes and auto generated architecture targets out of source. At the moment it is not possible for Gitlab to handle high verbose inside the web-engine while the build process. I discussed the problem with the gitlab team and open an issue [19]. The CI builder is very helpful for the developing process of the monitoring drone. Here you can see the result for the local community image[20] and for the monitoring-drone [21].

Mainline Project.

The mainline project was to create a new firmware for monitoring and quality assurance of open wireless networks. So I started reading of informatins about openWRT [13] and LEDE [2]. I decided to use LEDE as base system. I know there still no release to use this as a defined base structure (we are all looking forward to this moment) but since july 2016 I am on the developer list and the way where LEDE is growing looks good. Next I looked for a build management script. First I thought about using make and Makefiles but this was not my favourite, so I decided to use the buildscript from the Franken Freifunk community as base, which is written in bash. Now I’ll explain the structure how to work with and use it. The following directories and files are important for basic work:

buildscript ← File

BSP ← Dir

Community ← Dir

build_patches ← Dir

modules ← File

The buildscript is mainly a bash script for a humanly working with this buildroot. In other words it is an abstraction from the LEDE build ENV.

Inside the BSP directory are all necessary architecture specific informations.

BSP means Board-Support-Package. Also inside this directory are default informations like the shell banner system configs and so on.

The community directory includes community specific configurations, similar like the gluon siteconf [22]. Currently there are only two config parameters inside: first the “AP SSID” to set a default SSID with witch WIFI network should the monitoring-drone connect and the second parameter, the “AP BSSID” to set a node specific BSSID in case if more than one router with the same SSID is present. Then the monitoring drone is pinned on one specific node. This config parameter will be dropped in the future because it is not really effectively if a default BSSID is set . In future I plan to configure thous parameters over an extra web interface[23].

In the build_patches directory you can put patches for LEDE or if you what you can also put patches for each package repository. Here is a schemata:

build_patches

lede

0001-this is a patch.patch

0002-this is another one.patch

ffnw-packages

0001-this is a patch.patch

0002-this is another one.patch

The last file is called modules. Inside this file you can add external package repositories and also select specific packages out of this repositories. Following an example:

OPENWRT=(ffnw-packages

$PACKAGEURL

$PACKAGEREV)

OPENWRT_PKGS=”libwlocate lwtrace ffnw-node-info hoodselector”

Clemens and I discussed about the API design[24] regarding the communication between the monitoring drone and the netmon core. So we met together. Here a picture:

At last point were the seminars. During the Google Summer of Code we started to gave seminars for technical aspects of Freifunk because we have not enough developers and system administrators. That is mostly a big problem in volunteer activities. On the hacking sessions we follow a simple structure:

– two lectures about Freifunk technical aspects.

– discus about the contend of the lectures

– work session on projects

On the first hacking session at the 28. may 2:00 PM we created video recordings of the two lectures, you can find them here[25]. The next hacking session were failures because of Clemens and my exams. In future there will follow other streams about tecnical aspects of Freifunk.

Last but not least, my future plans:

For the hoodselector, I plan to close up the last urgent issue[15] that I mentored in the above for the migration into gluon I also started prework on gluon for this. One part was the implementation of a sequential code minifying process at compile time [26]. Also some other issues are still open so I will continuing the work on the hoodselector[16].

For the monitoring firmware they currently is just configurable over ssh. A web interface should follow soon and also a plugin system for community specific monitoring data requests.

On the Kieler Linux information days[27] inside the Kieler Innovations- and technology center I will hold amongst others 4 presantations about Freifunk relevants themes:

Hoodselector – Network segmentation for Layer 2 routing at 11:00 (16.09.2016)[28]

Wireless-based localization (openwifi.su project) at 13:00 (16.09.2016)[29]

OpenWRT Embedded Linux distribution at 16:00 (16.09.2016)[30]

Freifunk Kiel/Nordwest (2016) – year review at 16:00 (17.09.2016)[31]

[0] http://blog.freifunk.net/2016/monitoring-and-quality-assurance-open-wifi-networks-out-client-view

[1] http://blog.freifunk.net/2016/monitoring-and-quality-assurance-open-wifi-networks-out-client-view-midterm-evaluation

[2] https://lede-project.org/

[3] https://git.nordwest.freifunk.net/ffnw-firmware/packages/blob/master/hoods/files/lib/ffnw/hoods/hoods.json

[4] https://git.nordwest.freifunk.net/ffnw-firmware/packages/blob/master/hoodselector/luasrc/hoodselector

[5] https://git.nordwest.freifunk.net/ffnw-firmware/packages/commits/master/hoods

[6] http://hood.ffnw/hoodgen.html#

[7] https://git.nordwest.freifunk.net/ffnw-server/hoodgen

[8] https://git.nordwest.freifunk.net/ffnw-firmware/packages/tree/master/libwlocate

[9] https://git.nordwest.freifunk.net/ffnw-firmware/packages/tree/master/lwtrace

[10] https://git.nordwest.freifunk.net/ffnw-firmware/packages/tree/master/ffnw-node-info

[11] http://openwifi.su/

[12] https://github.com/freifunk-gluon/gluon/issues/789

[13] https://openwrt.org/

[14] http://gluon.readthedocs.io/en/latest/

[15] https://git.nordwest.freifunk.net/ffnw-firmware/packages/issues/63

[16] https://git.nordwest.freifunk.net/ffnw-firmware/packages/issues?label_name%5B%5D=hoodselector

[17] https://git.nordwest.freifunk.net/ffnw-firmware/packages/commits/master/hoodselector

[18] https://about.gitlab.com/gitlab-ci/

[19] https://gitlab.com/gitlab-org/gitlab-ce/issues/18039

[20] https://git.nordwest.freifunk.net/ffnw-firmware/siteconf/builds

[21] https://git.nordwest.freifunk.net/ffnw-firmware/monitoring-drone/builds

[22] https://github.com/freifunk-gluon/gluon/tree/master/docs/site-example

[23] https://git.nordwest.freifunk.net/ffnw-firmware/monitoring-drone/tree/master

[24] https://git.nordwest.freifunk.net/groups/netmon-sc

[25] https://www.youtube.com/channel/UCX0nJzimLNF38pfgQIuZLEQ

[26] https://github.com/freifunk-gluon/gluon/commits/master?author=2tata

[27] http://www.kilux.de/

[28] http://www.kilux.de/index.php?seite=programm.html&untermenu=Besucher-Info#248

[29] http://www.kilux.de/index.php?seite=programm.html&untermenu=Besucher-Info#246

[30] http://www.kilux.de/index.php?seite=programm.html&untermenu=Besucher-Info#247

[31] http://www.kilux.de/index.php?seite=programm.html&untermenu=Besucher-Info#263

SWOON: Simultaneous Wireless Organic Optimization within Nodewatcher – Final steps.

Posted on 21. August 2016 by David Marn

This blog post concludes the work I did during the Google Summer of Code 2016.

Spending the summer on nodewatcher was a pleasantly laborious exercise. Although my proposal concerned a development of a new algorithm, I realized that I also had to prepare various other components so that the algorithm would work. I started by sketching a short plan of different components I had to integrate for my algorithm to work. To remind the readers, SWOON is an algorithm for an efficient allocation of wireless frequencies in a densely populated area where many wireless access points interfere with each other. Before any work could begin, I had to install a local version of nodewatcher. I did not want to interfere with wlanslovenija’s mature network and I needed a small network of my own to run these tests. To find such a network, I collaborated with a non-profit organization based in Berkeley, California. They provided me with access to a building outfitted with more than 10 wireless access points. The wireless access points in this testbed are extremely diverse. They come from various manufacturers and not all broadcast on the 5GHz band. This allowed me to work with issues unique to community networks: software has to be built ground-up to be extremely modular and it has to support a diverse range of equipment, since there is no central planning authority.

I started with writing the data collection module, on which I have reported in my previous blog posts. Once I was able to work with the data, I worked on a different algorithm which detected rogue nodes in networks. This is especially useful for network maintainers who now have information on which nodes are most harmful to their network. They can now significantly reduce interference by working together. Algorithms on this scale have to be extremely efficient: I derived a complexity bound of O(E * log(V)) where E is the number of edges and V is the number of vertices in a graph. This allows the algorithm to scale all the way to wlanslovenija’s network or more, as it is almost linear. In order to achieve such scalability and efficiency, I had to peruse state-of-the-art graph theory research and utilize complex data structures (the one I used is called a disjoint-set data structure) to quickly order datasets in accordance with our needs.

I then moved on to the main algorithm, which took weeks to even rigorously define, let alone code up. I first considered implementing the Beigel-Eppstein coloring algorithm, which is known to run in ~O(1.3^n) time, but the complexity of the algorithm rendered it impractical. It relies on 19 different subcases which are iteratively used to simplify the constraint satisfaction (CSP) instance. Instead, I reverted to a custom implementation of a state-of-the-art greedy approach, which had to be modified from the source code of NetworkX, a leading open-source graph manipulation package. I devised a custom way of ordering the nodes in a graph to correctly prioritize wireless access points. The algorithm starts with looking at all the ‘neighbors’ (other nodes your radio detects) and computes the ‘busyness’ of each frequency band. We then minimize joint interference through the algorithm.

We observed an interesting property when running this algorithm on our wireless testbed. Most frequencies were moved to minimize interference, but some got worse. At first, we thought it was a bug, but we did eventually discover that some nodes must pick up more interference for others to remain noiseless. This perfectly coincides with our goal, which is to minimize the interference over all nodes, not every single node.

Next steps

One major problem with the algorithm is how unreliable ‘signal’ data is. We were thinking of coming up with a better metric to measure link quality instead of using signal strength. We were considering using the number of packet collisions instead. The modular design allows us to painlessly replace the metric without having to understand the entire algorithm.

We also noticed openWRT’s iwinfo doesn’t provide information about channel widths of access points. Adding this data would allow more accurate calculations instead of assuming all channels are 20MHz wide.

Finally, nodewatcher doesn’t yet have support for issuing node-specific warnings. I’d love to use such a warning to let a maintainer know that their node needs to change the channel.

However, I am confident this algorithm will enhance our user’s experience.

All my contributions can be seen on this link:
https://github.com/wlanslovenija/nodewatcher/commits/development?author=CdavM

If you’re interested in the timeline of the development, look at the following pull requests on nodewatcher’s GitHub:
Bug fixes related to a local installation of nodewatcher:
https://github.com/wlanslovenija/nodewatcher/pull/18
https://github.com/wlanslovenija/nodewatcher/pull/20
Data collection module:
https://github.com/wlanslovenija/nodewatcher/pull/22
Command that users can use to export data from the data collection module:
https://github.com/wlanslovenija/nodewatcher/pull/23
Improving the data collection module:
https://github.com/wlanslovenija/nodewatcher/pull/24
Algorithm that detects rogue nodes in one’s network:
https://github.com/wlanslovenija/nodewatcher/pull/25
Additional options for user-friendly data export command:
https://github.com/wlanslovenija/nodewatcher/pull/26
Testing framework for the rogue node algorithm + improvements:
https://github.com/wlanslovenija/nodewatcher/pull/27
Channel allocation algorithm implementation (not merged yet):
https://github.com/wlanslovenija/nodewatcher/pull/28
Updated data in nodewatcher/core:
https://github.com/wlanslovenija/nodewatcher/pull/29

GSoC: The ECE configuration system – summary

Posted on 19. August 2016 by NeoRaider

The Google Summer of Code is almost over, so in this blog post I’ll give a overview over the targets I’ve met (and those I haven’t).

Code repositories

https://gitlab.com/neoraider/ece/commits/gsoc2016 (daemon, client libraries, CLI client)
https://gitlab.com/neoraider/pkg-ece/commits/gsoc2016 (OpenWrt/LEDE package feed)
https://gitlab.com/neoraider/uci-ece/compare/gsoc2016-upstream…gsoc2016 (UCI ECE backend)

All code in the first two repositories has been developed by me during the GSoC. The third link shows the work I’ve done to integrate a ECE backend into libuci.

What is working

As described in earlier posts, my GSoC project was a configuration storage system for OpenWrt/LEDE, trying to solve various issues of the UCI config system. The principal points of this new system are

ubus-based config daemon maintaining a central storage database file
JSON-based configuration data model
Validation based on simplified JSON-Schema

The Wiki at https://gitlab.com/neoraider/ece/wikis/home gives a good overview of the design and the usage of ECE and describes many features in detail. The pkg-ece package feed can be used to build and install the different components of ECE on OpenWrt/LEDE easily.

If you’ve worked with OpenWrt/LEDE, you probably know the UCI config system. A UCI config file looks like this:

config system
        option hostname 'lede'
        option timezone 'UTC'

This format is very simple: Each file (called “package”) has a number of sections (named or unnamed) of different types (this example from the “system” package has a single unnamed section of type “system”). These section contain options with single values or lists of values.

Unnamed sections are usually accessed using indices, for example a command to set the hostname would look like this:

uci set system.@system[0].hostname='betterhostname'

With the simplicity of UCI, there come various issues and missing features; these are only a few of them:

The fixed data model (package/section/option) makes some kinds of configuration very awkward: In the example above, the index 0 must be given for the system section, but having a second section of this kind would not make sense. In other cases, deeper configuration trees must be flattened to be stored in UCI, making the configuration harder to understand
All values in UCI are strings, which often causes inconsistencies (booleans are usually stored as ‘0’/’1′, but several other pairs like ‘false’/’true’ and ‘off’/’on’ are supported as well; different users of UCI sometimes parse numbers differently)
UCI doesn’t have built-in validation. Frontends like LuCI usually validate the entered data, but as soon as the CLI client is used, no validation is done.
UCI always stored the whole configuration file and not only changes from the defaults, making the storage inefficient on overlay-based filesystem setups as they are common on OpenWrt/LEDE
In some situations, upgrades to default values should also affect the effective values; but only if the user didn’t change the values themselves. With UCI, this is not possible, as it doesn’t store the information if a value was changed by a user.
UCI allows comments in config files, but they are lost as soon as libuci or the CLI tool is used to modify it

The configuration given above could be represented in ECE as this JSON document:

{
  "system": {
    "hostname": "lede",
    "timezone": "UCI"
  }
}

Note that this is only the external representation of the configuration; internally, it is stored in a more efficient binary format.

JSON gives us a lot of features for free: arbitrary configuration trees with proper data types. Existing standards and standard drafts like JSON Pointer and JSON Schema can be used to reference and validate configuration (the JSON Schema specification is simplified for ECE a bit though to allow more efficient validation on embedded systems).

The command for changing the hostname would look like this in ECE:

ece set /system/hostname '"betterhostname"'

The quoting is currently necessary to make the string a valid JSON document; this may change in a future version.

The whole configuration is saved in a single JSON document, but the specific format is not defined by a single schema; instead, each package can provide a schema, and the configuration tree is validated against a merged schema definition.

The schemas also provide default values for the configuration. Adding documentation for the configuration options to the schemas is planned as a future addition and might be used to support the user in configuration utilities and automatically generate web-based or other interfaces.

This gives only a small example for the usage of ECE, the abovementioned ECE Wiki contains much more information about the usage of ECE and the ideas behind it.

In addition to the daemon and a simple CLI utility, I’ve developed libraries for C, Lua and Shell which allow to access the configuration. While there are still some features missing (some points for future work are given in https://gitlab.com/neoraider/ece/wikis/todo ), I think most of the missing pieces can be added in the near future.

The UCI/ECE bridge

When I proposed my project for the GSoC, I didn’t aim at making it a full replacement for the current UCI system, at least not in the near future. While the possibility to move some of UCI config files into the ECE config database had been my plan from the beginning, my ideas for backwards compatibility didn’t go further than a one-time import from UCI to ECE, and one-way generation of UCI config files from ECE.

After talking to a few LEDE developers and package maintainers, it became clear to me and my mentors that many people are interested in replacing UCI with a better system in the not-too-far future. But for ECE to become this replacement, a real two-way binding between UCI and ECE would be necessary to allow gradual migration, so configuration utilities like LuCI (and many other utilities somehow interacting with UCI) don’t need to be adjusted in a flag-day change.

An incomplete design draft for this UCI/ECE bridge has been outlined in https://gitlab.com/neoraider/ece/wikis/design/uci-bridge . The code found in the UCI ECE backend repository linked above implements a part of this bridge (it can load “static” and “named” bindings from ECE into UCI, and commit “static” bindings back to ECE) and has been implemented as an API- and ABI-compatible extension to libuci. The development of this bridge has taken a lot of time (much more time than I had originally scheduled for UCI compatibility features), as the data models of UCI and ECE are very different.

Future work

Of all points given in https://gitlab.com/neoraider/ece/wikis/todo , finalizing the database format is the most important, as any future change in the storage format will either break compatibility or involve some kind of conversion. When it is clear the format won’t be changed anymore, ECE should be added to the OpenWrt community package repository to make it easily accessible to all OpenWrt and LEDE users.

After that, other points given in the TODO should be dealt with, but none of those seem too pressing to prevent actually using ECE for some software (but some of the points given in the first section of the TODO page would need to be addressed to properly support software that requires more complex configuration).

Last, but no least, I’d like to express my gratitude to my mentors and all people in the OpenWrt, LEDE and Freifunk communities who have helped me develop ECE by giving guidance and lots of useful feedback, and to Google, who allowed me to focus on this project throughout this summer.