openwrt

DynaPoint update

Hi everyone,

here are some updates regarding DynaPoint. The idea was to create a daemon, which regularily checks the Internet connection and changes the used access point depending on the result. That way the handling with APs would become easier as you already could tell the status by the AP's SSID.

A daemon with basic functionality is already working. After installation, there is one configuration step necessary.
You have to choose in /etc/config/wireless which AP should be used, if Internet connectivity is available and which one if the connectivity is lost. You can do that by adding "dynapoint 1" or "dynapoint 0" to the respecive wifi-iface section.

You can also configure dynapoint via LuCI, although it's not yet complete.
I was struggeling a bit with it, because the documentation of LuCI is a bit minimalistic...
Here is a screenshot of how it currently looks like:

Next steps

To verify Internet connectivity, it is probably better to make a small http download than just ping an IP address.
Using "wget --spider" should be suitable for that.

Also, I will see if I can get rid of the required configuration step in /etc/config/wireless in the next weeks and provide fully automatic configuration.

If you want to test dynapoint for yourself, just go to https://github.com/thuehn/dynapoint.

Google Summer of Code 2016: External netifd Device Handlers - Milestone 1

OVERVIEW OF THE LAST WEEKS
During the last 5 to 6 weeks I have implemented the possibility to include wireless interfaces in Open vSwitch bridges, rewritten a lot of the code for creating bridges with external device handlers and brought my development environment up to speed. I am now working with an up-to-date copy of the LEDE repository.
I have also implemented the possibility for users of external device handlers to define what information and statistics of their devices look like and to query the external device handler for that data through netifd.

CHANGES TO THE DEVELOPMENT ENVIRONMENT
So far, I have been using quilt to create and manage patches for my alterations of the netifd code.
One day they were completely broken. I still do not know what and how it happened but it was not the first time and this time recovery would have been way too tedious.
This is why I switched to a git-only setup. I now have a clone of the netifd repo on my development machine that is symlinked into the LEDE source tree using the nice 'enable package source tree override' option in the main Makefile. I used the oppportunity to update both the LEDE source tree and the netifd repository to the most recent versions.
Before, I was working on an OpenWRT Chaos Calmer tree, because of a bug causing the Open vSwitch package to segfault with more recent kernels.
Now, everything is up-to-date: LEDE, netifd and Open vSwitch.

MY PROGRESS IN DETAIL

More Dynamic Device Creation
In actual coding news, I have refined the callback mechanism for creating bridges with external device handlers and the way they are created and brought up.
Previously, a bridge and its ports were created and activated immediately when /etc/config/network was parsed. Now, the ubus call is postponed until the first port on the bridge is brought up.
Because of the added asynchronicity, I had to add a 'timeout and retry'-mechanism to keep track of the state of the structures in netifd and the external device handler.

A few questions have come up regarding the device handler interface. As I have explained in my first blog post, I am working on Open vSwitch integration into LEDE writing an external device handler called ovsd. Obviously, this is very useful for testing as well.
I have come across the issue of wanting to disable an bridge without deleting it. This means bringing down the bridge and removing the L2 devices from it. The device handler interface that I mirror for my ubus methods doesn't really have a method for this. The closest thing is 'hotplug remove', which using feels a bit like a dirty hack to me.
I have reached out no netifd's maintainer about this issue. For the meantime, I stick to (ab)using the hotplug mechanism.

On the ovsd side I have added a pretty central feature: OpenFlow controllers. Obviously, someone who uses Open vSwitch bridges is likely to want to use its SDN capabilities.
Controllers can be configured directly in /etc/config/network with the UCI option 'ofcontrollers':

config interface 'lan'
        option ifname 'eth0'
        option type 'Open vSwitch'
        option proto 'static'
        option ipaddr '1.2.3.4'
        option gateway '1.2.3.1'
        option netmask '255.255.255.0'
        option ofcontrollers 'tcp:1.2.3.4:5678'
        option controller_fail_mode 'standalone'

The format in which the controllers are defined is exactly the one the ovs-vsctl command line tool expects.
The other new UCI option below ofcontrollers configures the bridge's behavior in case the configured controller is unreachable. It is a direct mapping to the ovs-vsctl command 'set-fail-mode'. The default behavior in case of controller absence is called 'standalone' which makes the Open vSwitch behave like a learning switch. 'secure' disables the adding of flows if no controller is present.


Function Coverage: Information and Statistics
Netifd device handlers have functions to dump information and statistics about devices in JSON format: 'dump_info' and 'dump_stats'. Usually, these just collect data from the structures in netifd and the kernel but with my external device handlers, it is not as simple. I have to relay the query to an external device handler program and parse the response. Since the interface is generic, I cannot hard-code the fields and types in the response. This is why I relied once more on the JSON data type description mechanism that I have already used for dynamic creation of device configuration descriptions.
In addition to the mandatory 'config' description, users can now optionally provide 'info' and/or 'stats' fields. Just like the configuration descriptions they are stored in the stub device handler structs within netifd where they are available to serve as blueprints for how information and statistics coming from external device handlers have to be parsed.

For my Open vSwitch setup, it currently looks like this in /lib/netifd/ubusdev-config/ovsd.json:
{
    "name" : "Open vSwitch",
    "ubus_name" : "ovs",
    "bridge" : "1",
    "br-prefix" : "ovs", 
    "config" : [
        ["name", 3],
        ["ifname", 1],
        ["empty", 7],
        ["parent", 3],
        ["ofcontrollers", 1],
        ["controller_fail_mode", 3],
        ["vlan", 6]
    ],
    "info" : [
        ["ofcontrollers", 1],
        ["fail_mode", 3],
        ["ports", 1]
    ]
}

This is how it looks when I query the Open vSwitch bridge 'ovs-lan':



THE NEXT STEPS

During the weeks to come I want to look into some issues which occurred sometimes when I disabled and re-enabled a bridge: Some protocol-realated configuration went missing. This could mean that sometimes the configured IP address was gone. Something which could help me overcome the problem is also in need of some work: reloading/reconfiguring devices.
Along with this, I want to get started with the documentation to prepare for the publication of the source code.

OpenWrt - poWquty (poWer quality): Computing and providing power quality information on OpenWrt routers.

Dear Freifunkers,

Please allow me to introduce the poWquty Project within Google Summer of Code 2016 at Freifunk.

The big picture behind this project relates to the energy production and consumption. Sustainable energy production and consumption are crucial for a prospering life on earth. The importance of energy led many theorists to even define the level of civilization by its energy profile. With the renewable energies shift the energy production paradigm from centralized to decentralized energy production which poses one of the next big challenges, which will influence the energy production in the future years.

From the big picture we move to the concrete case, increasingly faced when dealing with renewable energies: monitoring and control.
The emerging smart grids include an inherent need for communication for monitoring and control purposes in a more and more dynamic environment. One of the major challenges is monitoring the power quality parameters in a decentralized manner. In such case, decentralized information need to be retrieved and transported with the lowest latency possible. One way to solve this challenge could be to install expensive infrastructure to each point. The better way is to use wireless mesh infrastructure that could also serve this purpose.

Here where Freifunk comes in: The Freifunk mesh network is an outstanding example for a decentralized infrastructure that could be augmented with grid related functionalities to cope with future energy challenges. In order to use wireless mesh networks such as Freifunk for energy monitoring, we could use extra hardware that does the power measurements and use the wireless networks solely for transporting the information. The drawback of this is the need to install separate hardware. But, since all routers run on power, we could integrate the measurements into the router, which is the main goal of this project: to enable power quality measurements on OpenWrt.

Here is the initial plan how to do this. First we need to retrieve voltage samples from the power grid. For the beginning we will rely on an oscilloscope device that delivers the real time samples over a USB interface. This way voltage samples from the electric socket are retrieved at the router. With these voltage samples we can go ahead and calculate the power quality parameters, using moving window algorithms, fourrier transform, and z-transform to get the phase angle, the effective power, the frequency, and the harmonics. This calculation should be time, and memory efficient since it has to run on the OpenWrt embedded devices. Once these values are calculated we need to think about how we want to make them available for retrieval over IP networks.

Now we come to the Code: The goal of the project is to create an OpenWrt package which ensures three functionalities:
1-    Retrieving sample data from the measurement device
2-    Calculating power quality parameters form the retrieved samples
3-    Provisioning of the calculated parameters for retrieval

This project is intended to strengthen the role of open software in the uprising smart grids by providing some essential functionalities, communication devices need to have in the context of smart grids, especially in regard to the future role of the home routers in the future energy solutions.

More updates on this will follow in the next weeks.

Cheers,

Neez

GSoC: A new configuration system for OpenWrt/LEDE

Hi,
I'm Matthias (aka NeoRaider), and this year, I'll participate in the Google Summer of Code for the Freifunk project.

The goal of my project is to develop an alternative to the UCI configuration system, as UCI has a number of issues that make it cumbersome to use in some situations.

One of the basic issues of UCI that affects many Freifunk (or generally community mesh) firmwares is the upgrade behaviour. Mesh firmwares usually contain elaborate default configurations, which set up network interfaces and other things to allow participating in a mesh without deep knowledge of the setup.

But this setup needs to change from time to time, as the firmware is upgraded. In the Gluon firmware framework, we usually solve this by providing upgrade scripts which modify the configuration after flashing the firmware. Writing these script is often a tedious task, and the scripts easily break when the configuration differs too much from the expected one.

But the ability to change the configuration is important for many Freifunk users: They want to change the role of ethernet ports, WLAN settings, and a lot more. But UCI doesn't provide information how a setting was changed: if a script encounters an unexpected value, it can't find out if it is an old default value, or was changed by the user. This often leads to a difficult choice for the script author: either to overwrite the value unconditionally, maybe disregarding voluntary configuration changes, or not to overwrite it, rendering communities unable to change certain configuration during upgrades.

My project aims at solving this by saving the user configuration independently of the defaults provided by packages. This way, a package upgrade can change the default values, but explicit user configuration will not be affected.

Another issue is that the upgrade scripts are usually part of the packages that bring the configuration. Removing a package will leave the configuration behind, which is usually a good thing (for users which know about this and may be interested in the the old config), but for mostly-automatic upgrades, old configuration may accumulate, which can quickly become problematic on devices with very limited flash.

I plan to fix this by basing the new configuration system on schema definitions, which specify which configuration options and values are valid. The schema will probably be based on JSON, as there are already lots of existing systems for defining JSON schemas, which may be used for my project or at least serve as inspiration. This will also finally provide real datatypes for the configuration and make things more consistent (finally no more 1/true/on/yes/0/false/off/no booleans!). If we want too keep the package/section/option organization of UCI, or rather allow defining schemas for any JSON structure, is still a subject of debate.

Instead of going into detail even more in this post, I'll provide a link to my Gitlab project: https://gitlab.com/neoraider/ece

Design documents, examples for configuration and schemas, and first code will start to appear there very soon Laughing

DynaPoint - A dynamic access point validator and creator

Hi everybody,

 

I am Tobias, a Computer Science student at the TU-Berlin. I am glad to have the opportunity to participate at GSoC for Freifunk this year.

 

My project aims at making the handling with access points in OpenWrt/LEDE easier. The goal is defined as follows: Find an easily configurable solution (with reasonable defaults) for making the wireless access SSID in OpenWrt/LEDE dependent on a set of network conditions.

 

What does that mean? Consider the following example. You have a wireless access point with SSID "Freifunk". Suddenly for whatever reason the AP looses Internet connectivity without anyone noticing it. When users now connect to this AP, expecting a working Internet connection, they get frustrated, because they can't check their emails or surf the Internet.

 

With DynaPoint I want to develop a daemon, which regularly monitors the Internet connectivity. When it's lost, the SSID will automatically be changed. In this example it could be changed to "Freifunk-offline". When Internet connectivity is re-established, the SSID would automatically be changed back to "Freifunk".

This way users as well as admins get informed about the state of an access point just by looking at the SSID.

 

To verify Internet connectivity the first obvious step would be to do a ping. For this purpose there already exists a package called pingcheck, which I am planning to use. Further steps could include DNS-Queries and HTTP-Downloads.

 

Speaking about easy configuration and reasonable defaults, I want to require as little configuration steps as possible, but also provide enough configuration options to be adjustable to different kind of setups. Ideally the configuration will also be possible via the LuCI web interface.

 

Until next time,

 

Tobias

SWOON: Simultaneous Wireless Organic Optimization within Nodewatcher

Hi everyone,

I will contribute to one of the Freifunk projects; nodewatcher, via Google Summer of Code this summer and I wanted to keep you updated on my progress as well as exchange thoughts about my ideas.

First of all, nodewatcher is an open source, modular community oriented platform used for network planning, node deployment, node monitoring and maintenance. nodewatcher was initially developed to be primarily used by the wlan slovenija project. With 1336 nodes, it's really successful and a great example for community networks. As nodewatcher gets deployed elsewhere with even more nodes, it's natural to ask ourselves if we can be smarter about allocating spectrum to our wireless nodes - these nodes are mostly inexpensive wireless routers but it's natural to extend the meaning of the term to dedicated wireless access points (i.e. Unifi AP).

The theoretical foundation for this problem is fascinating by itself: Each node has a different amount of noise in each channel (the 2.4GHz band allows 3 non-overlapping channels where each channel is 20MHz wide) and each node wants to maximize its SNR (signal-to-noise ratio). I will term this as the greedy approach, which is already used in enterprise level devices. However, in an urban setting, nodes are close enough to each other for their signal to act as noise to other nodes. The greedy approach is no longer optimal as it bears a high price of anarchy. Instead, our goal is now to maximize the sum of channel capacities (under a power constraint). I will have to devise an algorithm to solve this problem and the algorithm does not seem trivial since the number of combinations is increasing exponentially with the number of nodes in the system. Even with only 10 nodes, we haveover 59000 possible allocations on 2.4GHz band and over 95 trillion on the 5GHz band.

Traditional networking literature tackles this optimization problem with Lagrange multipliers. An alternative is to look at approximate graph coloring schemes and compute chromatic numbers. I hope to experiment across various settings and approaches.
Over the course of the project, I hope to experiment with a real network which consists of at least 10 nodes and measure the improvements. One exciting thing about real life experiments is that nodewatcher was mostly used inside wlan slovenija's network and I get to run it independently! This will probably allow me to fix some bugs on the way and contribute to nodewatcher in this aspect as well.

The algorithm will initally be developed as a nodewatcher module, but I hope to eventually port it to openwrt (possibly after the summer ends). The main difficulty is that nodewatcher can act as a central level planner, whereas the openwrt scenario requires negotiation among nodes. So it's harder to convince a node to decrease its TX power to benefit other users. But imagine a network where nodes can communicate and achieve a socially optimal point of spectrum allocation! A glorious future awaits us.

GSoC 2014 Final - OpenWrt: IEEE 802.1ad VLAN support

Hi all!

Because I have worked very hard in the first part of GSoC, the implementation was almost done already for mid term, in the second part I have been mostly testing the code, and taking advantage of it in a lot of setups :)

The GSoC experience have been very formative to me and I would like to repeat it next year either as student or mentor :)

Moreover I'll suggests to apply to GSoC to a lot of friends!

Many thanks to Freifunk to chose my proposal I hope you will take advantage of 802.1ad too :)

Cheers!

GSoC 2014 Updates - OpenWrt: IEEE 802.1ad VLAN support

As promised I have been working hard on the proposal. Thanks to community help I have already almost completed all deliverables and in fact some patches got merged in netifd [10-13] and OpenWrt [15] already and development branch of libre-mesh is already taking advantage of them [14] :)
 
Working on this project gave me more conviction that community collaboration is fundamental, community helped me to understand the problem and what to modify to fix it, moreover someone went beyond that submitting patches too [16-17] !
 
Here it goes a small resume of how I did it:
    - Talk multiple time with OpenWRT [0] folks via retroshare openwrt lobby, they gave me some indication on what I had to do
    - Get netifd code [18]
    - Create a vlandev netifd device
    - Implement routines to add and remove 802.1ad/802.1q vlan in Linux via netlink
    - Talk with OpenWRT folks for quality check and patch merging
    - Code cleaning and testing
    - Patches get merged in netifd and OpenWRT
 
Now who need to use 802.1ad is not unfortunately constrained to private software anymore, because it is now supported in OpenWRT [0] :D
 
More updates [6] soon, and don’t forget the best is yet to come! 
 
Syndicate content