Year: 2017

GSoC 2017 Attended Sysupgrade

Posted on by Paul Spooren

Hello, my name is Paul Spooren and I’ll be working on attended sysupgrades this Google Summer of Code. I’m 24 years old and studying computer science at the university of Leipzig. With this blog post I try to explain my project, the advantages and it’s challenges.

Topic Change from captive portals.

When I applied to GSoC my first application covered the implementation of “Captive Portals” for the LibreMesh. After discussing details with my mentors we decide to switch the project.
The main shortcomings where the following:
* Captive portals need testing on all kind of devices, Apple devices using a different approach than Android, Linux distribution differ, all kinds of Microsoft’s Windows as well. Testing would claim to much effort to provide a stable solution
* Captive portals usually intercept HTTP traffic and changing it content with a redirect to the login provider’s splash page. This does not work with encrypted traffic (https) and would result in certification errors.

Discussing what has generic use to OpenWRT/LEDE and LibreMesh we came up with the topic of a simple sysupgrade solution and fixed on that.

What are attended sysupgrades?

Performing updates on routers is quite different from full Linux distribution. It’s not always sustainable to do release upgrade via a packet manager. Instead it’s usually required to re-flash the system image. Depending on the installed packages an image rebuild may be to complex for regular users. A more convenient way is needed.

The main idea is to provide a simple function within the web interface to automatically download a custom sysupgrade-image with all currently installed packages preinstalled.
An opt-in option would check for new releases and notify via luci(-ng) or command line.

This approach would also help to upgrade a router without full computer hardware. The web interface can be accessed from mobile phones and as no complicated image downloading is required all users can perform sysupgrades on their own.

Distributions like LibreMesh may have a more frequent package release cycle and devices may don’t offer opkg due to limited flash storage. The simple sysupgrade approach could be used as a opkg replacement for these special cases and keep devices up to date.

How does it work?

The web interface will have a new menu entry called “Attended Upgrade”. The page send the currently installed release to the server and checks it response. If an upgrade is available a notification will be shown. A click on the download button sends a request to the server and downloads the image. Another click uses the sysupgrade mechanism and installs the image. After reboot the system should run as excepted with all before installed packages included.

This project will implement an “image as a service” server side which provides custom build images depending on installed packages. A JSON API will enable routers to send requests for custom images. Build images will be stored and reused for other requests with the same package selection and device model.
A simple FIFO queue will manage all builds requests. Created images will be stored by priority queue system so most requested combination are always in cache.

Challenges

* With new releases packages may be renamed. This can be due to a split after growing in size as more and more features are added or if different versions of a tool exists. The update server has to know about all renamed packages and created an image with all needed programs. Therefore a replacement table will be created which can be managed by the community. Merges, splits and new naming convention will be covered. To make updating easy the server will try to handle changed names as automatic as possible. If there exists different possibilities to choose from there will be a menu in the web interface.

* Currently luci is the de facto web interface of LEDE/OpenWRT. Eventually it will be replaced by luci-ng with a modern JavaScript framework. All router sided routing has to be easily portable to the new web interface.

Implementation details

The main logic will happen within the browser and so can use secure HTTPS to communicate with the update server. The users browser communicates between router and upgrade server. The following diagram tries to illustrate the idea.

Once opened the upgrade view will ask the router via an rpcdcall to receive the installed release and send the version to the update server as an *update availability request*. The server will answer with an *update availability response* containing information about the update if exists or a simple status 204 (No Content) code. If a new release exists the web interface will perform another rpcd request to get details of the device, installed packages versions and flash storage. The information are then combined and send as an JSON request to the update server as an *image request*.

The update availability request should looks like this:

{
    "distro": "LEDE",
    "target": "ar71xx"
    "subtarget": "generic"
    "version": "17.01.0",
    "packages": {
       "opkg": "2017-05-03-outdated"
       ...
    }
}

The update server will check the request and answer with an *update availability response*:

{
    "version": "17.01.1",
    "packages": {
        "opkg": "2017-05-04-new",
        "ppp-mod-pppoe2": "2.0"
    }
    "replacements": {
       "ppp-mod-pppeo": "ppp-mod-pppoe2"
    }
}

The response contains the new release version and packages that will be updated. Not that even if there is no new release, packages could be updated via a sysupgrade. The idea is that packages without opkg installed can receive package updates as well.

All changes will be shown within the web interface to let the user know what will change. If the user accepts the upgrade an request will be send to the server. The image requests would look something like this:

{
    "distro": "LEDE",
    "version": "17.01.0",
    "revision": "48d71ab502",
    "target": "ar71xx",
    "subtarget": "generic",
    "machine": "TP-LINK CPE510/520",
    "packages": [
        "ppp-mod-pppoe2": "2.0",
        "kmod-ipt-nat": "4.9.20-1",
        ...
     ]
}

Once the update server received the request it will check if the image was created before. If so it will deliver the update image straight away. If the request (meaning device and package combination) was done for the first time a couple of checks will be done if the image can be created. If all checks pass the wrapper around the LEDE ImageBuilder will be queued and a build status API is polled by the web interface. Once created a download link is provided.

In the unlikely event of an unsolvable package problem the replacement table can’t fix itself the user will be asked to choose from a list. The new combination of packages will be send to the server as a new request resulting in an sysupgrade image. This approach still needs some evaluation if utilizable and really needed.

Using the ImageBuilder offers an generic way to offer sysupgrades for different distribution. The image builder feeds can be extended to include distribution specific packages like LibreMesh package feed

The replacement table could be implemented as followed:

# ./lede/replacements/17.01.1
{
   "libmicrohttpd": [
   "libmicrohttpd-no-ssl": [
       "default": true
    ],
    "libmicrohttpd": []
    },
    "openvpn": [
        "openvpn-openssl" [
            "default": true
        ],
        "openvpn-mbedtls": [
            "installed" [
                "polarssl",
                "mbedtls"
            ]
         ],
         "openvpn-nossl": []
    ],
    "polarssl": [
        "mbedtls": [
            "default": true
        ]
     ]
 }

libmicrohttpd was replaced by libmicrohttpd-no-ssl (installed as default) and libmicrohttpd.
openvpn splittet into various packages depending on the installed crypto library, openvpn-openssl is the default while openvpn-mbedtls is only installed if mbedtls (or it’s prior name polarssl) was installed before.

For better readability the yaml format could be preferred.

LibreMesh introduced a simple way to manage community specific configurations. This configuration method is flexible for other communities as well and should be integrated into the update server. A optional parameter could contain the profile name which will be auto integrated into new images.

"community": "quintanalibre.org.ar/comun/",

The parameter could also contain a full domain with leads to the needed files, this feature need more evaluation.

Possible features

* The current design is an attended upgrade triggered by and dependent on the web interface. A feature could be to add logic to the command line as well.

* Once the sysupgrade is possible via shell, an unattended sysupgrade would be possible. A testing and a release channel could enable unattended upgrades for tested images (device specific) only. If an image works after an attended upgrade it could be tagged and offered via the release channel.

* Mesh protocols may change and outdated routers loose connectivity. A possible solution to upgrade the devices losing contact could be to automatically login the outdated routers to updated routers open access points, perform an update and reconnect to the mesh.

Final Thoughts?

All thoughts above are not final and are more likely an RFC. I’m very happy to receive comments and critic. My goal is to have an generic update service where all communities and LEDE/OpenWRT itself can benefit from.
Feel free to contact me at paul [a-t) spooren (do-t] de or on freenode/matrix as aparcar.

GSoC 2017-netjsongraph.js: visualization of NetJSON data

Posted on by GeekPlux

Project intro

NetJSON is a data format based on JSON(What is NetJSON?), and netjsongraph.js(GitHub) is a visualization library for it. This library has attracted quite some interest from around the world, but there are some defects, such as tests and modern build process lacking.

Therefore our goal is to improve the features and development workflow of netjsongraph.js. To be specific:

  • make it faster with large numbers
  • make it more mobile friendly
  • use modern tools that are familiar to JS developers, so they can contribute more easily
  • add automated tests so we can be more confident of introducing changes
  • get rid of complex features
  • make it easy to extend, so users can experiment and build their own derivatives
  • make it easy to redraw/update the graph as new data comes in, at least at the library level we should support this
  • geographic visualization (like https://ninux.nodeshot.org/ (nodeshot project)

Arrangement

About me

I’m a graduate student from China and also a front-end developer with more than one-year working experience. And now I am interested in the Data Visualization and already made several visualization projects of network structure. So lucky my proposal selected by Freifunk in Google Summer of Code 2017. It’s a great opportunity to participate in a promising open source project. Thanks for my mentor’s guidance and hope I can finish an excellent job. So I listed the following plan:

Tasks and Schedule

  • create a new branch: build the project with yarn, Webpack and Babel. 1 week
  • To build a (mostly) backward compatible version 1 week
  • draw a demo graph using canvas or WebGL. 2 week
  • make a example page to show visualization results. 1 week
  • add test(using Ava and XO) and CI. 1 week
  • discuss and design visualization view 1 week
  • import and integrate with OpenStreeMap or Mapbox to make a map. 1 week
  • visualization implemention. 8 weeks
  • beautify the visualization. 1 weeks
  • improve visualization and test. 4 weeks
  • design interface for plugin (to make this library extensible) *2 week
​

GSoC 2017 – wlan slovenija – HMAC signing of Nodewatcher data and IPv6 support for Tunneldigger

Posted on by red_moo

Howdy!

I am a student at Faculty of Computer and Information Science in Ljubljana, Slovenia. Like (almost) every “computer enthusiast” I liked gaming and later found myself developing an OpenGL graphics engine. All engrossed in C++ and all sorts of algorithmic challenges I slowly came to realize that something is missing. Yes, my knowledge of anything network related. So, combining my two other interests, those being information security and inexplicable love of tunnels, I applied myself to Google Summer Of Code with the following ideas. As a participant in this year’s Google Summer of Code I will develop some new goodies for two projects of wlan slovenija open wireless network.

The first one is for the nodewatcher, which is an open source system for planning, deployment and monitoring of the wireless network. It is a centralized web interface which is also used for generating on OpenWrt based firmware images for specific nodes. After flashing the wireless router with the generated image, it just needs to be fed some electricity and it automatically connects into the network using VPN, or in case of an existing nearby node wirelessly. Nodwatcher then collects all the data about node’s performance by connecting to nodes to obtain data, or by nodes pushing their data to nodewatcher. This data is not sensitive, but we can still worry about it being manipulated or faked while in transit between the node and nodewatcher. The problem though is that all the monitoring reports are currently unsigned. This poses a security risk in the form of a spoofing attack, where anyone could falsify the messages sent to the nodewatcher. The solution is to assign a unique nodewatcher signing key to every node. The node will then sign the monitoring output using a hash function in HMAC (Hash-based message authentication code) mode. This means that a computed “signature” would be sent along with every message and nodewatcher can check whether the data was altered in any way. In the event of a signature verification failure a warning will be generated within the nodewatcher monitoring system. This is imporatant, because it assures the integrity of recieved data and inspires confidence in using it to plan deployment of new nodes in the future.

The second contribution will be to the Tunneldigger, which is a simple VPN tunneling solution based on L2TPv3 tunnels. It is used to connect nodes which do not have a wireless link between them in to a common network. Using existing network connectivity it creates L2TP tunnels between nodes. The current limitation is that tunnels can only be established over IPv4. This poses a problem because due to dramatic growth of the internet, the depletion of the pool of unallocated IPv4 addresses is anticipated for some time now. The solution is the use of its successor, IPv6. Since the tunnels are already capable of carrying IPv6 traffic, the capability of establishing them over IPv6 will be developed. The Tunneldigger will also support IPv4/IPv6 mixed environment where both server and client have some form of IPv6 connectivity. That way the Tunneldigger will finally be made future proof!

Reports about my work will be available on developers mailing list.

Yay for the free internet!

GSoC 2017 – LuCI2 on LibreMesh

Posted on by Marcos Gutierrez

My name is Marcos Gutierrez, I am from Argentina and this year I participate in the GSoC 2017 in Freifunk. My main task is to incorporate LuCI2 into LibreMesh and to adapt or rewrite the modules that are currently used.

LuCI2 – UI

In my first approach to LuCI2 I realize that there is much more to do than it seemed. The development of Luci2 is still looking for a more stable path, there are good ideas, but a resolution, to my understanding, is incomplete. Only the base UI build weighs 1.4MB, this far exceeds what LibreMesh requires. So I should explore some alternatives to drastically reduce the size.

Continue reading “GSoC 2017 – LuCI2 on LibreMesh” →

GSoC 2017 – Add MPTCP support in LEDE/OpenWRT trunk

Posted on by Fejes Ferenc

Brief intro

My name is Ferenc Fejes. I’m MSc CE student at the University of Debrecen, infocommunication networks specialization. I glad to introduce my project which is Add MPTCP support in LEDE/OpenWRT. The main idea is from my mentor, Benjamin Henrion. If we have a Wi-Fi mesh network, there are probably more than one network path between the nodes. The point is, we can gain additional throughput if we use those path simultaneously.

The goal

Currently no MPTCP support in GNU/Linux nor Windows systems. So the key is, the user using regular TCP/UDP at his system until the first hop, which is a LEDE box with MPTCP support. At this point, this router operating as an intercepting proxy: read the incoming traffic from the host, and distribute that on multiple interface, using MPTCP. At receiver side, we can do the same but in reverse order: get the data from the MPTCP connection and send is to the sink host in regular TCP (or UDP). With this operation, the whole MPTCP transmission remains hidden for the end hosts, but they will enjoy the gain from it – because MPTCP has a good capability to aggregate the bandwitdh of the different paths. Of course, if the bottleneck between the end host and the first hop, the user did not get any gain from this operation.

The plan

The key steps for reach the goal of the project:
1. Set up virtual/real machines with MPTCP support. This is not a problem since kernel patches for MPTCP support available.
2. Make a simple multipath topology: end hosts with no MPTCP support, two machine with MPTCP support, functioning as a router and has two different network path between them
3. Search and configure an efficient and transparent proxy software on them, which can suitable for intercept TCP traffic and work as a tunnel for other protocols (like UDP, SCTP, etc.)
4. Reproduce the working configuration on LEDE supported devices, with two Wi-Fi bridge links between the LEDE boxes.
5. Verify the aggregation with iperf3 measurements.

geolocator (Software defined GPS)

Posted on by buttjantarek@googlemail.com
Hi everyone,
 
My name is Jan-Tarek Butt. I am in my fourth term of computer science at Emden University in Lower Saxony (Germany). Some of you may know me from the last Google Summer of Code. I am one of the students who is participating for Freifunk in the Google Summer of Code 2017. In the following post I would like to introduce you to my Google Summer of Code project. I split up the project, same like last year, into 3 main subjects: web backend, gps-share, LEDE Package,

web backend

The first one is a restful API backend[0], which should include a backwards compatibly to the api format of the old openwifi web backend[1]. The new backend should be written in Go, which will be clearer in the next few days. The New API should use Mozilla Location Service as fallback if it is not possible to determinate a position based on the own db entries. Received WiFi information can also redirect to Mozilla Location Service[2]. So some of you might have the question what is Mozilla Location Service. It is simply a service with a big database full of wardriving informations from mobilephones. Mobilephones scan for surrounded wifi networks and send this with its own position to the Mozilla Location Service. These datas will be integrated into the database. If you want to know your position without having GPS, just send only your surrunded wifis to the Mozilla Location Service and you will receive your position based on the wifi informations see the picture below.

gps-share

The first Idea was to create a software defined GPS as udev device which should provide NEMA-famata protocols over tty and receive the necessary informations over wifi. While discussing with people on the Mozilla Location Service Malinglist, it turns out that something similiar is already done in geoclue which provides geo coordiantes over dbus and can receive coordinates via Mozilla Location Service. geoclue works with network-NEMA and Mozilla Location Service. Over network-NEMA geoclue can receive GPS informations from Mobilephones and GPS modems working over local networks. To avoid of developing redundant software, i discussed with Zeeshan, the maintainer of geoclue, about building support in gps-share for standalone GPS instead [3]. This is a software which should provide GPS NEMA-Format from GPS hardware over network-NEMA for geoclue. gps-share is written in Rust for ensuring reliability.

geolocator

The third subproject is a new lede package called geolocator, which should provided geo positions received by their surrounded WIFIs. The WiFi informations should be sent to the new above mentioned backend[4].
cheers
Jan-Tarek Butt

GSOC 2017 – Implement NetJSON output in ubus (OpenWRT/LEDE)

Posted on by Arun

About Me:

I am Arunkumar Ravichandran, about to pursue my MS in Electrical and Computer Engineering at University of California, San Diego. I have almost 2 years of Software Engineering experience at EmbedUR Systems in the Wireless networks domain. I was working on wireless device drivers implementing Network assisted roaming and SDN support for the Linux based Enterprise Access Points and Wireless LAN Controllers.

What I am going to do in GSOC:

The main aim of the project is to bring in support for NETJSON object in the LEDE/OpenWRT Linux distributions. The support for NETJSON is brought in at the interconnect system – ubus. In this project, we need to add support for a new ubus API which allows retrieving these two NetJSON object types: DeviceConfiguration and DeviceMonitoring.

What is NetJSON ?

NetJSON is a data interchange format based on JSON designed to ease the development of software tools for computer networks. This specification defines several types of JSON objects and the manner in which they are combined to represent a network: configuration of devices, monitoring data, network topology and routing information.

Why NetJSON is important to OpenWRT/LEDE ?

Network infrastructure is deeply closed source. Open source won the battle in web development and system administration, but not in network management. The status-quo in networking is closed source software, poor interoperability and poor documentation.
NetJSON aim to challenge the status-quo and improve the openness of internet infrastructure by offering a set of tools (like a framework) which communities and companies can use to build network infrastructure using mainly open source software products.

– Federico Capoano, OpenWisp

  1. NetJSON would allow standardization similar to NETCONF. Since NetJSON uses JSON format, it makes the management of configurations done at a higher scale to be automated easily.
  2. By using NetJSON objects to either produce or collect information, in different vendor’s different hardware, it allows the developers to work on their ideas faster and in a better way.

Project goals:

  • Bring CLI support to accommodate the new parameters: DeviceConfiguration and DeviceMonitoring.
  • Write backend ubus APIs to collect the data and store the data in NetJSON objects: DeviceConfiguration and DeviceMonitoring and display them.
  • Revisit the implementation of JSON plugin in SCAL and try to extend the support for NetJSON. For the missing data models in the JSON plugin, try to write a data parser which uses the scal APIs to get data and convert that into NetJSON format and store in the DeviceConfiguration NetJSON object for outputting it.
  • Adapt the code in node-watcher agent and reuse it in SCAL as a plugin. This plugin should output the monitoring data in a NetJSON DeviceMonitoring format..
  • Writing makefile to compile the files added for supporting NetJSON in the ubus IPC/RPC system in OpenWRT/LEDE distribution.
  • Document the work done, installation guide and usage examples of the new CLIs.

Project Details:

  • CLI support: The CLI should be able to input the required data and display error message if entered wrongly. Generic CLI: ubus call netjson <config | monitor> <general | network | interface | none_to_display_all> <parameter | none_to_display_all> Example CLI: ubus call netjson monitor interface eth0 .The above CLI would display all the statistics related to eth0 in the DeviceMonitoring NetJSON format. CLI depth should be increased for certain commands like the interface and network entries to get the interface name. The cLI backend tool would be a separate API which calls the SCAL plugins.
  • ubus call netjson API support: We should register the namespace netjson in the libubus. This allows the procedures with any number of arguments.
  • NetJSON objects: SCAL is an abstraction layer to push information onto the ubus from the device.  SCAL allows gathering data on the device from UCI, ubus calls, shell commands, etc and publishes it on the ubus using a different datamodel. Already available SCAL JSON plugin takes these data models and convert it into a JSON format. My work involves extension of this plugin to support the NetJSON to be done. Add features to the JSON plugin in C which is needed in order to define the full data model using the NetJSON schema. Also for data formats which doesn’t have support in the SCAL JSON plugin, parse the scal datamodel and then store in a DeviceConfiguration NetJSON object. This is SCAL plugin is created for retrieving the information directly using C.

Timeline:

  • May 30th : GSOC coding officially begins.
  • Task to be completed by the 2nd June: Bring CLI support for the above things to be implemented. Test the CLI and check if error cases are handled by the CLI.
  •  5th June to 8th June: NetJSON DeviceMonitoring and DeviceConfiguration objects/schema structures to be defined in the ubus architecture. ubus call netjson API support is to be implemented in this time period.
  • June 12th till June 16th : Implement the NetJSON DeviceConfiguration to be retrieved for the whole device. The whole NetJSON DeviceConfiguration would be the output. SCAL plugin to be added to support the NetJSON format in C itself.
  • June 19th to 23rd : Making changes to the exisiting JSON plugin available in SCAL to transalate it into the NetJSON format.
  • June 26th : Demo to the OpenWRT/LEDE community apart from the mentors to get feedback. Based on the feedback improve the implementation.
  • June 26th to July 7th : Based on the inputs provided in the CLI, output the DeviceMonitoring NetJSON object. For eg: “ubus call netjson config interface wlan0 status
  • July 8th to July 20th : Implement the NetJSON DeviceMonitoring to be retrieved for the device in whole and display it.
  • July 20th : Demo to the OpenWRT/LEDE community apart from the mentors and get feedback.
  • July 21st to July 27th : Improve the implementation based on the feedback from the community.
  • July 28th : Phase 2 evaluation.
  • July 28th to August 10th : Based on the inputs provided in the CLI, output the NetJSON DeviceMonitoring object. For eg: This CLI would only output monitoring data for wlan0. “ubus call netjson monitor interface wlan0”
  • August 10th to August 21st : Improve upon any possible design based on the Mentor and community feedback.
  • August 21st to August 24th : Code cleanup, final documentation review and possible merge/commit to the LEDE/OpenWRT distribution or ubus.

GSoC 2017 Project – OpenWifi: LEDE/OpenWRT configuration Mangement

Posted on by Johannes Wegener

GSoC 2017 – OpenWifi

Hi, my name is Johannes Wegener and I’ll be working on OpenWifi this Google Summer of Code. I’m 27 years old and study computer engineering at TU Berlin. In this blog post I’m going to explain to you what OpenWifi is and what should be done during this summer of code.

What is OpenWifi?

OpenWifi is a OpenWRT/LEDE configuration management system. It is intended to manage a bigger or smaller amount of OpenWRT/LEDE devices.

If a new access point joins a network it is be able to auto detect the management Server and register to it. After the node has been registered its configuration (static configuration like what you’ll typically find in /etc/config/ on OpenWRT/LEDE devices) will be downloaded by the server and stored in a database.

It is now possible to query and modify aspects of the configuration. It also possible to change values depending on other values. This means configuration changes can be applied to a lot of different routers. There is an older templating system which shall be replaced by a new graph-oriented one. It also manages SSH-Keys, has a rudimentary file upload functionality (for uploading new images for example) and extensible API. It is possible for example for a node to ask for an image it should install. (Currently this is not very dynamic – but it is intended that an image could be selected on various parameters)

The Server also regularly pulls the health-state of the node and displays it in an overview. Furthermore the server acts as a luci2 proxy.

It is also extensible via PlugIns and is able to serve as an entry point to other services like icinga or location services. (Both already have a proof of concept PlugIn available)

How does it work? aka what makes it tick?

The main software is written in python and uses the pyramid framework and sqlalchemy as ORM. It uses pyramid-rpc for json-rpc requests (which is used for nodes to register for example) and cornice for a REST-style API (which is intended for the user of the system). The Core-System can be found in this repository.

Most of the tasks operating on nodes are done by a jobserver that uses celery – so that they don’t block the main thread.

All Webviews have been moved to a different repository and are realised in fact as a PlugIn. (If you just want to manage your nodes on CLI/with scripts this will be possible in the near future!)

The nodes use a notification script written as a shell script. It uses a fixed DNS entry (openwifi), a configuration file (/etc/config/openwifi) or mdns (using umdns or avahi) to detect a server and register to it. The packages can be found in the openwifi-feed repository. There is also a boot-flasher which is intended for flashing an alix2-style board from ramfs – since it is also possible to execute commands on the node I’ll integrate an update solution that uses sysupgrade.

The communication between the server and the node is realized currently via rpcd. But one goal during this Google Summer of Code is to abstract that and realize it via a rpcd-communication-PlugIn – this would make it possible to also a NetJSON-communication-PlugIn for example.

The PlugIns are realized with python entry points. There are some special named entry points which will extend the main application. You could have a look at the example plugin to get an idea how it works.

How to try it?

You can easily try the software with docker. Navigate to the Docker directory – there three files that will assist you with the setup. In conf.sh you setup if you want to use LDAP for authentication, avahi for mdns announcement and dnsmasq as dhcp server in the docker container.

With build_image.sh you build the docker image. It tries to detect if you need to use sudo for docker or not. Last but not least you need run_image.sh to start the image.

You can easily add PlugIns to the docker image if you just check them out or copy them inside the Plugins directory. To start off I would recommend to add the Web-Views Plugin and use avahi. Now you can navigate your browser to http://localhost:6543 to see the webviews (you need to restart the container after Plugin install – use docker stop OpenWifi and docker start OpenWifi).

What is needs to be done?

In the next section I’ll explain what should be done during Google Summer of Code. I need to prioritize these things because I’m not sure if it is possible to do everything. I think most important things are Testing, Authentication and the new graph-based database model.

Testing-Infrastructure

Currently just very few things have tests. But because it is possible to start a LEDE container in docker (there are some scripts helping to create a LEDE container in the Docker/LEDEImage directory) a lot of things are possible to test with docker orchestration. I want to provide a small LEDE image with the code to test it.

Before adding big new things I like to transfer the development to true test-driven development and having test for at least 90% of the core component code. And after that continuing development by writing tests first.

For that I would also have a look at coverage.py.

Security and authentication

There is a simplistic authorization for OpenWifi with a LDAP backend. I would like to add authorization based on user/password (without LDAP), API-Key and client side certificate (for the nodes to authenticate theirself against the API).

I would like to have the authorization as granular as possible. Authorization should check whether the authorized identity is able to operate on this node and what kind of action is allowed.

Security is provided by TLS. Last week I added the last bits to also use https for client communication – but right now it needs to be setup up by the user. I would like to add the appropriate hooks to the notification shell script.

Expand new graph-based DB-Model

The new DB-Configuration model is based on a graph – where configurations can be connected by links (the link can contain addition information – like what kind of link it is – currently the addition data is the option name (for example if you have a wifi-iface configuration the “device” links it to a wifi-device)).

The model is implemented and there is a query API to get and set options. But for communication purposes the configuration is stored as uci-json-config. There is code to convert between these two (but the one from uci to graph-model needs quite some more intelligence to detect everything right). There are also some hooks that update each part. But the update process is not consistent – the graph-based config gets a new ID for example. It would be nice to have a consistent conversion.

Furthermore I like nodes to share parts of a configuration. This should be easy to implement with some small DB changes. For this consistency is also very important. By default UCI generates unique strings for every configuration – this could be used for consistency.

The DB-Model also currently lacks creating new configurations and reordering configurations.

API

The API to modify nodes should get expanded and everything should get documented. (see above) This should be done in code and a document generated by sphinx.

I also like to implement a CLI program to use the API.

Versioning

The uci-json parser supports diffs between configurations. I would like to save a diff for each change which is applied – to have roll back configurations or delete a specific change.

There is a simplistic revisioning implemented. But the diff should be updated to a class of its own with json import and export and operations to apply a diff to a UCI configuration. (upgrade and downgrade)

Modularity and expandability

OpenWifi is right now is already quite modular – but I also want to modularize configuration parsing and communication. It would also be nice to have some interoperability with OpenWISP.

I use alembic for tracking database changes it would be nice to find a good way to use alembic for database changes needed by PlugIns as well.

Scheduled Updates

I want to schedule config updates – for example if you have a mesh you probably want to update the deepest nested nodes first and than the ones above etc. – this should be easy with celery (as long as the mesh is known and somehow represented in the database).

PowQuty Live Log introduction

Posted on by Stefan Venz

Hi everyone,

I’m Stefan and happy to present my GSoC 2017 project to you.
That is extending the PowQuty daemon and luci web-app with an event log and notification system for events violating the EN-50160 norm.
Probably most of you don’t know powquty. Therefore i will give a short introduction to it, before I present my actual project.
 

PowQuty

PowQuty[1] is an application for measuring power quality parameters of the power grid like voltage and frequency. It receives samples by a USB connected oscilloscope and writes the calculated values to a log file. This log can be read by the powquty luci web-app. The web interface will plot the measurements directly into several graphs. These are power over time, frequency over time and metrics over time.
It was implemented during the last GSoC by Neez El Sayed [2] and its intention is “to strengthen the role of open software in the uprising smart grids by providing some essential functionalities”.
 

PowQuty Live-log

The goal of this project is to provide an event log of measurements that are outside defined parameters.
Public power distribution networks have to meet certain criteria. These are formulated in the EN50160 norm.
If there is a power drop, or a frequency spike PowQuty will write this to its log like every other value. The user can examine the log file or its graphical representation for critical values, but essential equipment could already be damaged.
If such an event occurred, a user or power network administrator should be notified. This event log will get a graphical interpretation in the luci powquty app as well to enable a better tracking of events.
This will allow users of PowQuty to react quickly to changing conditions in power supply networks.

Therefore we will add a notification system to PowQuty as well as additional logging options.
To collect events at a central server we will extend the mqtt usage to send an mqtt notification on a violating event.
In addition we will add an email and slack notification option to allow quick responses.

 

[1]: https://github.com/thuehn/PowQuty
[2]: https://blog.freifunk.net/author/Neez-El-Sayed/

Our GSoC 2017 projects

Posted on by Andreas Bräu

In 2017 we got 15 slots for our projects. That’s great, we’ve never got so many slots for GSoC before. Our students are from 8 countries. They do their work for 6 sub organizations.

Until the end of May the Community Bonding Period takes place, so students can introduce their projects to the community. They also prepare together with their mentors the folling coding period.

This post will give a short overview on the selected projects. In the next weeks some detailed blog posts will follow describing all the projects.

GSoC project overview

Title Org Student Abstract
RetroShare Improvements retroshare ange The aim of this proposal is to improve RetroShare incrementally during the summer in the following work lines:
– Semi-automatic RetroShare friendship suggestion based on phone contacts
– Semi-automatize JSON API code and documentation generation
– Multiple simultaneous heterogeneous network connection for each peer
– Improve RetroShare Android usability and performances
– Seamless key exchange via Quiet Modem
– Chat and messages multiple device synchronization
– Directories multiple device synchronization
Implementing Pop-Routing in OSPF ninux Gabriele Gemmi Prince is a network daemon that continuosly monitors the network topology and sets the timers for OLSRv2, it is developed in C and we are currently deploying it in the Ninux community network in Florence. This year
we want to implement Pop-Routing for another link-state routing protocol: OSPF. OSPF is the state-of-the-art interior routing protocol for wired networks and it is used also in some wireless community networks. This
project consists in realizing a plug-in of the OSPF open source routing daemon (Quagga or Bird) that will:
– Expose the network topology to Prince using NetJson
– Receive and re-set the network parameters from Prince
Improving nodewatcher data representation capability wlan slovenija rasovica Currently nodewatcher has very limited overview of used IP space without more precise division of existing and used subnets. This could be improved using compact and colorful matrix with IP ranges and links to nodes. I would display data about used ips similar to the [IP Census 2014](http://census2012.sourceforge.net/images/hilbert_icmp_map_lowquality.jpg), better tables and other data representation that would help explain the network structure, it would help in explaining the network and its statistics even to an uninformed observer. I would also update the look and functionality of other graphs and tables representing statistical data from nodewatcher
Extending LoxiGen and ONOS to enable SDN control of wireless switches via OpenFlow lede Arne Kappen LoxiGen is a tool by the Floodlight project to generate OpenFlow libraries for several programming languages including Java and C. The goal of this project is to extend LoxiGen to include messages for controlling wireless switches using OpenFlow.
It will include the necessary extensions to LoxiGen as well as a proof-of-concept implementation of the control channel. This encompasses an application for the ONOS SDN controller and an agent for the LEDE router operating system.SDN-enabled wireless switches offer comfortable centralized management of larger deployments and may provide the basis for more complex use-cases in the future such as network slicing and seamless mobility.
OpenWRT/LEDE Configuration Management lede Johannes Wegener Improve the proof-of-conecpt version of a OpenWRT/LEDE configuration software.
Web interface for Retroshare, a secure communication software retroshare Stefan Cokovski I am applying to work on the web interface for RetroShare, secure communication software. RetroShare already has a web interface with limited functionality and questionable look, compared to the main Qt GUI which RetroShare uses. My goal will be to improve the already present functionalities, introduce new functionalities to the web interface and design a new look for the web interface.
Implement NetJSON output in ubus (OpenWRT/LEDE) lede Arunkumar Ravichandran To bring in support for NETJSON object in the LEDE/OpenWRT Linux distributions. The support for NETJSON is brought in at the interconnect system- ubus. To add support for a new ubus API which allows retrieving these two NetJSON object types: DeviceConfiguration and DeviceMonitoring.
DeviceConfiguration NetJSON object is filled in using the plugins available in System Configuration Abstraction Layer(SCAL).
DeviceMonitoring NetJSON is retrieved by reusing the code parts and adding hooks in the nodewatcher-agent.
LibreMesh Attended Sysupgrade libremesh paul0 Performing updates on routers is quite different from full Linux distribution. It’s not sustainable to do release upgrade via a packet manager. Instead it’s usually required to re-flash the system image. Depending on the installed packages an image rebuild may be to complex for regular users. A more convenient way is needed.

This project will implement an “image as a service” server side which provides custom build images depending on installed packages. A notification in the web interface will notify about the new release. After image creation a one-click installation is offered within the web interface.

The server side implementation will use established tools like LEDE’s ImageBuilder to provide an generic approach for image creating. In this way the entire OpenWrt/LEDE community including several community-mesh firmware projects will benefit from that new update routine.
Spectrum Analyzer libremesh Nicolas Andres Pace I believe that Community Networks’ Operating Systems could use the Spectrum Analysis a lot to make better decisions on how they use the electromagnetic field wisely and efficiently.

As of now, no Free OS has been using Spectrum Analysis, so they can’t know whether the radios are being used efficiently or not.

This module would allow all routers that use Atheros radios (the main radio manufaturer around) to see how is the spectrum and take better decisions on which channels they use.

Deliverables
* Create an OpenWRT/LEDE package to get Spectrum info from the radio.
* Create a LuCI interface to visualize the Spectrum information retrieved.
HMAC signing of Nodewatcher data & IPv6 support for Tunneldigger wlan slovenija red_moo Currently all monitoring reports by WLAN nodes are unsigned and can be spoofed by anyone. This represents a security problem and a possible solution is to add HMAC-based signing to encrypt node monitoring reports to prevent spoofing.
Nodewatcher’s monitoring system will also be improved so it will generate an event and a warning in case of a signature verification failure.The aim of the second part of this GSoC project is to design and implement IPv6 support for Tunneldigger so that it works in IPv6-only and IPv4/IPV6 mixed environment where both server and client have IPv6 connectivity in some form.
Add MPTCP support in LEDE/OpenWRT trunk lede SPYFF Create an MPTCP supported OpenWRT/LEDE trunk and ensure its operation in a multipath Wi-Fi aggregation environment. The trunk should contain the tooling for establish a SOCKS proxy for operating systems without MPTCP support.
geolocator (Software defined GPS) FF Nordwest Jan-Tarek Butt A dynamical and flexible geolocator which should provide a software defined GPS device and receives its position over WIFI hardware with various services like openwifi.su, MLS and etc.
netjsongraph.js: visualization of netjson data ninux GeekPlux NetJSON is a great work attracted some interest from around the world, but there are a lot of defects. And I’m a front-end developer and now focus on Data Visualization, so I want to contribute to NetJSON. In my proposal, I come up with my idea and list some tasks and schedules.
lime-webui: port to LuCI2 libremesh Marcos Gutierrez Make an inventory of all LuCI components that LibreMesh uses and that will not be compatible with LuCI2. Analyze which dependencies must be replaced or rewritten and generate the views according to the new framework selected.
Powquty Live-Log lede Stefan Venz PowQuty provides statistics from measurements taken by a USB oscilloscope. The statistics provide information about the power distribution network, but don’t highlight the most important information. Power networks need to comply with EN50160, if this norm is violated, damage to connected devices is possible.
To avoid damage, powqutyd will log violations and notify users on occurrence.