Tag: gsoc

GSoC 2017 – Attended Sysupgrade – Final evaluation update

Posted on by Paul Spooren

Hi Freifunk,

This is my last post within this years GSoC. I’ll cover the progress, challenges and future of the project.

tl;dr:
Direct links to the work I’ve done this GSoC:

First a small reminder what the project is about: Enable end-users to update their routers to new releases or bulk update installed packages via a simple click in the Luci web interface. The magic lies in a server producing sysupgrade images with the same packages preinstalled as installed on the router. That happens on demand when triggered via the web interface.The image is downloaded securely via encrypted HTTP as the used browser has all certificates installed. The router does neither need certificates nor a correct running clock.

Progress

Server

At this point I’m very happy to announce a test (and usable) version of the attended sysupgrade setup. All created package recipes were accepted into the official repository and are compiled within the daily build cycle. The server runs fine obeying the described API at the Github page. Images are build within a few seconds if the request appears for the first time. The server stores previous requests and forwards to existing images instead of building it again. This reduces significantly the amount of build images due to the likely case of identical images being requested again. In addition some basic information are offered via the web servers status pages.

The server is implemented in Python3 using flask for request routing and template rendering. While that’s only the tip of the iceberg behind the server lays a rather complex PostgreSQL database validating requests, checking for package changes or transforming packages when updating to another main release. More on the transformations later.

Images created on the server. A click on manifest shows all installed packages with version. All snapshot builds are deleted midnight UTC.

To try the the server at https://betaupdate.libremesh.org have a look at the demos I prepared. Continue reading “GSoC 2017 – Attended Sysupgrade – Final evaluation update” →

GSoC: Improving nodewatcher data representation capability (update 3)

Posted on by rasovica

I have been working on improving the wlan-slo main page. This page has not been updated in a long time and it is crucial at introducing new people to mesh networking.

Continue reading “GSoC: Improving nodewatcher data representation capability (update 3)” →

GSoC 2017-netjsongraph.js milestone 2

Posted on by GeekPlux

After a month of efforts, netjsongraph.js has been greatly improved.
First, I added some API and options are compatible with previous versions, except those that can not be added to the WebGL element. Followed by updating the documentation and some examples so that users can better understand and try out our projects. Finally, I have optimized interaction and performance.

netjsongraph dark theme
netjsongraph dark theme

Interaction

  • added highlight effect on nodes and links when hovered.
  • added tooltips with node information
  • added pan and zoom interaction
  • added node and link information panel when clicked
  • adjust the size of the canvas when the window resized

Performance

The large part of the performance of force-directed layout is consumed in each element position calculation in every tick. So if the position calculation of the elements is before canvas rendering, it is effective to reduce the performance loss.
So I made a static layout https://github.com/netjson/netjsongraph.js/issues/44, there is no animation when rendering, directly render a static force-directed layout in the page.

Other

Besides, I have refactored the event controller, in order to make zoom and pan or other events added easily.
But now the overall rendering results are not good as our expected. There is a little lag in animation, I currently have no idea about the reason, perhaps because this is not best practice of three.js. In the next stage, I will focus on solving performance problems and improving the visual effect.

Next Plan

The links below may be useful:

  • http://taligarsiel.com/Projects/howbrowserswork1.htm
  • http://igoro.com/archive/what-really-happens-when-you-navigate-to-a-url/
  • https://github.com/alex/what-happens-when
  • https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame
​

geolocator-software-defined-gps-second-evaluation

Posted on by buttjantarek@googlemail.com
Hi everyone,

 

I am now taking you over my second evaluation. A German version of this blog post can be found here [0]. For people who did not read over the geolocator (Software defined GPS) project before, here is a link to the project introducing (english)[1] and (german)[2]. The first evaluation from last month can be found here (english)[3] and (german)[4]. First of all I would like to tell you about the web backend:

 

Web Backend

As I said it in the blogpost before, the new web backend should replace the old one.
The current openwifi.su web backend consists of a collection of different programs in different programlanguages. The new one is written in Golang it has a fully backward compatibility. The new web backend is more secure.  It checks all requested strings at the end of the url that they contain only comma separated macaddresses with 12 hex characters, as a validation check.

 

Also the new web backend fix an issue by calculating the requested position. The method before just adds all latitude values and devides them over the amount of positions. Same for longitude. The new way is a geographic midpoint calculation. First the lat/lon values will be converted into radius and then the radius will be converted into a Cartesian coordinate system, after that the midpoint calculation will be done and will gave us a lat/lon in the form of radius and the values will be converted back into degrees, that is all.

 

The web backend has a new feature which should help to enhance the position quality. If there are BSSIDs unknown in the database of openwifi.su, the new backend sends an extra request for the unknown BSSIDs to Moziller location service [5] and brings that information into the calculation of the response.

 

At the moment the new web backend is being tested. In the next few days a first version of the new web backend will be released. The current code can be found here [6].

 

LEDE Package

The last project is to build packages for LEDE and send merge requests to Gluon. The idea behind that is, for WiFi routers and other WiFi devices to be able to automatically receive their own position. This could be a nice feature for mobile Freifunk Routers. It could also provide a good alternative instead of setting positions by hand. A first version of that package is here [7].

 

For the integration in Gluon I plan 4 new packages. The first one is the gluon-geolocator, which depends on gluon-node-info and provides the programme with the capability of receiving geopositions based on surrounding WiFis.

 

The next 3 packages are just for the Gluon configmode. The configmode is a state that begins at the initial start of a Gluon router and the owner of that router can configure it over a LAN connection.
I would like to integrate the following 3 packages within the gluon-config-mode-geo-location.
They are named:
gluon-config-mode-geo-location-with-geloc-map,
gluon-config-mode-geo-location-with-geloc,
gluon-config-mode-geo-location-with-map

 

As their names indicate, the package gluon-config-mode-geo-location-with-geloc-map should integrate the options of the geolocator and an OSM map into the configmode. The other 2 packages, gluon-config-mode-geo-location-with-map and gluon-config-mode-geo-location-with-geloc should either integrate only an OSM map or just the configuration options of the geolocator. Also the package gluon-config-mode-geo-location should be able to and will show the current location interface without any map or geolocator.

 

A merge request of the above new packages will follow soon. Some of the gluon-config-mode-geo-location extensions are currently not finish yet.

 

In the next few weeks I would like to finish the integration into Gluon, as well as continuing to develop the new web backend and releasing the first version of it. There are many ToDo points currently open e.g. Multicore functionality, optionally other geographic midpoint calculations. At the beginning of August there is the SHA2017[8], which I will use for a marathon session.

 

With best wishes
Jan-Tarek Butt

 

GSoC 2017 – wlan slovenija – Report 2 – HMAC signing of Nodewatcher data

Posted on by red_moo

What’s been done

The first blog post that describes the idea and goals can be read here and the first update here.

The data that the nodewatcher receives from nodes had no signature, so anyone could intercept the packets, change the content and send them on their merry way. Authentication using SSL certificates was already implemented, but a lightweight, fast solution, that doesn’t require the use of big libraries like OpenSSL and any extra dependencies on nodes was desired. That means two things. First finding a good sha256 implementation or writing my own. Some libraries implement extra precautions to prevent side-channel attacks, which take advantage of some physical characteristic of the implementation. Even then not all implementations are safe depending on the system on which the algorithms run. It was decided that in this context these attacks don’t bother us too much and the attacker would need physical access to the device anyway. Therefore Brad Conte’s open source implementation was used.

The second part involved writing the hash-based message authentication code. I tried to optimize the algorithm as much as possible saving space and time. After combining both parts the correctness of implemented HMAC-sha256 was thoroughly tested using official test vectors.

The extra option to select the desired authentication was added to nodewatcher-agent, with SSL certificates being the default. If HMAC signing was selected, a HMAC signing key needs to be provided. The signature converted to base64 is then inserted in to the message header and checked when received by the nodewatcher.

What’s next

Now I will start working on IPv6 support for the Tunneldigger.

I am contributing using my github account.

Forward! Good luck!

PowQuty Live Log Second Update

Posted on by Stefan Venz

It’s been nearly a month since my last update on the PowQuty Live Log project and i would like to tell you, what has been done
so far and provide information on what will be done in the next month.

PowQuty got updated, to support Slack and mqtt event notification and can already be used in the current PowQuty version.
In addition to this, there have been some bug fixes during the last month and some new features were added.
On event occurrence the event gets stored in a csv file and each entry is displayed in the luci-app. To increase the usability,
a traffic light system will be added, which will show for each event type its occurrence time and show if the current values
are in violation of EN50160.

  • Green: Everything is ok, no violation
  • Yellow: Close to a violation
  • Red: This event time is in violation of the norm

    • The event messages contain a times stamp, the duration of the event and updated event Type information, as well as event type related
    information and GPS data.
    .

    As receiving notifications or emails on every event occasion can get noisy, we decided to provide a weekly summary of the events in
    addition to the regular notifications.
    The user will be able to decide if he wants to receive this summary, every event, or both. We consider using the traffic light system
    here as well, to increase the readability and enable users to understand the quality of their power supply network, without a lot of
    knowledge of the EN50160 norm.
    We discussed individual intervals and keep it in mind as a possible later feature.

    Best regards
    Stefan

    Easy ubus Daemons with rpcd

    Posted on by nicopace@gmail.com
    Libremesh /etc/banner

    In this article we will see how to create a simple ubus daemon with RPCd.

    Ubus is an amazing piece of software that allow inter-process communication inside and in between nodes.

    For inter-process communications it uses a common bus with synchronous and asynchronous calling, publish/subscribe and event paradigms.

    For in-between nodes communication it provices a JSON-RPC 2.0 Server (and I hope will do WebSockets soon!).

    For more information on how ubus works check out OpenWRT’s Wiki page.

    If you want to expose simple functionality to ubus (or you are prototyping an interface before creating an efficient daemon) you can use rpcd for this.

    This is a simplified example based on the one in the wiki

    $ cat << EOF > /usr/libexec/rpcd/banner
#!/bin/sh

# The RPCd interfaces with commands via two methods: list and call.
case "$1" in
	list)
                # List method must return the list of methods and parameters that the daemon will accept. Only methods listed here will available to call.
		echo '{ "append": { "content": "str"}, "show": { } }'
	;;
	call)
                # The way rpcd calls the methods is by calling your script like this: <script-name> call <method-name> << <json-input>
                # So, in order to retrieve the called method, you need to read $2
		case "$2" in
			append)
				# And in order to retrieve the method parameter, you need to read stdin
				read input;
                                CONTENT=`echo $input | jsonfilter -e '@.content'` 
                                echo $CONTENT >> /etc/banner
				echo '{ "result": "ok" }'
			;;
			show)
				# return json object or an array
				echo -n '{ "content": "`; echo $(cat /etc/banner | text_to_json); echo '"}'
			;;
		esac
	;;
esac

function text_to_json() {
JSON_TOPIC_RAW=`echo $1`
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//\\/\\\\} # \ 
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//\//\\\/} # / 
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//\'/\\\'} # ' (not strictly needed ?)
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//\"/\\\"} # " 
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//   /\\t} # \t (tab)
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//
/\\\n} # \n (newline)
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//^M/\\\r} # \r (carriage return)
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//^L/\\\f} # \f (form feed)
JSON_TOPIC_RAW=${JSON_TOPIC_RAW//^H/\\\b} # \b (backspace)
echo JSON_TOPIC_RAW
}
EOF
$ chmod +x /usr/libexec/rpcd/banner
$ service rpcd restart

    And then we can try the new daemon:

    $ ubus -v list banner
'banner' @xxxxxxx
    "append":{"content":"String"}
    "show":{}

$ ubus call banner show
LEDE blah blah
...

$ ubus call banner append 'Hello world!'
LEDE blah blah
...
Hello world!

    Update:
    @Rundfreifunk comment suggested that if you want to use this RPC commands via JSON-RPC, you need to add authentication parameters.
    In this case, let’s suppose that we want to grant anonymous read access to the banner.show method.

    We could do that like this:

    $cat << EOF > /usr/share/rpcd/acl.d/banner.json
{
  "unauthenticated": {
    "description": "Alfred data types via batman-adv meshrouting",
    "read": {
      "ubus": {
        "banner": ["show"]
      }
    }
  }
}

    The documentation related to ACLs can be found in this OpenWRT wiki page.

    GSoC 2017-netjsongraph.js milestone 1

    Posted on by GeekPlux

    netjsongraph.js(GitHub) is a visualization library for NetJSON. During the first month, gratifying results have been achieved. I have done most tasks for the first stage of netjsongraph.json. Thanks for the nice mentor and this great chance to let me learn a lot of skill and techniques. Here are some results:

    Minor version published

    Npm is currently the largest open source package management community, it is very convenient for developers to release or iterative update netjsongraph.js.

    Development workflow imporved

    I have imported some modern front-end development tools to simplify the development process, including webpack (build tool), yarn (package management), babel, eslint, jest (test framework). Developer can run or test the project by one line command. all tools are advanced and familiar to JS developers, so they can contribute more easily.

    Rewrited visualization by webGL

    I had developed three demos with three different technologies: Sigma.js, Pixi.js and Three.js.
    Sigma.js code looks very concise, but importing of its plugins (such as force layout plugin) in modern way is so hard. Pixi.js and Three.js I used it only to render d3 force layout. And three.js looks slow than Pixi.js but its slowness may caused by lines render way.
    But I finally decide d3 4.0 and three.js is the best choice currently. Because they are both famous and iteration rapidly. Pixi.js is also great but it only support 2d render, so if we want to expand our visualisation to 3D in the future, we can only choose three.js. Three.js is a library which lightweight webGL packaging in order to avoid writing WebGL directly that is very hard and verbose.
    Then I refactored the demo to a Class and write some API backward compatible, including function to fetch data, theme switching, autosize, metadata and nodedata panel and zoom, hover, click event handler. It looks like:
    netjsongraph.js
    Metadata and node data panel are on top left and right corner, force graph for Netjson visualization in the center.

    Tests added

    Some unit tests have beed added and CI has beed integrated by Travis and Coverall. Because I wrote tests rarely before, so I meet some trouble and read a lot docs and examples to learn how to write test and how to use Jest. One problem I’ve met is test failed when import object with WebGL renderer. I’ve send a issue: https://github.com/facebook/jest/issues/3905 .

    Next stage

    I will complete the compatible API in the next week, and below are tasks for next milestone:

    • add optional geographic map
    • add a way to redraw graph by supplying new data
    • add more interactions
    • improve force layout details

    Conclusion

    Next version which have more interaction and rendered by WebGL is comming soon. and it will be soon integrated in the brand new OpenWISP 2 Network Topology module.

    During this period I learned a lot of things, including how to use git rebase, how to write commit messages more clearly, how to write test, KISS principles, worse is better principles and so on. Great thanks for GSoC, Freifunk and mentor Federico.

    ​

    geolocator (Software defined GPS) first evaluation

    Posted on by buttjantarek@googlemail.com
    Hi together,
    Now I am coming to my 1 phase: evaluation. An introduction into the geolocator (Software defined GPS) has been already done last month. You can find it here [0]. In the previously mentioned blog post I have told about the 3 main sub-projects: web backend, gps-share, LEDE Package. In this post I will show you what I have done in each sub-project. Beginning with the web backend:

    web backend

    This sub-project should replace the old web backend of the openwifi.su project. 
    The old backend is split into various programs and scripts, grown over years. First of all I started with the part of receiving positions. This program is written in ruby and just parses the URL as a string. So users can simply call 
    wget -O - http://openwifi.su/api/v1/bssids/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6 2&>1
    and will receive latitude and longitude. The BSSIDs in the above URL are from surrounded WiFi networks. The old ruby code just drops all characters up to the last splash and splits the BSSIDs by comma. The result array will be integrated into and SQL command which returns a position for each BSSID. The code for this part is non public, because the developer doesnt want to risk attacks by hackers. I started porting the old ruby code to golang. Currently I am working on the database connection. So in the next few days the backwards compatibility will be finished. The current code can be found here [1].

    gps-share

    gps-share is a software which provides GPS informations in the local network, written in rust [2]. It supports many GPS hardwares and shares the GPS informations over a network-NEMA format. Zeeshan Ali, the maintainer of geoclue, has already released a first version of gps-share [3]. Zeeshan and I discussed on the Mozilla Location Service Mailing-list about that project [4] . As I told it in the last blogpost, the gps-share is an alternate project to the original idea. This helps geoclue to work directly with GPS-hardware´s and builds an defined interface via dbus to receive geopositions over GPS hardware, surrounding WiFis or other ways.

    LEDE Package

    The last  sub-project is a new LEDE-package, which provides geo positions received by their surrounding WIFIs. I have already built a first version of that package here [5]. I also started discussing with maintainers of the gluon framework, which is a popular framework in the Freifunk community. My plan is to integrate the goelocator also in gluon so routers can automatically receive its position. A first draft can be seen here:

    An idea is that the Web interface of the router could detect installed packages and provide their options on the Web interface. This means, that in the drop down menu, the option “Automatic (geolocator)” will only be shown if the package geolocator is installed. This makes it easier for local (freifunk) communities to integrate new options of position detection into their firmware.
    This is also my last week of my exam period. Now I can work full time on the project. Next plans are improving functionality of the new backend, building CI unit test for the new planed API and building the Mozilla Location Service communication interface to sync the requests between MLS and openwifi.su.

    GSoC 2017 – wlan slovenija – Report 1 – HMAC signing of Nodewatcher data

    Posted on by red_moo

    What’s been done

    The first blog post that describes the idea and goals can be read here.

    After getting a bit more familiar with the code base the first thing that had to be done was to set up the work environment. I chose to go the virtual machine route so an actual router did not need to be used, making the development easier. An instance of the nodewatcher is running on a virtual machine running Ubuntu Linux. It runs in a Docker development environment and is set up by using the provided Docker Compose configuration. To enable data collection from nodes the monitor system also needs to be running.

    To test the the nodewatcher’s HMAC signing capabilities I set up a dummy test node, turned on HMAC signing and used a python script to push some data on to the nodewatcher.

    The next step was to set up a development node. With a lot of help from my mentor Kostko I used firmware-core to create a LEDE virtual machine, compile nodewatcher-agent packages, transferred them to the LEDE machine, used SSH to connect to it, installed them and tested the basic nodewatcher-agent functions. The whole setup and development process was documented in detail in the nodewatcher-agent README. Then I set up the agent’s http.push module and pushed some data on to the nodewatcher instance.

    A local network like one in the above image was set up for development. Due to my lack of experience with networking I encountered some issues like not being able to ssh into the LEDE VM and not being able to send packets from LEDE VM to Nodewatcher VM, but with patience from my mentor it all works and the lesson was surely learned.

    What’s next

    So the hardest part is done, right? I am now working on adding a HMAC signing like it’s already done in the python script. It needs to be developed for both pushing the data from a node to the nodewatcher and pulling the data from a node.

    I am contributing using my github account.

    Onwards! Good luck!