geolocator (Software defined GPS) final evaluation

Hi everyone,

with this blog post I would like to explain the full Google Summer of Code Project as a final post. For people who haven’t read over the geolocator (Software defined GPS) project before, it might be interesting to read these three blog posts at first:

– geolocator (Software defined GPS) (english)[1] and (german)[2]

– geolocator (Software defined GPS) first evaluation (english)[3] and (german)[4]

– geolocator-software-defined-GPS-second-evaluation (english)[5] and (german)[6]

Otherwise I will give in the following a short overview about the project structure to remind you of it. I structured the Google Summer of Code project into 3 main subprojects:

web backend,

– The web backend named sgps-core is a service, which should give requested clients their geo position.

gps-share,

– The idea of gps-share is to create an udev device, which provides NEMA-formata protocols over tty addicted on information, which is received from the above mentioned backend.

LEDE Package,

– The intention behind this subproject is to develop a new package for LEDE called geolocator, which should provide the geo position of LEDE devices.

Now I would like to give you a full state of each above mentioned subproject. Firstly I will explain about the web backend and finally a peroration including my valediction as a Student by Google Summer of Code.

web backend

Generally the backend service receives over the OpenWLANMap[7] App from mobile phones the mac addresses of surrounding wireless networks linked to GPS positions. This information will be stored into a database. If a device like a WiFi router requests its position, it will send surrounding Wireless mac addresses to the backend and get back a geo position, which is calculated from these information in the database.

The new web backend called sgps-core[8] is an API-core, which should replace the old openwifi.su backend. The old one consists of a collection of different programs in different program languages. sgps-core includes a fully backwards compatibility to the old openwifi API[9] for requesting a position. sgps-core is written in Golang, which processes a lot faster than the old API, which is written in Ruby. sgps-core is more secure because it checks and take only requested strings, which contain only comma separated macaddresses with 12 hex characters.

As a fallback feature, sgps-core is able to receive coordinates from unknown WIFIs by requesting them on Mozilla Location Service (MLS)[10] if there are no db entries for that WIFIs. The position for clients will be returned in form of latitude and longitude. As a quick reminder, here is the schemata from the first post, which represented the the functionality of sgps-core:

The sgps-core solved a problem about calculating the position. The old method counts the average of all latitude values. Analogous for longitude. The new method calls geographic midpoint calculation and needs 4 parameters lat0, lon0, lat1, lon1 (give two take one) which will be explained in detail in following:

deg have to be replace with latitude or longitude value.

rad = deg *π / 180 <- Generally conversion from degrees to radiant.

dlon = (lon1 – lon0) * pi / 180

lat0 = lat0 * π / 180 <- lat0 from degrees to radiant.

lat1 = lat1 * π / 180 <- lat1 from degrees to radiant.

lon0 = lon0 * π / 180 <- lon0 from degrees to radiant.

Converting into Cartesian coordinate system.

Bx = cos(lat1) * cos(dlon)

By = cos(lat1) * sin(dlon)

Calculate new position reference to sphere and Converting back from Cartesian coordinate system into new latitude and longitude:

lat2 = atan2(sin(lat0) + sin(lat1), (cos(lat0) + Bx)² + By²)^(1/2))

lon2 = lon0 + atan2(By, cos(lat0) + Bx)

On this point it is also possible to use a ellipsoid to increase the accuracy of positions. This may be interesting for long distances. For short ones like from seen wireless networks, it is not really relevant.

Converting back to degrees:

deg = rad / pi * 180 <- Generally conversion from radiant to degrees.

lat2 = lat2 / pi * 180 <- lat2 from radiant to degrees .

lon2 = lon2 / pi * 180 <- lon2 from radiant to degrees.

In the last few weeks, I spent a lot of time on discussing with the current server administrator of openwifi.su to deploy the sgps-core on the server for a test environment. But he did not have much time, so we decided to migrate the openwifi.su to our Nordwest Freifunk infrastructure to make the administration more accessible for other people. In my last report I wrote “I will release in the next few days a first version”. This could not be done because of the above mentioned discussion. After the migration I can test that backend on huge databases and compatibility to the DBS. The current code can be found here [11]. For people who want to try the sgps-core please check out the following URL[12].

gps-share

The Idea at the beginning of GSoC17 was to write a program to provide GPS NEMA-formats over a tty udev device. The information for the GPS NEMA-formats should come from the above mentioned sgps-core. As I told in the first blog post I discussed with some people from the Mozilla Location Service Malinglist and it turned out that something similar was already exist called geoclue. To avoid developing redundant software I decided to drop this idea. Instead of it the new plan was to build support for native GPS in gps-share[13], which is an add-on for geoclue. But during the Google Summer of code I had to focus more on the both other subprojects because they are more important, especially for Freifunk. In my peroration I will tell about the future plans, especially for gps-share.

LEDE Packages

The third subproject was to create some opkg packages for LEDE[14] and similar Frameworks. The main package called geolocator provides the geo position of the device via UCI[15]. Positions should be received from the above explained sgps-core. The 4 other packages are only for Gluon[16], which add the configuration options of the geolocator to the Web-interface.

This month I mainly worked on the LEDE Packeges. At the beginning of the august I sent a merge request to Gluon for integrating the Gluon-geolocator[17]. The containing geolocator program was written in ash shell code. While reviewing and discussing about the merge request, I realized that I had to rewrite the program from shell to lua code because Gluon mainly work with in lua written programs. You can find the shell code here[18] and the Lua version here[19]. At the moment I am waiting here for another review and subsequently merging.

The other packages for the Gluon Web-interface are also already in process. The first idea was to create a detection of installed packages to show related configuration options on the Web-interface. This idea was dropped because I found a better solution. The problem is detecting packages on runtime, which means many extra code on Routers, which only have 4MB Flash for example. So I decided to generate the package with their options on compile time. These packages are:

gluon-config-mode-geo-location,

gluon-config-mode-geo-location-with-geloc-map,

gluon-config-mode-geo-location-with-geloc,

gluon-config-mode-geo-location-with-map

The main package is gluon-config-mode-geo-location, which is already exist in gluon, but with a difference Web-interface. Each package should either integrate an open street map or the geolocator options. Integrating both are also possible. For communities which would like to stay on the currently variety of functionalities, it is also no problem not integrate any of these extra options.

Here is how the new packages look like:

I wrote some C++ programs, which generate me the Lua code for the Gluon Web-interface, which is written in Lua. Base on preprocessor variables, the amount of options for each package will be included into the Lua output from the C++ program. These preprocessor variables will be set by selecting one of the above packages. Also PO files for the translation will be generated in the same way. A merge request of the above new packages can be found here[20] I am still working on it.

Peroration and Future plans

Now I am coming to my peroration.The last 3 months were really awesome, just like last year as a student on the Google Summer of Code. I would love to recommend this great opportunity for not only students but also for open source organizations. Students can not only learn a lot of new things but also meet new great people, make new friends and take part in many events. For example: at the beginning of august I was on the SHA2017[21] (Still Hacking Anyway) and had a meetup with some Freifunk communities there. We had a great discussion about a lot of technical stuffs and a nice time for socializing. The SHA2017 took place in Netherland nearby Amsterdam. Another example is : this week I flew to Spain to start my exchange semester. Coincidentally a student from Germany who I met at the beginning of the GSoC17 in Berlin on the WCW[22] (Wireless Community Weekend) is also doing an exchange semester in Spain. We have already emailed each other and planned to meet up in the next months, probly in Barcelona or any other places. As I said above, this is my second time as a student on the GSoC, which means this is also my last time and sadly I have to say goodbye to GSoC as a student now. But maybe next year I can work as mentor to support other students in their great opportunities.

Back to the projects, as i said I’m still working on it. I will finish the Integration into Gluon and LEDE and continue developing sgps-core integrate new features and migrate the infrastructure to a better server. I would like to contact Zeeshan Ali, the maintainer of gps-share and try to help on this project as well. Also I am still working on the hoodselector which is my Google Summer of Code project from the last yeah. You can read about it here[23]. The hoodselector should also integrated into Gluon but it requires for sure a few weeks of work to integrate VXLAN on it. A merge request for can be found here[24].

Also I would like to say thank you to my Mentors Clemens John from the Google Summer of code 2016 and Johannes Rudolph from 2017 and especially to Andreas Bräu who works so hard on the Freifunk Org for many years to give students these opportunities to be a part of the Freifunk Community.

[1] https://blog.freifunk.net/2017/05/29/geolocator-software-defined-gps/

[2] https://ffnw.de/geolocator-software-defined-gps/

[3] https://blog.freifunk.net/2017/06/28/geolocator-software-defined-gps-first-evaluation/

[4] https://ffnw.de/geolocator-software-defined-gps-erste-evaluation/

[5] https://blog.freifunk.net/2017/07/26/geolocator-software-defined-gps-second-evaluation/

[6] https://ffnw.de/geolocator-software-defined-gps-zweite-evaluation/

[7] https://f-droid.org/packages/com.vwp.owmap/

[8] https://github.com/openwifi-su/sgps-core/blob/master/README.md

[9] https://sourceforge.net/p/libwlocate/code/ci/master/tree/master/

[10] https://location.services.mozilla.com/

[11] https://github.com/openwifi-su/sgps-core

[12] http://runner01.ffnw.de:8082/api/v1/bssids/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6http://runner02.ffnw.de/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6

[13] https://github.com/zeenix/gps-share

[14] https://lede-project.org/

[15] https://wiki.openwrt.org/doc/uci

[16] https://gluon.readthedocs.io/en/latest/

[17] https://github.com/freifunk-gluon/gluon/pull/1201

[18] https://git.ffnw.de/ffnw-firmware/packages/blob/ba8007e5a6b99306068847ba98f2bb72b7fb2745/ffnw-node-info/files/lib/ffnw/geolocator/geolocator.sh

[19] https://github.com/2tata/gluon/blob/fa935d83a64c27dd23f133bd8cecf5d72ef9281b/package/gluon-geolocator/luasrc/lib/gluon/geolocator/geolocator

[20] https://github.com/freifunk-gluon/gluon/pull/1211

[21] https://sha2017.org/

[22] https://wiki.freifunk.net/Wireless_Community_Weekend_2017

[23] https://blog.freifunk.net/2016/08/22/monitoring-and-quality-assurance-open-wifi-networks-out-client-view-final-evaluation/

[24] https://github.com/freifunk-gluon/gluon/pull/997

 

netjsongraph.js – Google Summer of Code (GSoC) 2017 summary

Throughout the last three months, I was quite fortunate to work for Freifunk on netjsongraph.js under the guidance of my mentor Federico Capoano. Thanks for this invaluable experience that I learned a lot of knowledge and use them in a practical project. Here is a summary of the work I have done during the Google Summer of Code (GSoC) 2017.

Google Summer of Code project page

Goal

netjsongraph.js is a visualization library for NetJSON, a network topology data format. The main goal of netjsongraph.js may be concluded in below three lines (more details you can see in GSoC 2017-netjsongraph.js: visualization of NetJSON data):

  • Apply the modern front-end development tools and add tests workflow (#1, #45)
  • Rewrite it with WebGL (#11, #29, #39, #42, #47)
  • Improve the performance (#41, #44, #46)

Achievement

Github Repository : https://github.com/netjson/netjsongraph.js

Examples on GitHub pages: https://netjson.github.io/netjsongraph.js/

You can browse all examples on GitHub pages. Some screen shots of the application:
basic example
performance example
The force-directed layout is usually used to visualize network data. It offers insights on the relationships between nodes and links. The previous version of netjsongraph.js is implemented by d3 and it’s rendered using SVG. It would be very slow if there were thousands or ten of thousands nodes or links. So I have to embrace the WebGL speeded up by GPU to have a better performance.

I have recorded my work in the blog every milestone:

BTW, It’s a great management method to make members submit weekly reports and blog posts in Freifunk.

During the three months, there have been 116 commits from me. I created a big Pull Request include them:
netjsongraph.js #48
netjsongraph.js project panels
Almost all goals have achieved:

  • Published a minor version
  • Improved development workflow
  • Tests Added
  • Refactored visualization by Three.js and d3-force
  • Added more interaction like hover (show nodes tooltips), click (show nodes or links information panel), pan and zoom
  • Improved performance

Especially on performance aspect, it runs efficiently on Chrome reached 60FPS under 5k nodes and 10k links. And if you don’t wanna animation, you can choose the static rendering layout.

Challenges

I also encounter some challenges I never met before.

Event binding and handling

As you know, WebGL renders all objects in one canvas tag. How to bind events on every geometry? You should use the Ray casting. Raycasting is used for mouse picking (working out what objects in the 3d space the mouse is over) amongst other things. So you can know which geometry your mouse over and add some interaction effect.
There are thousands of objects and every object has several events you should handle, I had to develop an event controller to manage it.

Performance

The bottleneck in this visualizer is performance(#41). I tried many methods to improve it include:

Reuse geometry and material

However, the color of every node is different and the one link should highlight itself when it hovered, so the material should be independent and can not use in common.

Combine the mesh

Same problem with above. It’s not flexible to combine them to one mesh, different nodes and links should have different positions.

Static rendering

Make calculation before rendering, so there is no animation and repaint.

Using Web Worker

Web Workers is a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering with the user interface. So put static layout calculation into it will be efficient.

Force-directed algorithm

There are different complexity and cost in the different force-directed algorithm. The Force-Atlas2 algorithm has some benefits over the force layout implemented in d3-force. So current version may be refactored by an advanced algorithm in the future.

What is left to be done

  • Add optional geographic map (#40)
  • Using Force-Atlas2 algorithm

More interactions and features should be added, and performance may be optimized by using new algorithm. I’d like to continue developing this project after GSoC.

In the end, thanks for the great patience and guidance from my mentors. Thanks for Google to provide me with this rare chance to contribute to an open source community together with awesome members from all over the world. I really appreciate this invaluable experience accumulated this summer and I believe it will have the profound impact on my career and life.

GSoC: Improving nodewatcher data representation capability (update 3)

I have been working on improving the wlan-slo main page. This page has not been updated in a long time and it is crucial at introducing new people to mesh networking.

Continue reading “GSoC: Improving nodewatcher data representation capability (update 3)”

GSoC 2017-netjsongraph.js milestone 2

After a month of efforts, netjsongraph.js has been greatly improved.
First, I added some API and options are compatible with previous versions, except those that can not be added to the WebGL element. Followed by updating the documentation and some examples so that users can better understand and try out our projects. Finally, I have optimized interaction and performance.

netjsongraph dark theme
netjsongraph dark theme

Interaction

  • added highlight effect on nodes and links when hovered.
  • added tooltips with node information
  • added pan and zoom interaction
  • added node and link information panel when clicked
  • adjust the size of the canvas when the window resized

Performance

The large part of the performance of force-directed layout is consumed in each element position calculation in every tick. So if the position calculation of the elements is before canvas rendering, it is effective to reduce the performance loss.
So I made a static layout https://github.com/netjson/netjsongraph.js/issues/44, there is no animation when rendering, directly render a static force-directed layout in the page.

Other

Besides, I have refactored the event controller, in order to make zoom and pan or other events added easily.
But now the overall rendering results are not good as our expected. There is a little lag in animation, I currently have no idea about the reason, perhaps because this is not best practice of three.js. In the next stage, I will focus on solving performance problems and improving the visual effect.

Next Plan

The links below may be useful:

  • http://taligarsiel.com/Projects/howbrowserswork1.htm
  • http://igoro.com/archive/what-really-happens-when-you-navigate-to-a-url/
  • https://github.com/alex/what-happens-when
  • https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame

geolocator-software-defined-gps-second-evaluation

Hi everyone,

 

I am now taking you over my second evaluation. A German version of this blog post can be found here [0]. For people who did not read over the geolocator (Software defined GPS) project before, here is a link to the project introducing (english)[1] and (german)[2]. The first evaluation from last month can be found here (english)[3] and (german)[4]. First of all I would like to tell you about the web backend:

 

Web Backend

As I said it in the blogpost before, the new web backend should replace the old one.
The current openwifi.su web backend consists of a collection of different programs in different programlanguages. The new one is written in Golang it has a fully backward compatibility. The new web backend is more secure.  It checks all requested strings at the end of the url that they contain only comma separated macaddresses with 12 hex characters, as a validation check.

 

Also the new web backend fix an issue by calculating the requested position. The method before just adds all latitude values and devides them over the amount of positions. Same for longitude. The new way is a geographic midpoint calculation. First the lat/lon values will be converted into radius and then the radius will be converted into a Cartesian coordinate system, after that the midpoint calculation will be done and will gave us a lat/lon in the form of radius and the values will be converted back into degrees, that is all.

 

The web backend has a new feature which should help to enhance the position quality. If there are BSSIDs unknown in the database of openwifi.su, the new backend sends an extra request for the unknown BSSIDs to Moziller location service [5] and brings that information into the calculation of the response.

 

At the moment the new web backend is being tested. In the next few days a first version of the new web backend will be released. The current code can be found here [6].

 

LEDE Package

The last project is to build packages for LEDE and send merge requests to Gluon. The idea behind that is, for WiFi routers and other WiFi devices to be able to automatically receive their own position. This could be a nice feature for mobile Freifunk Routers. It could also provide a good alternative instead of setting positions by hand. A first version of that package is here [7].

 

For the integration in Gluon I plan 4 new packages. The first one is the gluon-geolocator, which depends on gluon-node-info and provides the programme with the capability of receiving geopositions based on surrounding WiFis.

 

The next 3 packages are just for the Gluon configmode. The configmode is a state that begins at the initial start of a Gluon router and the owner of that router can configure it over a LAN connection.
I would like to integrate the following 3 packages within the gluon-config-mode-geo-location.
They are named:
gluon-config-mode-geo-location-with-geloc-map,
gluon-config-mode-geo-location-with-geloc,
gluon-config-mode-geo-location-with-map

 

As their names indicate, the package gluon-config-mode-geo-location-with-geloc-map should integrate the options of the geolocator and an OSM map into the configmode. The other 2 packages, gluon-config-mode-geo-location-with-map and gluon-config-mode-geo-location-with-geloc should either integrate only an OSM map or just the configuration options of the geolocator. Also the package gluon-config-mode-geo-location should be able to and will show the current location interface without any map or geolocator.

 

A merge request of the above new packages will follow soon. Some of the gluon-config-mode-geo-location extensions are currently not finish yet.

 

In the next few weeks I would like to finish the integration into Gluon, as well as continuing to develop the new web backend and releasing the first version of it. There are many ToDo points currently open e.g. Multicore functionality, optionally other geographic midpoint calculations. At the beginning of August there is the SHA2017[8], which I will use for a marathon session.

 

With best wishes
Jan-Tarek Butt

 

GSoC 2017-netjsongraph.js milestone 1

netjsongraph.js(GitHub) is a visualization library for NetJSON. During the first month, gratifying results have been achieved. I have done most tasks for the first stage of netjsongraph.json. Thanks for the nice mentor and this great chance to let me learn a lot of skill and techniques. Here are some results:

Minor version published

Npm is currently the largest open source package management community, it is very convenient for developers to release or iterative update netjsongraph.js.

Development workflow imporved

I have imported some modern front-end development tools to simplify the development process, including webpack (build tool), yarn (package management), babel, eslint, jest (test framework). Developer can run or test the project by one line command. all tools are advanced and familiar to JS developers, so they can contribute more easily.

Rewrited visualization by webGL

I had developed three demos with three different technologies: Sigma.js, Pixi.js and Three.js.
Sigma.js code looks very concise, but importing of its plugins (such as force layout plugin) in modern way is so hard. Pixi.js and Three.js I used it only to render d3 force layout. And three.js looks slow than Pixi.js but its slowness may caused by lines render way.
But I finally decide d3 4.0 and three.js is the best choice currently. Because they are both famous and iteration rapidly. Pixi.js is also great but it only support 2d render, so if we want to expand our visualisation to 3D in the future, we can only choose three.js. Three.js is a library which lightweight webGL packaging in order to avoid writing WebGL directly that is very hard and verbose.
Then I refactored the demo to a Class and write some API backward compatible, including function to fetch data, theme switching, autosize, metadata and nodedata panel and zoom, hover, click event handler. It looks like:
netjsongraph.js
Metadata and node data panel are on top left and right corner, force graph for Netjson visualization in the center.

Tests added

Some unit tests have beed added and CI has beed integrated by Travis and Coverall. Because I wrote tests rarely before, so I meet some trouble and read a lot docs and examples to learn how to write test and how to use Jest. One problem I’ve met is test failed when import object with WebGL renderer. I’ve send a issue: https://github.com/facebook/jest/issues/3905 .

Next stage

I will complete the compatible API in the next week, and below are tasks for next milestone:

  • add optional geographic map
  • add a way to redraw graph by supplying new data
  • add more interactions
  • improve force layout details

Conclusion

Next version which have more interaction and rendered by WebGL is comming soon. and it will be soon integrated in the brand new OpenWISP 2 Network Topology module.

During this period I learned a lot of things, including how to use git rebase, how to write commit messages more clearly, how to write test, KISS principles, worse is better principles and so on. Great thanks for GSoC, Freifunk and mentor Federico.

geolocator (Software defined GPS) first evaluation

Hi together,
Now I am coming to my 1 phase: evaluation. An introduction into the geolocator (Software defined GPS) has been already done last month. You can find it here [0]. In the previously mentioned blog post I have told about the 3 main sub-projects: web backend, gps-share, LEDE Package. In this post I will show you what I have done in each sub-project. Beginning with the web backend:

web backend

This sub-project should replace the old web backend of the openwifi.su project. 
The old backend is split into various programs and scripts, grown over years. First of all I started with the part of receiving positions. This program is written in ruby and just parses the URL as a string. So users can simply call
wget -O - http://openwifi.su/api/v1/bssids/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6 2&>1
and will receive latitude and longitude. The BSSIDs in the above URL are from surrounded WiFi networks. The old ruby code just drops all characters up to the last splash and splits the BSSIDs by comma. The result array will be integrated into and SQL command which returns a position for each BSSID. The code for this part is non public, because the developer doesnt want to risk attacks by hackers. I started porting the old ruby code to golang. Currently I am working on the database connection. So in the next few days the backwards compatibility will be finished. The current code can be found here [1].

gps-share

gps-share is a software which provides GPS informations in the local network, written in rust [2]. It supports many GPS hardwares and shares the GPS informations over a network-NEMA format. Zeeshan Ali, the maintainer of geoclue, has already released a first version of gps-share [3]. Zeeshan and I discussed on the Mozilla Location Service Mailing-list about that project [4] . As I told it in the last blogpost, the gps-share is an alternate project to the original idea. This helps geoclue to work directly with GPS-hardware´s and builds an defined interface via dbus to receive geopositions over GPS hardware, surrounding WiFis or other ways.

LEDE Package

The last  sub-project is a new LEDE-package, which provides geo positions received by their surrounding WIFIs. I have already built a first version of that package here [5]. I also started discussing with maintainers of the gluon framework, which is a popular framework in the Freifunk community. My plan is to integrate the goelocator also in gluon so routers can automatically receive its position. A first draft can be seen here:

An idea is that the Web interface of the router could detect installed packages and provide their options on the Web interface. This means, that in the drop down menu, the option “Automatic (geolocator)” will only be shown if the package geolocator is installed. This makes it easier for local (freifunk) communities to integrate new options of position detection into their firmware.
This is also my last week of my exam period. Now I can work full time on the project. Next plans are improving functionality of the new backend, building CI unit test for the new planed API and building the Mozilla Location Service communication interface to sync the requests between MLS and openwifi.su.

GSoC 2017-netjsongraph.js: visualization of NetJSON data

Project intro

NetJSON is a data format based on JSON(What is NetJSON?), and netjsongraph.js(GitHub) is a visualization library for it. This library has attracted quite some interest from around the world, but there are some defects, such as tests and modern build process lacking.

Therefore our goal is to improve the features and development workflow of netjsongraph.js. To be specific:

  • make it faster with large numbers
  • make it more mobile friendly
  • use modern tools that are familiar to JS developers, so they can contribute more easily
  • add automated tests so we can be more confident of introducing changes
  • get rid of complex features
  • make it easy to extend, so users can experiment and build their own derivatives
  • make it easy to redraw/update the graph as new data comes in, at least at the library level we should support this
  • geographic visualization (like https://ninux.nodeshot.org/ (nodeshot project)

Arrangement

About me

I’m a graduate student from China and also a front-end developer with more than one-year working experience. And now I am interested in the Data Visualization and already made several visualization projects of network structure. So lucky my proposal selected by Freifunk in Google Summer of Code 2017. It’s a great opportunity to participate in a promising open source project. Thanks for my mentor’s guidance and hope I can finish an excellent job. So I listed the following plan:

Tasks and Schedule

  • create a new branch: build the project with yarn, Webpack and Babel. 1 week
  • To build a (mostly) backward compatible version 1 week
  • draw a demo graph using canvas or WebGL. 2 week
  • make a example page to show visualization results. 1 week
  • add test(using Ava and XO) and CI. 1 week
  • discuss and design visualization view 1 week
  • import and integrate with OpenStreeMap or Mapbox to make a map. 1 week
  • visualization implemention. 8 weeks
  • beautify the visualization. 1 weeks
  • improve visualization and test. 4 weeks
  • design interface for plugin (to make this library extensible) *2 week

geolocator (Software defined GPS)

Hi everyone,
 
My name is Jan-Tarek Butt. I am in my fourth term of computer science at Emden University in Lower Saxony (Germany). Some of you may know me from the last Google Summer of Code. I am one of the students who is participating for Freifunk in the Google Summer of Code 2017. In the following post I would like to introduce you to my Google Summer of Code project. I split up the project, same like last year, into 3 main subjects: web backend, gps-share, LEDE Package,

web backend

The first one is a restful API backend[0], which should include a backwards compatibly to the api format of the old openwifi web backend[1]. The new backend should be written in Go, which will be clearer in the next few days. The New API should use Mozilla Location Service as fallback if it is not possible to determinate a position based on the own db entries. Received WiFi information can also redirect to Mozilla Location Service[2]. So some of you might have the question what is Mozilla Location Service. It is simply a service with a big database full of wardriving informations from mobilephones. Mobilephones scan for surrounded wifi networks and send this with its own position to the Mozilla Location Service. These datas will be integrated into the database. If you want to know your position without having GPS, just send only your surrunded wifis to the Mozilla Location Service and you will receive your position based on the wifi informations see the picture below.

gps-share

The first Idea was to create a software defined GPS as udev device which should provide NEMA-famata protocols over tty and receive the necessary informations over wifi. While discussing with people on the Mozilla Location Service Malinglist, it turns out that something similiar is already done in geoclue which provides geo coordiantes over dbus and can receive coordinates via Mozilla Location Service. geoclue works with network-NEMA and Mozilla Location Service. Over network-NEMA geoclue can receive GPS informations from Mobilephones and GPS modems working over local networks. To avoid of developing redundant software, i discussed with Zeeshan, the maintainer of geoclue, about building support in gps-share for standalone GPS instead [3]. This is a software which should provide GPS NEMA-Format from GPS hardware over network-NEMA for geoclue. gps-share is written in Rust for ensuring reliability.

geolocator

The third subproject is a new lede package called geolocator, which should provided geo positions received by their surrounded WIFIs. The WiFi informations should be sent to the new above mentioned backend[4].
cheers
Jan-Tarek Butt

Our GSoC 2017 projects

In 2017 we got 15 slots for our projects. That’s great, we’ve never got so many slots for GSoC before. Our students are from 8 countries. They do their work for 6 sub organizations.

Until the end of May the Community Bonding Period takes place, so students can introduce their projects to the community. They also prepare together with their mentors the folling coding period.

This post will give a short overview on the selected projects. In the next weeks some detailed blog posts will follow describing all the projects.

GSoC project overview

Title Org Student Abstract
RetroShare Improvements retroshare ange The aim of this proposal is to improve RetroShare incrementally during the summer in the following work lines:
– Semi-automatic RetroShare friendship suggestion based on phone contacts
– Semi-automatize JSON API code and documentation generation
– Multiple simultaneous heterogeneous network connection for each peer
– Improve RetroShare Android usability and performances
– Seamless key exchange via Quiet Modem
– Chat and messages multiple device synchronization
– Directories multiple device synchronization
Implementing Pop-Routing in OSPF ninux Gabriele Gemmi Prince is a network daemon that continuosly monitors the network topology and sets the timers for OLSRv2, it is developed in C and we are currently deploying it in the Ninux community network in Florence. This year
we want to implement Pop-Routing for another link-state routing protocol: OSPF. OSPF is the state-of-the-art interior routing protocol for wired networks and it is used also in some wireless community networks. This
project consists in realizing a plug-in of the OSPF open source routing daemon (Quagga or Bird) that will:
– Expose the network topology to Prince using NetJson
– Receive and re-set the network parameters from Prince
Improving nodewatcher data representation capability wlan slovenija rasovica Currently nodewatcher has very limited overview of used IP space without more precise division of existing and used subnets. This could be improved using compact and colorful matrix with IP ranges and links to nodes. I would display data about used ips similar to the [IP Census 2014](http://census2012.sourceforge.net/images/hilbert_icmp_map_lowquality.jpg), better tables and other data representation that would help explain the network structure, it would help in explaining the network and its statistics even to an uninformed observer. I would also update the look and functionality of other graphs and tables representing statistical data from nodewatcher
Extending LoxiGen and ONOS to enable SDN control of wireless switches via OpenFlow lede Arne Kappen LoxiGen is a tool by the Floodlight project to generate OpenFlow libraries for several programming languages including Java and C. The goal of this project is to extend LoxiGen to include messages for controlling wireless switches using OpenFlow.
It will include the necessary extensions to LoxiGen as well as a proof-of-concept implementation of the control channel. This encompasses an application for the ONOS SDN controller and an agent for the LEDE router operating system.SDN-enabled wireless switches offer comfortable centralized management of larger deployments and may provide the basis for more complex use-cases in the future such as network slicing and seamless mobility.
OpenWRT/LEDE Configuration Management lede Johannes Wegener Improve the proof-of-conecpt version of a OpenWRT/LEDE configuration software.
Web interface for Retroshare, a secure communication software retroshare Stefan Cokovski I am applying to work on the web interface for RetroShare, secure communication software. RetroShare already has a web interface with limited functionality and questionable look, compared to the main Qt GUI which RetroShare uses. My goal will be to improve the already present functionalities, introduce new functionalities to the web interface and design a new look for the web interface.
Implement NetJSON output in ubus (OpenWRT/LEDE) lede Arunkumar Ravichandran To bring in support for NETJSON object in the LEDE/OpenWRT Linux distributions. The support for NETJSON is brought in at the interconnect system- ubus. To add support for a new ubus API which allows retrieving these two NetJSON object types: DeviceConfiguration and DeviceMonitoring.
DeviceConfiguration NetJSON object is filled in using the plugins available in System Configuration Abstraction Layer(SCAL).
DeviceMonitoring NetJSON is retrieved by reusing the code parts and adding hooks in the nodewatcher-agent.
LibreMesh Attended Sysupgrade libremesh paul0 Performing updates on routers is quite different from full Linux distribution. It’s not sustainable to do release upgrade via a packet manager. Instead it’s usually required to re-flash the system image. Depending on the installed packages an image rebuild may be to complex for regular users. A more convenient way is needed.

This project will implement an “image as a service” server side which provides custom build images depending on installed packages. A notification in the web interface will notify about the new release. After image creation a one-click installation is offered within the web interface.

The server side implementation will use established tools like LEDE’s ImageBuilder to provide an generic approach for image creating. In this way the entire OpenWrt/LEDE community including several community-mesh firmware projects will benefit from that new update routine.

Spectrum Analyzer libremesh Nicolas Andres Pace I believe that Community Networks’ Operating Systems could use the Spectrum Analysis a lot to make better decisions on how they use the electromagnetic field wisely and efficiently.

As of now, no Free OS has been using Spectrum Analysis, so they can’t know whether the radios are being used efficiently or not.

This module would allow all routers that use Atheros radios (the main radio manufaturer around) to see how is the spectrum and take better decisions on which channels they use.

Deliverables
* Create an OpenWRT/LEDE package to get Spectrum info from the radio.
* Create a LuCI interface to visualize the Spectrum information retrieved.

HMAC signing of Nodewatcher data & IPv6 support for Tunneldigger wlan slovenija red_moo Currently all monitoring reports by WLAN nodes are unsigned and can be spoofed by anyone. This represents a security problem and a possible solution is to add HMAC-based signing to encrypt node monitoring reports to prevent spoofing.
Nodewatcher’s monitoring system will also be improved so it will generate an event and a warning in case of a signature verification failure.The aim of the second part of this GSoC project is to design and implement IPv6 support for Tunneldigger so that it works in IPv6-only and IPv4/IPV6 mixed environment where both server and client have IPv6 connectivity in some form.
Add MPTCP support in LEDE/OpenWRT trunk lede SPYFF Create an MPTCP supported OpenWRT/LEDE trunk and ensure its operation in a multipath Wi-Fi aggregation environment. The trunk should contain the tooling for establish a SOCKS proxy for operating systems without MPTCP support.
geolocator (Software defined GPS) FF Nordwest Jan-Tarek Butt A dynamical and flexible geolocator which should provide a software defined GPS device and receives its position over WIFI hardware with various services like openwifi.su, MLS and etc.
netjsongraph.js: visualization of netjson data ninux GeekPlux NetJSON is a great work attracted some interest from around the world, but there are a lot of defects. And I’m a front-end developer and now focus on Data Visualization, so I want to contribute to NetJSON. In my proposal, I come up with my idea and list some tasks and schedules.
lime-webui: port to LuCI2 libremesh Marcos Gutierrez Make an inventory of all LuCI components that LibreMesh uses and that will not be compatible with LuCI2. Analyze which dependencies must be replaced or rewritten and generate the views according to the new framework selected.
Powquty Live-Log lede Stefan Venz PowQuty provides statistics from measurements taken by a USB oscilloscope. The statistics provide information about the power distribution network, but don’t highlight the most important information. Power networks need to comply with EN50160, if this norm is violated, damage to connected devices is possible.
To avoid damage, powqutyd will log violations and notify users on occurrence.