Category: Allgemein

geolocator (Software defined GPS) final evaluation

Posted on by buttjantarek@googlemail.com

Hi everyone,

with this blog post I would like to explain the full Google Summer of Code Project as a final post. For people who haven’t read over the geolocator (Software defined GPS) project before, it might be interesting to read these three blog posts at first:

– geolocator (Software defined GPS) (english)[1] and (german)[2]

– geolocator (Software defined GPS) first evaluation (english)[3] and (german)[4]

– geolocator-software-defined-GPS-second-evaluation (english)[5] and (german)[6]

Otherwise I will give in the following a short overview about the project structure to remind you of it. I structured the Google Summer of Code project into 3 main subprojects:

web backend,

– The web backend named sgps-core is a service, which should give requested clients their geo position.

gps-share,

– The idea of gps-share is to create an udev device, which provides NEMA-formata protocols over tty addicted on information, which is received from the above mentioned backend.

LEDE Package,

– The intention behind this subproject is to develop a new package for LEDE called geolocator, which should provide the geo position of LEDE devices.

Now I would like to give you a full state of each above mentioned subproject. Firstly I will explain about the web backend and finally a peroration including my valediction as a Student by Google Summer of Code.

web backend

Generally the backend service receives over the OpenWLANMap[7] App from mobile phones the mac addresses of surrounding wireless networks linked to GPS positions. This information will be stored into a database. If a device like a WiFi router requests its position, it will send surrounding Wireless mac addresses to the backend and get back a geo position, which is calculated from these information in the database.

The new web backend called sgps-core[8] is an API-core, which should replace the old openwifi.su backend. The old one consists of a collection of different programs in different program languages. sgps-core includes a fully backwards compatibility to the old openwifi API[9] for requesting a position. sgps-core is written in Golang, which processes a lot faster than the old API, which is written in Ruby. sgps-core is more secure because it checks and take only requested strings, which contain only comma separated macaddresses with 12 hex characters.

As a fallback feature, sgps-core is able to receive coordinates from unknown WIFIs by requesting them on Mozilla Location Service (MLS)[10] if there are no db entries for that WIFIs. The position for clients will be returned in form of latitude and longitude. As a quick reminder, here is the schemata from the first post, which represented the the functionality of sgps-core:

The sgps-core solved a problem about calculating the position. The old method counts the average of all latitude values. Analogous for longitude. The new method calls geographic midpoint calculation and needs 4 parameters lat0, lon0, lat1, lon1 (give two take one) which will be explained in detail in following:

deg have to be replace with latitude or longitude value.

rad = deg *π / 180 <- Generally conversion from degrees to radiant.

dlon = (lon1 – lon0) * pi / 180

lat0 = lat0 * π / 180 <- lat0 from degrees to radiant.

lat1 = lat1 * π / 180 <- lat1 from degrees to radiant.

lon0 = lon0 * π / 180 <- lon0 from degrees to radiant.

Converting into Cartesian coordinate system.

Bx = cos(lat1) * cos(dlon)

By = cos(lat1) * sin(dlon)

Calculate new position reference to sphere and Converting back from Cartesian coordinate system into new latitude and longitude:

lat2 = atan2(sin(lat0) + sin(lat1), (cos(lat0) + Bx)² + By²)^(1/2))

lon2 = lon0 + atan2(By, cos(lat0) + Bx)

On this point it is also possible to use a ellipsoid to increase the accuracy of positions. This may be interesting for long distances. For short ones like from seen wireless networks, it is not really relevant.

Converting back to degrees:

deg = rad / pi * 180 <- Generally conversion from radiant to degrees.

lat2 = lat2 / pi * 180 <- lat2 from radiant to degrees .

lon2 = lon2 / pi * 180 <- lon2 from radiant to degrees.

In the last few weeks, I spent a lot of time on discussing with the current server administrator of openwifi.su to deploy the sgps-core on the server for a test environment. But he did not have much time, so we decided to migrate the openwifi.su to our Nordwest Freifunk infrastructure to make the administration more accessible for other people. In my last report I wrote “I will release in the next few days a first version”. This could not be done because of the above mentioned discussion. After the migration I can test that backend on huge databases and compatibility to the DBS. The current code can be found here [11]. For people who want to try the sgps-core please check out the following URL[12].

gps-share

The Idea at the beginning of GSoC17 was to write a program to provide GPS NEMA-formats over a tty udev device. The information for the GPS NEMA-formats should come from the above mentioned sgps-core. As I told in the first blog post I discussed with some people from the Mozilla Location Service Malinglist and it turned out that something similar was already exist called geoclue. To avoid developing redundant software I decided to drop this idea. Instead of it the new plan was to build support for native GPS in gps-share[13], which is an add-on for geoclue. But during the Google Summer of code I had to focus more on the both other subprojects because they are more important, especially for Freifunk. In my peroration I will tell about the future plans, especially for gps-share.

LEDE Packages

The third subproject was to create some opkg packages for LEDE[14] and similar Frameworks. The main package called geolocator provides the geo position of the device via UCI[15]. Positions should be received from the above explained sgps-core. The 4 other packages are only for Gluon[16], which add the configuration options of the geolocator to the Web-interface.

This month I mainly worked on the LEDE Packeges. At the beginning of the august I sent a merge request to Gluon for integrating the Gluon-geolocator[17]. The containing geolocator program was written in ash shell code. While reviewing and discussing about the merge request, I realized that I had to rewrite the program from shell to lua code because Gluon mainly work with in lua written programs. You can find the shell code here[18] and the Lua version here[19]. At the moment I am waiting here for another review and subsequently merging.

The other packages for the Gluon Web-interface are also already in process. The first idea was to create a detection of installed packages to show related configuration options on the Web-interface. This idea was dropped because I found a better solution. The problem is detecting packages on runtime, which means many extra code on Routers, which only have 4MB Flash for example. So I decided to generate the package with their options on compile time. These packages are:

gluon-config-mode-geo-location,

gluon-config-mode-geo-location-with-geloc-map,

gluon-config-mode-geo-location-with-geloc,

gluon-config-mode-geo-location-with-map

The main package is gluon-config-mode-geo-location, which is already exist in gluon, but with a difference Web-interface. Each package should either integrate an open street map or the geolocator options. Integrating both are also possible. For communities which would like to stay on the currently variety of functionalities, it is also no problem not integrate any of these extra options.

Here is how the new packages look like:

I wrote some C++ programs, which generate me the Lua code for the Gluon Web-interface, which is written in Lua. Base on preprocessor variables, the amount of options for each package will be included into the Lua output from the C++ program. These preprocessor variables will be set by selecting one of the above packages. Also PO files for the translation will be generated in the same way. A merge request of the above new packages can be found here[20] I am still working on it.

Peroration and Future plans

Now I am coming to my peroration.The last 3 months were really awesome, just like last year as a student on the Google Summer of Code. I would love to recommend this great opportunity for not only students but also for open source organizations. Students can not only learn a lot of new things but also meet new great people, make new friends and take part in many events. For example: at the beginning of august I was on the SHA2017[21] (Still Hacking Anyway) and had a meetup with some Freifunk communities there. We had a great discussion about a lot of technical stuffs and a nice time for socializing. The SHA2017 took place in Netherland nearby Amsterdam. Another example is : this week I flew to Spain to start my exchange semester. Coincidentally a student from Germany who I met at the beginning of the GSoC17 in Berlin on the WCW[22] (Wireless Community Weekend) is also doing an exchange semester in Spain. We have already emailed each other and planned to meet up in the next months, probly in Barcelona or any other places. As I said above, this is my second time as a student on the GSoC, which means this is also my last time and sadly I have to say goodbye to GSoC as a student now. But maybe next year I can work as mentor to support other students in their great opportunities.

Back to the projects, as i said I’m still working on it. I will finish the Integration into Gluon and LEDE and continue developing sgps-core integrate new features and migrate the infrastructure to a better server. I would like to contact Zeeshan Ali, the maintainer of gps-share and try to help on this project as well. Also I am still working on the hoodselector which is my Google Summer of Code project from the last yeah. You can read about it here[23]. The hoodselector should also integrated into Gluon but it requires for sure a few weeks of work to integrate VXLAN on it. A merge request for can be found here[24].

Also I would like to say thank you to my Mentors Clemens John from the Google Summer of code 2016 and Johannes Rudolph from 2017 and especially to Andreas Bräu who works so hard on the Freifunk Org for many years to give students these opportunities to be a part of the Freifunk Community.

[1] https://blog.freifunk.net/2017/05/29/geolocator-software-defined-gps/

[2] https://ffnw.de/geolocator-software-defined-gps/

[3] https://blog.freifunk.net/2017/06/28/geolocator-software-defined-gps-first-evaluation/

[4] https://ffnw.de/geolocator-software-defined-gps-erste-evaluation/

[5] https://blog.freifunk.net/2017/07/26/geolocator-software-defined-gps-second-evaluation/

[6] https://ffnw.de/geolocator-software-defined-gps-zweite-evaluation/

[7] https://f-droid.org/packages/com.vwp.owmap/

[8] https://github.com/openwifi-su/sgps-core/blob/master/README.md

[9] https://sourceforge.net/p/libwlocate/code/ci/master/tree/master/

[10] https://location.services.mozilla.com/

[11] https://github.com/openwifi-su/sgps-core

[12] http://runner01.ffnw.de:8082/api/v1/bssids/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6http://runner02.ffnw.de/64700274945A,0018E7DC21BB,30FC681F37A6,F81A673626E8,EC086BA8ED18,8416F9A8E2FA,E894F6A230E4,98DED020D00A,EC086B3349B2,F81A677F5CD8,A0F3C1654BA6

[13] https://github.com/zeenix/gps-share

[14] https://lede-project.org/

[15] https://wiki.openwrt.org/doc/uci

[16] https://gluon.readthedocs.io/en/latest/

[17] https://github.com/freifunk-gluon/gluon/pull/1201

[18] https://git.ffnw.de/ffnw-firmware/packages/blob/ba8007e5a6b99306068847ba98f2bb72b7fb2745/ffnw-node-info/files/lib/ffnw/geolocator/geolocator.sh

[19] https://github.com/2tata/gluon/blob/fa935d83a64c27dd23f133bd8cecf5d72ef9281b/package/gluon-geolocator/luasrc/lib/gluon/geolocator/geolocator

[20] https://github.com/freifunk-gluon/gluon/pull/1211

[21] https://sha2017.org/

[22] https://wiki.freifunk.net/Wireless_Community_Weekend_2017

[23] https://blog.freifunk.net/2016/08/22/monitoring-and-quality-assurance-open-wifi-networks-out-client-view-final-evaluation/

[24] https://github.com/freifunk-gluon/gluon/pull/997

 

Implementing Pop-Routing in OSPF – Final evaluation updates

Posted on by Gabriele Gemmi

Hello again, since last updates I worked hard to finish my project and to reach the final milestone for this project.

As I explained in my previous post[1], due to some issues, we’ve decided to change the topic of the project to Implementing Pop-Routing in OLSRd instead of OSPF.

In this last month I completed the code for the OLSRd plugin[2], which I hope will be merged soon [3].In order to allow PRINCE to interact with OLSRd I had to modify the PRINCE source code[4] and create a new plugin[5].

The last part of my GSOC was testing the functionalities of my project.
To perform this tests I used a tool developed by the University of Trento, called “NePA TesT”[6]. NePA allowed me to simulate a mesh network in my laptop and to perform tests on it. The network topology was defined using NetJSON, but for my purpose modified it to use graph generators[7]

To ensure that PRINCE was working correctly on this virtual network I measured the centrality and the tuned timer for each node. Then I compared these values to the ones calculated by the original algorithm. Since the simulated network was real, and it needed a bit of time to converge, I took the last 10 values to avoid to measure errors. This are the maximum errors for each size and each kind of graph:

Maximum of percentage errors calculating nodes centrality

I also measured the “hello” messages’ rate to check if it was being calculated correctly by PRINCE. As I did for the centrality I took the mean of the last 10 values for each node and I compared them against the ones calculated using the python Pop-Routing algorithm.

Maximum of percentage errors calculating “Hello” messages’ emission rate

Hence, as we can see from these tables, PRINCE is calculating the Centrality, and the timer’s value, with a really small error. This test also highlighted a bug (*) in the c_graph_parser library with very that particular kind of graph [8].

The last test I performed was to check whether the message to update the timers’ emission rate was actually modifying the emission rate of the messages.
I used a simple graph: 2 nodes connected by one link. And I captured the traffic with tcpdump, before and after the update message.
After 30 seconds I sent a message to the OLSRd poprouting plugin to update the hello timer to 5s. As you can see from the graph below it is working correctly!

Hello messages measured emission rate

I can conclude that PRINCE is working correctly with OLSRd and now it can be used to enhance the Wireless Community Networks that are still using it.
I would like to thank Freifunk, Ninux and Google for giving me the opportunity to participate in GSoC.

Cheers, Gabriele Gemmi

[1]: https://blog.freifunk.net/2017/07/27/implementing-pop-routing-ospf-july-updates/
[2]: https://github.com/AdvancedNetworkingSystems/olsrd/tree/poprouting
[3]: https://github.com/OLSR/olsrd/pull/38
[4]: https://github.com/AdvancedNetworkingSystems/poprouting/tree/refactor_ospf
[5]: https://github.com/AdvancedNetworkingSystems/poprouting/tree/refactor_ospf/prince/lib/olsrd
[6]: https://ans.disi.unitn.it/redmine/projects/community-newtork-emulator/wiki
[7]: https://github.com/AdvancedNetworkingSystems/wcn_emulator
[8]: https://github.com/AdvancedNetworkingSystems/poprouting/issues/23

PowQuty Live Log GSoC 2017 Final Update

Posted on by Stefan Venz

This is the last blog entry in the series of Google Summer of Code project updates. It will describe, what has been done and what is left to improve in the future in the PowQuty project.

Powquty

PowQuty is a power quality monitoring tool, which can be installed on a router running LEDE or OpenWrt. The router can be connected to an USB oscilloscope providing measurements which powqutyd will process and provide to the user in human readable form.
All this was tested on a x86 based LEDE router.

GSoC 2017

During this Google Summer of Code a live log functionality was added to PowQuty to provide information on power quality events. These events are:

  • voltage dip of 10% – 90% of the reference voltage on the measurement signal
  • voltage swell > 110% of the reference voltage on the measurement signal
  • voltage dip < 10% of the reference voltage on the measurement signal
  • > 5% of the measured values of one specific harmonic are over the defined threshold

    • On event occurrence important information like time, duration and event type will be written to a log file and presented in the extended luci app.

    As shown in the above picture, the interface provides a traffic light like color system behind these events, green indicates everything is within the EN50160 power quality norm. Yellow means, that 80% of the maximum time per week is already reached, red means, that the norm was violated during the last week.
    In addition to log writes, notifications are send out with Mosquitto. Mosquitto is a message broker using the MQTT protocol. It provides a publish/subscribe model, which allows a central server to subscribe to a topic and clients to send out messages to the server with a topic. Mosquitto was already in use in powquty but was extended for EN50160 event notifications. This will allow a central logging of bigger power supply networks, monitored by multiple devices.

    As another option Slack messages can be send by powquty now. Slack is a messaging program, using (as one option among many) webhooks for interaction. Everyone with the webhook can send messages to the team. Sending out messages allows a user to react quickly to changing situations, or get immediately informed on power event occurrence.

    Beginning with pull request 20 [https://github.com/thuehn/PowQuty/pull/20] I started to implement these features.
    First an option was developed to read measurements from a file, as most power supply networks are pretty stable and wont provide many opportunities to test event handling in powquty.
    Afterwards slack and MQTT notifications where added.
    During testing of mosquitto event messages, some seemed to be lost on intervals with many En50160 events in a short period(sometimes more than 35 events per seconds). The solution seems to buffer all events before sending.
    Something similar happened with Slack. Slack only allows one message per second(short bursts excluded) [rate-limits].
    Buffering events would resolve this problem as well. An option for live email notification was considered at first, but was dropped as spam protection would stop most of the messages and probably list users as spammers.
    The last step was to add the traffic light system to the luci app, to enable users without knowledge of the norm to get an idea of the power quality of their power supply network.
    In addition a slack library was written [libwebslack] to send slack messages from PowQuty.

    What can be improved
  • As mentioned before event buffering is needed and will be added after GSoC
  • Email notification in form of a weekly summary
  • More Error checking and handling
  • improving libwebslack to not use libcurl to reduce its size
  • provide libwebslack as OpenWrt/LEDE package, for easier future use

    • Finally I have to thank Dr. Thomas Huehn for being my mentor and Freifunk for their work they do and especially for being a mentoring organisation for Google Summer of Code.
    Last but not least I would like to thank Google for making this all possible.

    If you want to review some of my earlier posts:

  • Introduction
  • First Update
  • Second Update

    • Best regards
    Stefan

    GSoC 2017 – Add MPTCP support in LEDE/OpenWRT trunk – Final

    Posted on by Fejes Ferenc

    Brief summary

    In the first post (beginning of the GSoC 2017 project) I made a few checkpoints to complete at the end of the summer. Now I don’t copy them here, but the good news is all of them completed successfully. The main goal was a very simple transparent multipath Wi-Fi link bandwidth aggregation. The proof in the video above. And the details below.

    Continue reading “GSoC 2017 – Add MPTCP support in LEDE/OpenWRT trunk – Final” →

    OpenWifi – GSoC 2017 final report

    Posted on by Johannes Wegener

    Hello everyone!
    First things first so here is the code that I’ve written. You can find it in these repositories:
    OpenWifiCore (core server application)
    OpenWifiFeed (LEDE/OpenWRT feed with boot flasher and boot notifier)
    OpenWifiWeb (web frontend)
    OpenWifiTemplates (old templating system)
    OpenWifiLocation (plugin that detects the location of a node via google location api and nearby wifi aps)

    I also worked together with Arne to interact with his SDWN controller and agent. We created a small website that informs you on how to use the two tools together. (As I’m writing this it is still somewhat under construction. But I hope everything will be there soon 🙂 ) Furthermore there is also specific documentation (WIP) for OpenWifi here.

    To be a little more precise here is the list of commits that have been done:

    May

    OpenWifiCore
    OpenWifiFeed
    OpenWifiWeb

    June

    OpenWifiCore
    OpenWifiWeb
    OpenWifiLocation

    July

    OpenWifiCore
    OpenWifiWeb
    OpenWifiFeed

    August

    OpenWifiCore
    OpenWifiFeed
    OpenWifiWeb
    OpenWifiTemplates

    Overview about what has been done

    Everything that has been done can be put into 4 categories: authentication/authorization, api, database-model and infrastructure. I want to give you a brief overview of all these categories.

    Infrastructure

    Docker images are now build automatically via TravisCI and deployed on docker hub. You can find more information in my first evaluation blog post. How to use the docker images is described in the documentation. These images are also used for testing.

    database-model

    The graph based database model was made a first class citizen. It now automatically converts between the internal representation and the representation needed to sync to the AP. It is also now possible to create new configurations and create links between these. The query format is also used for authentication/authorization.

    api

    Providing all the new functionality via a REST style API was the focus of this google summer of code (web views are still needed for quite some things). It is used for managing users, services, nodes and changing the graph based configuration model. It is described in the documentation.

    The concept of having a service is also something new that I created together with Arne in august. If an external application needs to make changes to configurations of specific nodes it can just subscribe as a service. The service takes a list of database queries, a name and a shell script with a compare string. If the output (stdout) of the shell scripts matches the compare value the node gets the name of the service as it’s capability (also something that was added during this GSoC). When a node has the capability the queries are applied to it’s config in a regular interval by the job-server.

    authentication/authorization

    This was the biggest part of this GSoC. There are two ways of giving access to a node – either by giving access to a path string of a configuration or by allowing a specific database query. For more details see my second evaluation blog post.

    There were quite some challenges as this system allows for some really complex access configurations and there are still some things to improve here. (see below)

    What else? Aka the smaller bits

    There were also some smaller bits that were done that don’t really fit the other categories. The boot-notifier and boot-flasher scripts were improved, a way to abstract communication with the nodes was started (see documentation), some small UI fixes to the graph view were done. And probably a lot of other smaller fixes I forgot 😉

    Future – or what needs to be done

    I guess the most important thing is to get all of these new features exposed via the web views. That shouldn’t be to hard but wasn’t the highest priority during GSoC. Everything related to the graph should be made aware of all possible path strings that lead to a configuration. Currently just one string is used – this should be lists! The fine grained authorization needs some more testing and I also want to improve the pattern matching by combining the regular expressions you could use to describe a path string. This could be done with greenery. The authorization right now is also just focused on nodes and configuration – it would be nice to restrict access to some views and actions. For example restrict the access to LUCI, sshkeys or executing commands on a node. Furthermore there should be a per node option where “the truth” of a config lies (like if there is a difference between the actual node configuration and the configuration on the server which one is considered the one to go with) – than it would also be nice to disable the sync for some nodes (if manual changes on the node need to be made for example).make

    Acknowledgments

    A big thanks goes out to Google for organizing something cool as the Google Summer of Code, to Freifunk for letting me do this project with them and last but not least Julius for being my mentor!

    Luci2 and Libremesh – GSoC – Final

    Posted on by Marcos Gutierrez

    Result

    In these three months I was working on the implementation of Luci2 (the graphic interface of LEDE / OpenWrt). The project was to translate the functionalities that Libremesh currently uses in Luci to the new proposal. The new environment consists of a backend based on UBUS that exposes JSON with data and the structure of the view.

    As far as Google Summer of Code goes, I was able to make the ubus modules that emit information about bmx6, batman-adv, alignment, spectrum analysis, libremap and finally a series of several utilities. The results can be found in the lime-packages-ui repository.

    Documentation

    Each module has its documentation on the calls and the expected answers.

    To future

    Finally I want to clarify that it is my intention to continue until achieving a complete implementation with the front end and keep the packages made. I will adapt the current luci packages to consume the data from the UBUS modules, so its use can be immediate and not wait for the complete development of luci2.

    Thanks to the Freifunk community and the Libremesh team for giving me the opportunity to participate in this GSoC. No doubt I will continue to contribute to free software, to have more community and free networks.

    netjsongraph.js – Google Summer of Code (GSoC) 2017 summary

    Posted on by GeekPlux

    Throughout the last three months, I was quite fortunate to work for Freifunk on netjsongraph.js under the guidance of my mentor Federico Capoano. Thanks for this invaluable experience that I learned a lot of knowledge and use them in a practical project. Here is a summary of the work I have done during the Google Summer of Code (GSoC) 2017.

    Google Summer of Code project page

    Goal

    netjsongraph.js is a visualization library for NetJSON, a network topology data format. The main goal of netjsongraph.js may be concluded in below three lines (more details you can see in GSoC 2017-netjsongraph.js: visualization of NetJSON data):

    • Apply the modern front-end development tools and add tests workflow (#1, #45)
    • Rewrite it with WebGL (#11, #29, #39, #42, #47)
    • Improve the performance (#41, #44, #46)

    Achievement

    Github Repository : https://github.com/netjson/netjsongraph.js

    Examples on GitHub pages: https://netjson.github.io/netjsongraph.js/

    You can browse all examples on GitHub pages. Some screen shots of the application:
    basic example
    performance example
    The force-directed layout is usually used to visualize network data. It offers insights on the relationships between nodes and links. The previous version of netjsongraph.js is implemented by d3 and it’s rendered using SVG. It would be very slow if there were thousands or ten of thousands nodes or links. So I have to embrace the WebGL speeded up by GPU to have a better performance.

    I have recorded my work in the blog every milestone:

    BTW, It’s a great management method to make members submit weekly reports and blog posts in Freifunk.

    During the three months, there have been 116 commits from me. I created a big Pull Request include them:
    netjsongraph.js #48
    netjsongraph.js project panels
    Almost all goals have achieved:

    • Published a minor version
    • Improved development workflow
    • Tests Added
    • Refactored visualization by Three.js and d3-force
    • Added more interaction like hover (show nodes tooltips), click (show nodes or links information panel), pan and zoom
    • Improved performance

    Especially on performance aspect, it runs efficiently on Chrome reached 60FPS under 5k nodes and 10k links. And if you don’t wanna animation, you can choose the static rendering layout.

    Challenges

    I also encounter some challenges I never met before.

    Event binding and handling

    As you know, WebGL renders all objects in one canvas tag. How to bind events on every geometry? You should use the Ray casting. Raycasting is used for mouse picking (working out what objects in the 3d space the mouse is over) amongst other things. So you can know which geometry your mouse over and add some interaction effect.
    There are thousands of objects and every object has several events you should handle, I had to develop an event controller to manage it.

    Performance

    The bottleneck in this visualizer is performance(#41). I tried many methods to improve it include:

    Reuse geometry and material

    However, the color of every node is different and the one link should highlight itself when it hovered, so the material should be independent and can not use in common.

    Combine the mesh

    Same problem with above. It’s not flexible to combine them to one mesh, different nodes and links should have different positions.

    Static rendering

    Make calculation before rendering, so there is no animation and repaint.

    Using Web Worker

    Web Workers is a simple means for web content to run scripts in background threads. The worker thread can perform tasks without interfering with the user interface. So put static layout calculation into it will be efficient.

    Force-directed algorithm

    There are different complexity and cost in the different force-directed algorithm. The Force-Atlas2 algorithm has some benefits over the force layout implemented in d3-force. So current version may be refactored by an advanced algorithm in the future.

    What is left to be done

    • Add optional geographic map (#40)
    • Using Force-Atlas2 algorithm

    More interactions and features should be added, and performance may be optimized by using new algorithm. I’d like to continue developing this project after GSoC.

    In the end, thanks for the great patience and guidance from my mentors. Thanks for Google to provide me with this rare chance to contribute to an open source community together with awesome members from all over the world. I really appreciate this invaluable experience accumulated this summer and I believe it will have the profound impact on my career and life.

    ​

    GSoC 2017 – Final Milestone

    Posted on by ArneKappen

    tl;dr:
    If you are looking for the work I have done for GSoC 2017, here are the principal repositories containing the source code and documentation:
    – [1] ONOS
    – [2] SDWN controller
    – [3] SDWN agent

    Also, check out the website that Johannes and I created to document how our two GSoC projects can be combined.

    August Progress

    In my last blog post I wrote that during the final coding phase I wanted to focus on applications for my controller platform – mainly an app that integrates my controller with Johannes’ OpenWifi server. While I was programming it, I noticed that I was getting tangled in unstructured dependencies between the different parts of my code. I realized that the controller needed a refactoring to better define its main application-facing interfaces. So I decided to modularize it into 4 separate pieces:

      the SDWN core API,
      its core implementation,
      a command-line interface,
      and a REST API.

    With the refactoring done, I built two applications: the aforementioned OpenWifi integration and a simple client hearing map. Unfortunately I ran out of time just when adding features was fun again. The basics are all in place and working but I did not finish the more advanced applications such as the client load balancer or client authenticator that I mentioned in my last blog post.

    On the more positive side, I really like how the newly structured controller behaves when adding features as well as how the OpenWifi integration turned out. A combined SDWN/OpenWifi setup is shaping up to become a neat management tool to operate a large deployment of LEDE nodes in which most of the initial configuration is automated. Nodes discover the OpenWifi server, download a configuration which makes them initiate a connection to the SDWN controller which gathers information about the nodes’ wireless capabilities and running access points. Throw in the hearing map and the nodes start reporting clients in their vicinity to the controller which makes that information available via its various APIs. Seeing it in action felt rewarding and dampened my initial disappointment of not getting everything done.
    Johannes and I have put together a small website using GitHub pages which explains how to set up and use a combined SDWN/OpenWifi system.

    Also during the past month, I have reached out to the ONOS developers about my work on the compatibility of ONOS’ fork of Loxigen and its upstream version. ONOS recently moved to a much more recent version of the loxigen-generated Java library for its OpenFlow subsystem which should make it easier for me to merge with it my modified ONOS. I want to explore if I can completely separate my code from the controller and have it provide the same functionality as external modules. This would of course greatly simplify future maintenance of the work I did.

    The SDWN System

    The following sections document the structure of the system that I have built and list possible applications.

    The Controller

    The SDWN controller that I have written comes as an OSGi bundle that gets deployed as an ONOS application. It relies on ONOS’ OpenFlow subsystem which manages the connection to the wireless switches in the network. Since I am using a custom OpenFlow protocol extension, I have added a corresponding protocol driver into ONOS’ south-bound interface which is responsible for the handshake with the SDWN agent. The SDWN controller’s core interacts with that driver and ONOS’ OpenFlow controller to create its abstraction of the network elements. It maintains so-called SDWN entities to represent the relevant wireless network resources and their relationships. For example, an access point is running on a network interface card which supports a certain set of transmission frequencies and rates. The controller would represent this state in the form of Java objects for the network interface card, its capabilities as well as the access points and set the references between them appropriately. This information is stored and made accessible through different APIs. If clients associate with the access point, the agent on the node notifies the controller which learns the client’s properties and stores the relationship with the AP. SDWN applications need not communicate with the wireless hardware directly but only work with the abstractions provided by the controller. There are also means for applications to subscribe to certain events such as the association of a new client or the reception of a probe request frame by an access point.

    SDWN Applications

    Applications can provide their services without having to interact directly with the network, they can leave communication details to the controller. For instance, the simple hearing map that I wrote subscribes to 802.11 management frame events. Every time the controller gets notified about such a frame by one of its APs, it informs the hearing map service. The service in turn stores the information about which AP on which switch received the frame on which frequency at what time and with how strong a signal.

    Although I did not get to implement it, all the interfaces necessary to implement a blacklist of banned clients are in place. Simply register with the SDWN controller, hook into the stream of authentication and association request events, and deny unwanted clients based on their MAC address.

    Or you could automatically push clients towards certain frequency bands: Query the hearing map for clients in the vicinity of an AP broadcasting at the desired frequency and check if they are currently associated with an access point on an unfavored channel. If yes, instruct that AP to disassociate and ban the clients for some time and grant their association request at the chosen target AP.

    The Agent

    To manage the network elements from the central controller, each of them needs to run an agent that maintains the connection, responds to queries, and executes commands. The SDWN agent is called wlanflow and it is built with the indigo OpenFow agent framework by the Floodlight project. It is available as a feed to integrate with the LEDE build system and uses UCI for configuration. Please refer to the instructions in the repository at GitHub (REPO LINK) about installation and configuration.
    Wlanflow interacts with different other programs and services on the same machine. For example, it learns about most wireless capabilities through Linux’ netlink interface and subscribes to the standard Linux wifi soft-mac daemon hostapd via LEDE’s ubus IPC bus to receive notifications about incoming wifi management frames.

    The Protocol

    The SDWN controller and agent talk using an extended version of OpenFlow 1.3. The libraries for both the Java controller and the C agent are generated using a modified version of Loxigen.

    Deploying the SDWN System

    Like I said in the beginning of this blog post, I did not manage to realize all the ideas I had for this system. For this reason, the final product is a bit bare and lacks finish. Nonetheless, I encourage you to build a small installation and try out its features if you have spare wireless LEDE nodes lying around. I would appreciate your feedback and suggestions for improvements or new features.

    You can find instructions on how to build and deploy the components of the SDWN system in their respective repositories which are listed at the top of this page.

    Acknowledgments

    Finally, I would like to thank Freifunk for letting me participate in GSoC again. Thanks to Andreas for organizing everything on Freifunk’s side and to my mentor, Julius.

    GSoC 2017 – Attended Sysupgrade – Final evaluation update

    Posted on by Paul Spooren

    Hi Freifunk,

    This is my last post within this years GSoC. I’ll cover the progress, challenges and future of the project.

    tl;dr:
    Direct links to the work I’ve done this GSoC:

    First a small reminder what the project is about: Enable end-users to update their routers to new releases or bulk update installed packages via a simple click in the Luci web interface. The magic lies in a server producing sysupgrade images with the same packages preinstalled as installed on the router. That happens on demand when triggered via the web interface.The image is downloaded securely via encrypted HTTP as the used browser has all certificates installed. The router does neither need certificates nor a correct running clock.

    Progress

    Server

    At this point I’m very happy to announce a test (and usable) version of the attended sysupgrade setup. All created package recipes were accepted into the official repository and are compiled within the daily build cycle. The server runs fine obeying the described API at the Github page. Images are build within a few seconds if the request appears for the first time. The server stores previous requests and forwards to existing images instead of building it again. This reduces significantly the amount of build images due to the likely case of identical images being requested again. In addition some basic information are offered via the web servers status pages.

    The server is implemented in Python3 using flask for request routing and template rendering. While that’s only the tip of the iceberg behind the server lays a rather complex PostgreSQL database validating requests, checking for package changes or transforming packages when updating to another main release. More on the transformations later.

    Images created on the server. A click on manifest shows all installed packages with version. All snapshot builds are deleted midnight UTC.

    To try the the server at https://betaupdate.libremesh.org have a look at the demos I prepared. Continue reading “GSoC 2017 – Attended Sysupgrade – Final evaluation update” →

    GSoC 2017 – RetroShare mobile improvements – Final evaluation update

    Posted on by Angela Mazzurco

    Hi Freifunk and GSoC communities!

    Finally we arrive at the end of the summer and also at this third final evaluation of GSoC!
    I’m proud I have participated to this edition of GSoC and I am thankful for this opportunity.

    This past month I did lot of new work and lot of interesting improvements.

    One of the funniest and visible things are the implementation of the 8biticon library that provides a reliable avatar face generated using GXS id as input, it is reliable in the sense that having the same GXS id as input will generate always the same avatar.

    The nice thing about this is that it can be used as an user friendly way to verify identities, the user doesn’t need anymore to check long fingerprints but she can do a quick verification just by looking at the generated face.

    Another interesting thing I have implemented is sending little files and images via chat using data URL. This works encoding a file to base64 and send it with a header specifying the mime type. The maximum file size is 60KB, and for images, if it’s bigger than that or if it’s size is more than 160×160 pixel, the program resizes it to fit.

    This branch results very interesting for me because it uses all the things that I learned this summer: call C++ code from QML, call Android Java code from C++ interact with Android OS, create custom QML components from C++ code…

     

    Of course, a big part of work has been to find and fix bugs, on Android and on desktop. At the moment the last commit has little known bugs that I’m going to solve as soon as possible.

    One of this bugs causes program crash on some Android devices when send or receive an emoji. This happen because I implement an alternative emoji font if no native compatible emoji font is found. On Android, the Android Emoji font is who provides the typography of emojis, but in some devices its not shown. So I decided to integrate the OpenSansEmoji project. That way, when the program doesn’t detect a compatible emoji font, it uses the OpenSansEmoji.

    But the implementation of that is not working as good as expected and causes the crash mentioned above.


    Another typical work has been to improve the UI aesthetics in little things: correct a height in one place, change the color somewhere else, draw a line separating the contacts. I create a custom button component with animation and the possibility to attach to them an icon. This is done thinking about to also centralize the application style sheets.

    And thinking about the end user, I tried to improve how the user share or import the nodes identities. I put a shortcut on the menu to share identity, and also refactor a little bit the import/export trusted nodes view, making an informative popup that shows you a little bit of information about each option.

     

    Also I have done some little attempt to implement key exchange using the device sound card using libquiet available in C, Java, JavaScript and more languages.

    This is a very interesting functionality that would improve the key exchange process and make it user friendly.

    The functionality is not implemented yet as I believe it would take more time then the GSoC as it seemed more difficult than we thought initially, but I would like to continue working on this topic in the future.

    In latest days I have been working with my mentors to do the final polishing and integrate most of the code in RetroShare official repository master branch, most of the code have already been merged as you can see in this commit e06bffdb by one of my mentors.

    As always you can find all the code and activity track on my gitlab repository, I also suggest to take a look to the detailed technical report and if you are eager to test the results on your phone you can download the Android Package too.

    Finally I want to say goodbye, and again appreciate the opportunity to participate to this exciting GSoC!