Year: 2017

Choosing a Spectrogram visualization Library in Javascript

Posted on by nicopace@gmail.com

Today I decided to invest time on choosing the right Spectrogram library in order to create a visualization of the spectral scans.

The requirements that I thought relevant are:

  • size: as this library needs to be installed inside the routers, the Spectrogram should be as small as possible.
  • customizable: most of them are prepared to show information in the audio spectrum… they should be prepared to show spectrum in different frequencies.
  • realtime: the library must render information streamed in realtime. The ones that uses a microphone can be adapted to do that.
  • well mantained: this can be measured based on stars, commits, commiters and forks.
  • responsive: if you resize the container, the graph resizes.

This is the result of the Survey:

rank repository works realtime weight (kb) responsive stars forks last update notes
1 https://github.com/drandrewthomas/Speccy yes yes 3 no 0 0 2017/03/22
2 https://github.com/sebleier/spectrogram.js yes yes 4 no 7 3 2013/09/09
3 https://github.com/pachacamac/spectrum_analyzer yes yes 5 no 0 0 2015/09/09
4 https://github.com/miguelmota/spectrogram yes yes 8 no 51 9 2017/05/22
5 https://github.com/pietgeursen/inu-spectrogram yes yes 166 no 2 0 2016/07/29
6 https://github.com/borismus/spectrogram yes yes 2000 yes 78 12 2014/06/13 have to refactor, cause it uses the microphone
– https://github.com/vlandham/spectrogramJS yes no 500 no 65 5 2014/01/23 it doesn’t work in realtime
– https://github.com/abarbu/audio-annotation no? ? ? ? 4 0 2014 not instalable after some time
– https://github.com/arirusso/d3-audio-spectrum no yes 83 18 2016/11/22 throws errors in javascript console
– https://github.com/octatone/spectrogram no ? 110 8 0 2013/08/11 seems to work only in the safari browser, couldn’t test

Most of the Spectrum Analyzers are very simple ones… till now I havent found that allows advanced things like multiple levels of zoom, pause and resume, etc.
In the future we may need to do some work to implement this functionality.
Lets discuss the main ones.

Reviews

Speccy

It is a very Spectrogram traditional visualiation with an additional realtime curves graph on the upper part. It draws from top to bottom.
In relation with the visualization, the other relevant characteristic is that it draws the graph in a continuous fashion… so the old information get’s lost, and the rest scrolls each time new information arrives. We could call this one a `scrolling drawer`.
It is the most lightweight (pretty close to the next one) weighting 3kb.
The code is not elegant or customizable at all, but is small enougth to understand it easily and include new updates.

Spectrogram.js

This one is a standard Spectrogram, could be characterized as a `rolling drawer` in the sense that once it gets to the end of the canvas, it continues drawing at the beginning of the canvas, `rolls` to the other side, instead of `scrolling` the content.
This one is very lightweight also, 4kb, and the code is very easily updateble to support external sources of data.

Pachacamac’s spectrum_analyzer

This is also a `scrolling drawer`.
It looks better for my taste (probably related with the color scale choosen) but at the functionality level is quite similar to the previous one.
The code is also very small, so it would be easy to understand it. Many configurations are hardcoded.

Mighel Mota’s Spectrum_analyzer

This is still very lightweight, but is still very basic.
The one good thing about this is that is coded in a very professional fashion… The developer uses TDD, the code is very well organized, the documentation is simple but comprenhensive, has pause and resume functionality.

inu-spectrogram

This one doesn’t bring any new stuff to the table.
It uses new technologies (like JSX) but doesn’t improve in the way it is done, or the functionality it brings.
It weights many orders of magnitude more (2000kb), so it is out of the discussion.

Next steps

I haven’t decided between Speccy and Spectrogram.js … but I know that with any of these I will need to do a lot of work, cause many of the desired features are not there yet (customization, mantainability, responsiveness) and others desirable are not there either (like drawing rulers, pause/resume).

I’ll build a prototype with one of those to move forward with the Spectrum analyzer for LibreMesh.

OpenWifi status report before 1st evaluation

Posted on by Johannes Wegener

Hello everyone,
this is the status report of the OpenWifi project before the first evaluation. I like to point out in this blog post what has been done and how I like to proceed the next weeks. Looking at my initial schedule I did some things that are not on that list and missed some others. So I also like to revamp the schedule. I did a Google-document for that and I’m happy to receive comments about it 🙂

I think a lot of things that have been done are the foundation for everything else to come and a lot of things that have not been done are low hanging fruits.

I think overall my initial schedule contained too many elements and I need to focus more on what is important and what are optional changes. I think I want to focus on testing/docker/https and authentication/authorization until the next evaluation and do the communication plugin-API from then until the end. Everything GUI-related might happen after GSoC.

What has been done

Testing, Docker and CI

That is the biggest part I did but I think also the most important point to get the project to a more professional level and make usage easier for new people. There is a docker hub for openwifi docker images. This way you don’t have to build the image – you just need to pull it. The image is build by TravisCI on new commits – so it is up to date.

I also started to implement tests based on docker images. There is a test that boots up a LEDE docker image and an openwifi docker image and checks if the LEDE container discovers the openwifi image and registers to it. This test is also done by TravisCI.

Just last week I also updated the docker image in the way that it can now also optionally ship with nginx and therefore use TLS! Nginx is also doing the client side certificate validation and sets a header value according to the outcome. (I guess something similar is also possible for apache and other servers)

TravisCI testing has also been added for pyuci.

Discovery Script

The discovery script has been completely revamped and is now a lot more readable. It also serves now as small client that updates the registration every 3 minutes.

pyuci

Pyuci has been refactored and extended. Diffs are now a separate class and can be applied and reverted from a config. Also the testing has been extended somewhat and is done by TravisCI.

Authentication

A basic user management has been added. That is a database entry and password hash storage using passlib and an openwifi authentication scheme. The admin user can add and modify users via a rest API.

I also thought about how to implement rights management – more on that below.

New DB Model

The new database model has not been on my initial schedule so much. But actually there is a lot to do. What has been done is proper syncing via sqlalchemy events of the database representation and the rpcd-json representation.

Revisions

OpenWifi now saves change to a config in a revisions database table.

What is going to happen next

Testing

Since there is now a proper infrastructure for docker based testing it should be used more like for testing https and configuration changes.

More tests are needed for the new database model and pyuci.

New DB Model

The new db model needs testing and methods for adding and removing of links and configurations. It should keep IDs consistent upon updating (to allow configuration sharing) -> so maybe make use of new pyuci diff features.

Also I should get rid of all old configuration access (like old templating and configuration changing). It might be nice to have a gui for the DB configuration but this has not the highest priority.

Authentication

The authentication needs to authenticate API keys and client side certificates.

Authorization

Give users/API-keys access based on node and pattern matching. Add option to restrict registration to client side authentication.

The pattern matching could be in the form the current master-config query or a path string (like network.lan.ipaddr or network.[interface].ipaddr or network.lan.*). It should be possible to set patters for read only, read write and restrict. (For example to have a eveything mathing pattern but restrict access to one specific pattern).

It should also be possible to restrict access luci2 (it doesn’t make sense to restrict the db querys if luci2 access is allowed ;)).

Discovery Script

The discovery script should be updated to setup capabilities and communication protocol (like rpcd, rpcd-TLS, NetJSON, etc.) on registration.

Nice to have features would be that it has an option to stop once a registration was successful and to accept a retry count.

I the future it would be nice to work together with OpenWISP and have and share the discovery mechanism.

Luci2 on Libremesh – June

Posted on by Marcos Gutierrez

In this month I was making several advances in the implementation of Luci2 in Libremesh. Mainly the tasks were to generate a image of the firmware with Luci2 and analyze through a study of the traffic the way in which the rpc acutally builds the menu and manages the ACLs. The idea is to reduce the size to the maximum so that its use is feasible and at the same time reduce the dependency to the frontend frameworks.

All this was possible because I traveled to the Battelmesh in Vienna, where I was able to meet and talk with different members of the team and especially with Jow, who explained the current state of development and put me in contact with other developers. We also talked about possible changes and implementations to ensure some retrocompatibility or at least a simple form of migration from Luci to Luci2.

What am I doing

  • Provisioning a rendering system on the client based on a JSON structure (not yet fully determined)
  • Analyzing how to modify the output of CBI so that the output is not an html but a JSON structure
  • Trying to implement an automatic view generator for exposed elements in uci via ubus and rpc

Spectrum Analyzer for LibreRouter June Updates

Posted on by nicopace@gmail.com

What I have been working on

During the past weeks, I had the chance to work on the spectrum analyzer in the following topics:

  • During the BattleMesh, I has the chance to engage with several developers that are working on topics related:
    • Felix Fietkau is one of the developers of the ath9k module, and we had a conversation to understand better the inner workings of the driver, and what the output of the module will serve me for the Spectrum Analyzer functionality
    • Paul Fuxjäger from FunkFeuer, with whom we discussed some potential uses of the module, along with other collaborations that could arise in the future related with the module.
  • I engaged with the FFT_eval project’s source code, that is used to decode radio i/q signals into easily representable values, and added a JSON output for the data. Instead of continue our own fork of the project, I did this in the mainstream project, promoting one codebase. The merge request is currently pending: https://github.com/simonwunderlich/FFT_eval/pull/13 . Many thanks to Gui Irribarren and BrainSlayer from the DD-WRT project to provide most of this implementation.

Once I engaged with the LibreMesh projects, I understood that one of the purposes of having the Spectrum Analyzer was to be able to do a Frequency Survey in the TVWS spectrum. This is very valuable because one of the proposals of using this part of the spectrum is through a regional database of usage, where you can ask for permission to use a frequency and the database needs to authorize you.

So, my other job has been to seek for current implementations of TVWS Database and, in particular, the PAWS protocol (an IETF draft proposal for TVWS Databases). I managed to found a team that is working on this (Prof. Karandikar from the IITB of Mombay and his OpenPAWS project) and we are talking to see if we can collaborate.

That’s a rougth report on what has been happening during the last weeks.

What I’ll be working on

I’ll describe the architecture that I expect to implement in the upcoming weeks.

With the help of ubus, I will be working on an event based architecture that involves the following parts:

The yellow parts are the new parts.

A brief description of the parts involved:
* Spectral Scan Manager: It manages ath9k states, recovers i/q data from the atheros modules and hands them over through ubus
* Spectral Scan Decoder: FFT_eval wrapper that will receive Spectral Scan Manager i/q data and turn it into JSON
* Spectral Analysis Collector: A configurable daemon that will collect the Spectral Scan Decoder data for further analysis. This collection could be kept in memory or sent to a secondary server (like the OpenPAWS server)
* Visualization Module: Will access the information handed by the Decoder or the Collector (depending which information we would like to access) and visualize it in a Waterfall graph.
* Spectrum Availability CLI Interface: This is a potential proposal (if time allows) to have a simpler interface that can be accessed from command-line. Could implement something similar to this: https://wiki.mikrotik.com/wiki/Manual:Spectral_scan

For the next week, I’ll implement the Spectral Scan Manager, the Spectral Scan Decoder wrapper, and a simple visualization.

Spectrum Analyzer in the context of LibreRouter

Posted on by nicopace@gmail.com

Hello to all!

My name is Nicolas Pace and this is the first time and engage into participating in the GSoC for Freifunk.

For this opportunity I’m engaging with the LibreMesh community on the context of the LibreRouter project by implementing a Spectrum Analizer for LibreMesh and also for LEDE/OpenWRT.

Spectrum Analisis is a very powerful tool for anyone that wants to enhance the quality of the links created, but also can be used as a base for more complex functions like diagnose of the physical layer, or many other things that have been implemented in other firmwares.

 

What has been done already

During this last weeks I’ve the chance to engage the community, and also to deepen my understanding of the problem at hand.

Also, I’ve got a working prototype of the command-line interface, and a prototype of code that has been used to show that information.

Next steps

  • To create a lightweight service that shares the information with the web
  • To make a nice interface for the output

Thanks for the opportunity of joining you, and hope to deliver as expected!

Bringing a Little SDN to LEDE Access Points

Posted on by ArneKappen

Hi everyone!

My name is Arne Kappen and this is the beginning of my second participation in GSoC for Freifunk.

Last year, I implemented an extension for LEDE’s netifd which enabled network device handling logic to be outsourced to an external program and still integrate with netifd via ubus. My work included a proof-of-concept implementation of such an external device handler allowing the creation and configuration of OpenVSwitch OpenFlow switches from the central /etc/config/network file [1].

Sticking with Software-Defined Networking (SDN), this year I am going to provide the tools to build SDN applications which manage wireless access points via OpenFlow. The main component will be establishing the necessary message types for the control channel. I am going to extend LoxiGen to achieve this. In the end, there should be OpenFlow libraries for C and Java for the development of SDN applications and their agents running on LEDE.
I will also write one such agent and a control application for the ONOS platform to test my implementation.

My ideal outcome would be a REST interface putting as many of the APs configuration parameters under control of the SDN application as possible. Such a system could provide comfortable management of a larger deployment of LEDE access points and be a stepping stone for more complex use cases in the future.

I am looking forward to working with Freifunk again. Last year’s GSoC was a great experience during which I learned a lot.

[1] Last Year’s GSoC Project

Implementing Pop-Routing in OSPF

Posted on by Gabriele Gemmi

Hello everyone.

I’m Gabriele Gemmi, you may remeber me for… Implementing Pop-Routing[1]
This is the second time I participate in GSoC and first of all I’d like to thanks the organization for giving me this opportunity.
Last year I implemented PR for OLSR2. The daemon, called Prince [2], is now available in the LEDE and the OpenWRT feeds.

What is Pop-Routing

PR is an algorithm that calculate the betwenness centrality [3] of every nodes of a network and then uses this values to calculate the optimal message timers for the routing protocol on each node. In this way a central node will send messages more frequently and an outer one less frequently.
At the end the overall overhead of the network doesn’t change, but the convergence gets faster.

Objectives

My project focuses on extending Prince functionalities to use Pop-Routing with OSPF. I decided to work with BIRD, since it’s written in C and it’s already available for OpenWRT/LEDE
In order to do this I need to develop 2 components:
— A plugin for BIRD that expose the OSPF topology in NetJSON and allows to update the timers
— A plugin for Prince that communicate with the BIRD plugin

I already started developing the former [4], and I’m looking forward to implement the latter.
I’ll keep reporting my updates here, so stay tuned if you wanna hear more.

Cheers, Gabriele

[1]: https://blog.freifunk.net/2016/05/23/implementing-poprouting/
[2]: https://github.com/AdvancedNetworkingSystems/poprouting/
[3]: https://en.wikipedia.org/wiki/Betweenness_centrality
[4]: https://github.com/AdvancedNetworkingSystems/bird

GSoC 2017 – RetroShare mobile improvements

Posted on by Angela Mazzurco
Hi readers! I am Angela Mazzurco and I am very grateful to the GSoC community
(Google, Freifunk, RetroShare etc.) for giving me te possibility to participate
as GSoC student this year!
I study Architecture and Engineering at Pisa University, and here in Pisa I am
involved in the local community network (eigenNet/Ninux.org).
Thanks to the local community I get to know RetroShare and now I use it on my
daily life when I am in front of my laptop. Remote comunication today is very often
displaced from the personal computer to the smart-phones, because of this very
often I have to downgrade to less ethical and less secure communication
platforms, because most of my friends are reachable only on the smart-phone.
This last unfortunate situation inspired me to help developing RetroShare for
mobile phones.
In this deirection the RetroShare community has already done some effort but
still the Retroshare Android app is in an early stage and need much improvement.
I‘ll give my contribution to this big project, trying to solve issues with the
interface and helping to develop it, to make it user friendly and easy to use
for all users.
During the community bonding period I started to prepare the developing
environement with suggestions from my mentors, I have been remotely meeting them
on RetroShare and I have been successful compiling RetroShare for desktop, and
now I am preparing the toolchain to compile RetroShare for Android, that is not
so easy as it may seems.
The application interface is writted in Qml, a language part of Qt framework,
so my first steps have been prepare Qt Creator IDE to, and to create my own fork
of the Retroshare project [0]. 
The app comunicates with the Retroshare API to get the information, using unix 
sockets, and also with the native Android operating system, using JNI (Java Native
Interface).
After having the toolchain working I’m going to start improving the QML
interface, adding features, improve the integration with Android operating
system, improve usability, and fix a bunch of bugs.
Cheers!

GSoC 2017 – RetroShare via Freifunk

Posted on by scokovski

Hello, my name is Stefan Cokovski and I’m an undergraduate student at the Faculty of Computer Science and Engineering, Saints Cyril and Methodius University of Skopje. My field is Computer Networks Technologies.

Firstly, I would like to thank Google and the team responsible for organizing GSoC. GSoC is a wonderful opportunity for many students all over the world to gain some real experience working on open-source projects, but also to expand their network with new friends and potential colleagues. I would also like to thank Freifunk (for taking many projects related with computer networks under their wing and for also supporting the project RetroShare) and the lead developers (and my mentors) of RetroShare for being here for me during this community bonding period, answering my questions and helping me to improve my ideas. I’m sure they will continue to help me during the later parts of GSoC.

Before I tell you what my project involves, I would like to introduce you to what exactly RetroShare is and maybe convince you to start using it (if you don’t use it already) and possibly join the development process.

RetroShare is a decentralized, private and secure commmunication and sharing platform which provides many interesting features like file sharing, chat, messages, forums and channels. RetroShare is a free and open-source project, completely free of any costs, ads and terms of service. RetroShare is available on several operating systems, including various GNU/Linux distributions, FreeBSD, Microsoft Windows and Mac OS X.

Sounds interesting? Read more.

Continue reading “GSoC 2017 – RetroShare via Freifunk” →

GSoC: Improving nodewatcher data representation capability (update 1)

Posted on by rasovica
¡Hola! 
I am a student of computer science, but most of my knowledge comes from my diy projects. I am a jack of all trades kind of a guy; I have tinkered with low level stuff like add-ons and fpgas,
but I also worked with everything UE4 gaming engine, blender and other high level programs. I like creating visual things such as music visualizations, graphs and other more interactive ways of displaying data. This summer I will help improve the visualization capabilities of nodewatcher.

Continue reading “GSoC: Improving nodewatcher data representation capability (update 1)” →