Tag: yaffmap

Node Database and Map Server Yaffmap: Yet Another Freifunk Map

Posted on by Mario Behling
yaffmap is a project from Dennis Bartsch from Freifunk Berlin, which he started with a friend. It is called yaffmap. It got its name because at the beginning of the project we had many mapservers in Berlin and so it started as yet another approach for a Freifunk map. 
 
yaffmap – screenshot
 
The intention was to make a server that not just produces points and lines (nodes and their links) but to gather all information that might help to understand why a link is as bad as it is. This includes to gather wireless scan results, the effective rate chosen/calculated by the wireless driver to a specific neighbour and so on. Furthermore it had to be independent from the routing-protocol and its daemons (but needs it to gather useful info) and the IP version (or even no IP version for RPs like batman) and had to be able to upload and store data from multiple routing protocols on the same node. In order to sample so much information we went the route of scripting an agent for the map-server which runs on the nodes gathering the information and uploading it through a JSON interface to the server. For link-state-protocols like OLSR we even implemented the upload of the global topology to the server, which gave us some headache. From the beginning on the need for decentralized operation was stressed, so replication between the servers was implemented and any community which wants can have their own map data server. Moreover they cleary wanted the datacollection/storage to be independent from the frontend the map user is presented. In Berlin everytime a new map came along and an old one was gone we saw huge ammounts of data simply disappear. So the server only provides a SOAP interface for UIs and other services to use to get map data. The representation of a node in the database ist best seen on a graphic. Dennis uploaded it to imageshack, see [1]. A little bit of documentation is found under [2], but wasn’t thoroughly updated after we left concept stage.
 
We got to the point where the agent (bunch of shell/AWK) for the nodes runs on openWRT (best with madwifi and ath9k/ath5k) with olsrd (other routing daemons simply need scripts for data gathering, agent is held modular), uploades many useful information to the server (or another if the first does not respond; multiple can be configured) using JSON, where it gets stored correctly into a SQL database and replicated to other servers and provided through a SOAP interface. The agent even is already provided as a package in the respected PBerg Freifunk Firmware. There also exists a ‘proof-of-concept’ implementation for a map frontend, see [3]. The code is hosted on github as seen und [4]. Because of the frontend development got stuck the frontend SOAP interface is not very sophisticated nor much tested as is the frontend itself.
 
The result is a database which can answer more complex question to the mesh engineer or software developer. Maybe you want to know the average effective rate on all 11n interfaces or all 5GHz interfaces? Or you might be interested in the average ETX on different channels in a certain geographic area. Or you want to compare how different routing daemons evaluate the quality of a certain link. Maybe you even want to draw a noise map. Or wouldn’t it be interesting to see the correlation between the effective rate of a wifi link and the metrics which result in the routing daemons? Because the node data includes a ‘misc’ field through which any kind of node id or statistics could be stored/sent this is can even be integrated into existing community portals.
 
 

International Community Map Project and Node Database

Posted on by Mario Behling
One of the things every community would like to see are maps of their network. There are many initiatives and map projects around, but how much more progress could we make, if we work together? This is why a group of free networks contributors started a group to work together on a common project. We started a mailing list to coordinate efforts and welcome everyone who would like to collaborate: https://lists.funkfeuer.at/mailman/listinfo/interop-dev
 
The current goal is to define a basis for a common map. We are looking at different map solutions and try to find out how we can merge the best of each into one. The best approach seems to be a modular system. In the following three great projects of different communities.
 
Nodeshot comes from the community in Italy and is widely being used there (map.ninux.org). The main developer is Federico Capoano. The goal of the Django application is to have “a nice snapshot of your wireless community network”. Nodeshot is a web tool for wireless community network. It allows members to add their node and to share and manage information about their configurations like devices, ip addresses, wireless parameters etc. In this way, newcomers can easily contact/connect with them. Internal scripts will update the topology and retrieve nodes information via snmp, or parsing routing information given by olsr, batman or whatever. It is super-fast, nice and easy to use. It rises from the ashes of WNMap (sourceforge.net/projects/wnmap/), powered by django, released under GPLv3 and tested inside the Ninux wireless community network (wiki.ninux.org). How to install: A basic guide on how to setup nodeshot for your community is available here: wiki.ninux.org/InstallNodeshot
 
Nodewatcher, another Django based application was started in Slowenia by Mitar. Its main goal is the development of an open source network planning, deployment, monitoring and maintanance platform. The project is divided into multiple components:
* node telemetry provider: A simple shell script that is accessible via HTTP interface and is used for node status data acquisition that is performed by the data collection system. Its role is similar to the one of SNMP, however it uses less resources (CPU-wise and, mostly, when it comes to memory and flash space). Our firmware has this preinstalled, others can follow these instructions to install it.
* data collection system : A daemon written in Python that periodically collects data regarding the OLSR topology, active HNAs, node telemetry and performs active reachability tests for visible nodes. Using rrdtool it can then generate graphs that are used by the web interface.
* web interface: A web-based application, written in Python and using the Django framework. It is used by the users to monitor the status of the network and individual nodes and by the node owners to manage their nodes.
* firmware image generator: A daemon that handles per-node configuration and firmware image generation via the OpenWrt buildroot using our custom firmware. It receives requests from the web interface. Live version We use nodewatcher in wlan slovenija network (this is why we are developing it), so you can see a live version of nodewatcher for real deployed network here
 
yaffmap is a project from Dennis Bartsch from Freifunk Berlin. He is working on an implementation of a node database he started with a friend. It is called yaffmap. It got its name because at the beginning of the project we had many mapservers in Berlin and so it started as yet another approach for a Freifunk map. The intention was to make a server that not just produces points and lines (nodes and their links) but to gather all information that might help to understand why a link is as bad as it is. This includes to gather wireless scan results, the effective rate chosen/calculated by the wireless driver to a specific neighbour and so on. Furthermore it had to be independent from the routing-protocol and its daemons (but needs it to gather useful info) and the IP version (or even no IP version for RPs like batman) and had to be able to upload and store data from multiple routing protocols on the same node. In order to sample so much information we went the route of scripting an agent for the map-server which runs on the nodes gathering the information and uploading it through a JSON interface to the server. For link-state-protocols like OLSR we even implemented the upload of the global topology to the server, which gave us some headache. From the beginning on I stressed the need for decentralized operation, so replication between the servers was implemented and any community which wants can have their own map data server.