Here it is my second blog post for the Mid Term Evaluation of GSOC2014.
As I wrote in the previous one, I'm working on Netengine, a Python module to abstract network devices and get information from them.
The work is going very well, I'm learning new things every day with the help of my mentor, Federico Capoano, and I'm very happy with the development.
In this first part of the work we completed as per goals, the SNMP back-end for AirOS and OpenWRT firmwares.
The most difficult part of this first part was to work with SNMP (Simple Network Management Protocol) because I had never worked with it, so I had to learn it's basics and how it works, in particular it's way of retrieving info from devices.
It uses different codes (MIB), everyone of it gives access to different information of the device (e.g device name, addresses, interfaces); so before starting to write I had to look for the correct MIBs to query.
Now we are focusing on the ssh.OpenWRT back-end ready to switch the next one on the list once completed.
I'm definitely very happy with how the work is going, with the communications I'm having with my mentor and with all coding practices I'm learning from him.
The program gave me not only the possibility to improve my skills but also to meet new people very experienced on the field.
The next step is to start coding on the new back-end, probably an HTTP back-end for AirOS, to complete the program in time.
Photo taken from https://twitter.com/christianheise/status/472746947569520641
From the 29th May to the 1st of June we were with Andi and Bernd from Weimarnetz at the wunderful c-base Raumstation. We visited the Wireless Community Weekend. It was my first experience of this kind of community event and I enjoyed it very much.
Beer and Bratwurst did harmonize quite well with technical talks about OpenWrt and the Freifunk Community. I was especially surprised how diverse and open the community is and how enthusiastic everyone involved was.
Andi and Monic talked about the progress on the Freifunk API and presented their work. At the end of their talk I had the chance to present my work on the query client for the API. Here are the slides.
Shortly after the talk Jürgen Neumann from Freifunk Berlin came to me and introduced me to DeepaMehta. My original plan was to use something like NodeJS for the backend the storage of the API data but DeepaMehta looked promising and offers unique features I didn't even thought off.
So after talking with Andi about it we decided to use DeepaMehta as the foundation and storage tool for the API data. A seperate blogpost for the GSoC midterm evaluation will outline my work in this direction.
Overall it was a very exciting weekend for me at #ffwcw 2014.
The first weeks of my Google Summer of Code project were a little complicated, as I still had exams at university, and I was not really aware of what mesh networks were about. I also needed a little time to get to know network and routing better, both pratically and theorotically.
At the beginning, in order to understand quagga and babeld, Iread a lot of documentation on the routing topic including theoretical papers and some RFCs, while also browsing the code of both babeld and quagga.
On the other hand, I have been able to experiment mesh routing with the mesh network available at university. In order to use that network from home, and being able to test my programs at all time, I also established a VPN connection between university and my home computer. By doing so, I can connect to the university's babel network at any time. I have also been able to understand the functionning of quagga and zebra and to install source-sensitive static routes on a mesh network.
After having spent much time reading the codes of babel, quagga, and babel in quagga, I achieved to use the zebra's API in babeld and began to add support for source-sensitive routing in babeld. Currently, my code runs and segfaults proudly ! I hope to see the first results of source-sensitive routing with my version of babel, in the worst case, at the end of the week.
At first, my goals were not really clear, but now, I have precise objectives on the short, middle, and long term. In brief, my short term objectives would be to get a source-sensitive routing Babel running by the end of the week. After getting a working version of Babel source sensitive, I will implement the same work in RIPng. RIPng is a quite simple protocol and Juliusz and Matthieu told
me it would be a good idea to offer it source-sensitive routing. And finally, after everything will be tested and running fine, I will be implementing the source-sensitive commands in Babel. Then, I will complete the documentation about my work. And in the end, the ultimate goal would be to be included in the official repository of Quagga.
If you want more details about the work I did, you can read my blog here : http://ariane.wifi.pps.univ-paris-diderot.fr/~olden/. I posted an entry every week to keep you informed of the progress on the short term.
In the last post i wrote about the decentralized structure of Retroshare. I said: similar features to Facebook are also possible in a decentralized manner. Now let's have a closer look at those features.
Communication tools have the purpose to take a piece of information and transmit it. The question is who should receive it, when will it be received and to which next hop should the information be send.
Very simple forms of communcation are one to one messages. We have a clearly defined receiver which can be adresse by a RSA public key. This is very simple if we are directly connected to the receiver. If the receiver is not a direct neigbor, then Retroshare uses it's distant tunnels. Current distant chat and distant messages only work when sender and receiver are online at the same time. Retroshare v0.6 will cache messages on neighbor nodes to deliver them when the receiver is online.
Retroshare can also do many to many communication. The Chatlobbies are for real time communication. If you don't receive the message a few minutes after it was send you will never receive it. Messages are send to all neighbors in the chatlobby. RS keeps messages for a few minutes to prevent echos. Forums and channels offer persistant messages. This means messages are kept for a year in forums and a month in channels. The messages are not targeted at a person, but they belong to a context. This context is a chatlobby or a forum/channel. Users have to subscribe to the context.
Every message is bound to the context. It appears only at one place. In Retroshare forums and channels the context is a topic like "Developer's Discussion" or "RetroShare Windows builds by Thunder". In the social network plugin the posts are decoupled from a context. The context only stores a reference to the post. This allows posts to appear at different places. For example: a post can appear on the walls of user A and user B. And if i see the post and like it, i can forward it to my wall. Forwarding of specific posts only, increases the quality of the received content.
With forums and channels you first have to subscribe to them. Retroshare will then download the posts in it. The social network plugin automatically downloads the posts your friends like. And if your friends like it, then the probability is high that you also like it.
While forums are about a topic, a social network wall is about the user. It is a personal place where you can leave message for your friends, and your friends can leave messages for you. Still your are not limited to own content. If you want to share existing posts with your friends, just put it onto your wall.
Scope of the GSoC project
The goal of the GSoC project is to make a basic wall service. This includes the backend for message processing and the user interface. Chat and private messages are not planned for GSoC. But this is something that is very easy to add later.
Internally the social network plugin is based on the new Retroshare General Exchange System(GXS). GXS has two basic data structures: gxs-groups and gxs-messages. A gxs-message belongs to a gxs-group. If a gxs-group is marked as subscribed, then all messages in this group are downloaded. In the gxs-forums this mechanism is clearly visible to the user: gxs-groups are downloaded automatically, but the user has to subscribe to download the contained gxs-messages. The posts in the social network plugin should be independent. This requires that every root-post has its own gxs-group. The wall services automatically subscribes to interesting gxs-groups. Gxs-groups can be interesting because:
- the posts appears on a wall of a friend
- the user follows the author of the post or the user is friend with the author
- a friend made a comment on this post
So the interesting content is not defined by a forum title, but by the way friends interact with it.
A quick update on what is happening regarding my GSoC project with bringing nodewatcher v3 platform closer to reality. Version 2 of nodewatcher used its own simple key-value format and a bunch of shell scripts to provide monitoring data. In order to bring this into the modern era where JSON and ubus are available as compact libraries on OpenWrt by default, I have in the last few days created a new modular OpenWrt monitoring agent that can run on nodes and periodically obtain data from various sources (directly from procfs, from uci, from netifd via ubus, from nl80211 netlink API via OpenWrt's libiwinfo, etc.).
The daemon is implemented in C and different data sources are implemented as loadable shared object modules, enabling simple extensibility. Nodewatcher agent then provides all of the collected data in two ways: a) it can directly output data to a JSON file that can be served via HTTP; and b) it also provides an ubus object called nodewatcher.agent that exposes a get_data method, so other applications can obtain the same structured data as an ubus blobmsg. The nodewatcher monitoring agent could perhaps also be reused by the proposed Freifunk Monitoring and Administration Panel.
The nodewatcher-agent repository is hosted on GitHub with a README file providing a quick description of the used format, the ubus API and the currently implemented modules:
I have also packaged the agent for OpenWrt so it can be installed together with its various data source modules via opkg. The packages are available in the nodewatcher firmware package repository:
The agent and its packages should be considered alpha and the schema is still subject to change.