The Turnantenna – Final evaluation update

We are at the end of the journey. Today is the last day of the 2018 version of the Google Summer of Code.

So, here what I have done during this month of hard (and hot) work!

States Machine

The SM presented in the previous article has evolved to a newer and more complete version. The whole machine is defined through the following states and transitions:

# In controller.py

class Controller(object):
    states = ["INIT", "STILL", "ERROR", "MOVING"]
    transitions = [
        {"trigger": "api_config", "source": "INIT", "dest": "STILL", "before": "setup_environment"},
        {"trigger": "api_init", "source": "STILL", "dest": "INIT", "after": "api_config"},
        {"trigger": "api_move", "source": "STILL", "dest": "MOVING", "conditions": "correct_inputs",
         "after": "engine_move"},
        {"trigger": "api_move", "source": "STILL", "dest": "ERROR", "unless": "correct_inputs",
         "after": "handle_error"},
        {"trigger": "api_error", "source": "STILL", "dest": "ERROR", "after": "handle_error"},
        {"trigger": "engine_reached_destination", "source": "MOVING", "dest": "STILL",
         "before": "check_position"},
        {"trigger": "engine_fail", "source": "MOVING", "dest": "ERROR", "after": "handle_error"},
        {"trigger": "error_solved", "source": "ERROR", "dest": "STILL", "after": "tell_position"},
        {"trigger": "error_unsolved", "source": "ERROR", "dest": "INIT", "after": ["reconfig", "tell_position"]}
    ]

There are not many differences with the older graph but, behind the appearance, there is a lot of work. Now every arrow correspond to a series of defined actions, and the scheme was implemented as a real working program.

The structure of the Turnantenna’s brain

During the last week I worked on the refactoring of all the work done until that time. The final code is available in the new dedicated “refactor” branch on GitHub.

The States Machine above is implemented in the main process, which is able to communicate with 2 other processes: the engine driver and the RESTful server.

# In turnantenna.py

from multiprocessing import Process, Queue
from controller import Controller      # import the states machine structure
from stepmotor import engine_main      # import the engine process
from api import run                    # import the api process

def main():
    engine_q = Queue()
    api_q = Queue()
    api_reader_p = Process(target=run, args=(api_q, ))
    engine_p = Process(target=engine_main, args=(engine_q, ))
    controller = Controller(api_q, engine_q)                  # start the SM

    api_reader_p.start()                                      # start the api process
    engine_p.start()                                          # start the engine process
    controller.api_config()

The processes communicate with each other through messages in the queues. Messages are json, and have the following format:

{
    'id': '1',
    'dest': 'controller',
    'command': 'move',
    'parameter': angle
}

The “id” key is needed in order to control more than one engine, this is useful for future upgrades. “dest” specify the process that should read the message, and avoid wrong deliveries. “command” is the central content of the message, while “parameter” contains detailed (optional) informations.

Processes are infinite loops, where the queues are checked continuously. An example of this loop is:

# In api.py

from queue import Empty

while True:
    try:
        msg = queue.get(block=False)
        if msg["dest"] != "api":
            queue.put(msg)       # send back the message
            msg = None
    except Empty:
        msg = None

    if msg and msg["id"] == "1":
        command = msg["command"]
        parameter = msg["parameter"]
        if command == "known_command":
            # do something

API

In order to interact with the turnantenna, I defined 3 methods: get_position(), init_engine() and move().

It is possible to call them through an HTTP request. A json needs to be attached to the request in order to make things work. In fact, APIs need some critical data: e.g. the id of the specific engine targeted, or a valid angle value to move the engine of that amount. If the request come without a json, or with a wrong one, the RESTful service respond with an error 400.

Here an example of input controls:

import requests

if not request.json or not 'id' in request.json:
    abort(400)
id = request.json['id']
if id != '1':            # still mono-engine
    abort(404)

For the moment the system works with only one engine, but in the future it will be very simple to handle more motors

...
# if id != '1':
if id not in ('1', '2')
    abort(404)
...

Final results

In these moths we started from an idea and a basic implementation, and we build-up a complete system ready to be tested. It is possible to see the Turnantenna logic run cloning the Turnantenna code from GitHub from the link Musuuu/punter_node_driver/tree/refactor.
Following the instructions in the readme file, you can run the turnantenna.py file and observe how it reacts to the HTTP requests made with curl.
The full documentation of the project could be found at turnantenna.readthedocs.io.

We are proud of the work done, and we’re ready to implement the whole system onto the hardware and make the Turnantenna turn!

The Turnantenna – Second evaluation update

Time is passing, and work is proceeding.

Last month I reported a problem concerning speed of our beloved Turnantenna: the acceleration was not constant during movement of the stepper engine, as I wanted. The error was caused by implementation of a bad algorithm. A constant acceleration is important to provide a smoother movement, and is needed to reduce the load on the engine. Force is equal to mass times the acceleration; if the acceleration is constant, so is the force; but if the acceleration grows, the stepper’s force grows as well, as long as it can keep up. Uncontrollable acceleration lead to unpredictable forces (or better, toques).

To understand the issue, a brief summary should be given: the way to control the stepper’s speed consists of changing the time between two consecutive steps. The shorter the time, the faster the movement. The previous (and wrong) algorithm, is documented in the older post. It wasn’t a good way to control a torque-limited engine because, as said before, the acceleration was not constant. In the previous algorithm, speed was taken like this:

vn = vn-1 + const

namely, at time tn the speed was a fixed amount more than tn-1.
Time between two steps was

dt = (n – n-1)/vn = 1 / vn

It may appear correct, but the resulting graph was the following:

As can be seen, the speed is not linear. This mean that the acceleration is not constant, but increasing.

I found a solution thanks to a document written by Atmel Corporation. It made me think about the relationships between speed (v), space (s), time (t) and acceleration (a) that comes from physics laws:

s = a ½  t² + v₀ * t + s₀

this equation is always true, when accelerating, when the speed is constant, and even when decelerating. Quantities change inside the formula, but it always remain true.

Now, to keep it simple, let’s consider the first phase: the acceleration. A the beginning of its movement, the engine is still (v0 = 0), and it starts without having already done one single step (s0 = 0). The resulting equation is evaluated at v₀ =0  and s₀= 0:

s = a ½  t² + 0 * t + 0
s = a ½  t²

Now, let’s think about what is known: the acceleration a, that it is constant (because I want it so), s and t; s is the number of steps already done at time t. If I know how many steps -s- I have to do I can find how much time I have to wait –t-, and vice versa.

To find the time between two steps (the step #n and the step #n+1) the formula is:

s = a ½  t²
==>  t = sqrt(2 * s / a)

# at the step number ‘n’
tn = sqrt(2 * n / a)

# time between step ‘n’ and ‘n+1’
dt = tn-1 – tn = sqrt(2 / a) * (sqrt(n+1)-sqrt(n))

Using this calculation, acceleration is constant, and speed increases linearly, as it can be seen in the graph below:

AAAAAH.. a perfect blue line! 😀

Problem: Solved!

Working on tests

Now, more progresses have been done in tests. For those who don’t knows, I’ve started my programming adventure with with this project. Everything for me is anexciting discovery, and during this month I learned and implemented the “argparse” and “logging” libraries. Now it is possible to execute the tests with three verbose levels: the first is silent, the second shows debugging informations and the third shows the info level.
It could appear trivial, but I’d never done it before, and now tests are smarter!

It is not all: I reviewed all the tests, fixed problems and improved their reliability. They’re still not perfect, but I’m working on them daily to get details right.

Fly across borders

It was time to go outside the boundaries, and to think about an interface that bring into communication the web interface and the driver. This is what I’m working on in these days.

To achieve that goal, the problem has to be studied starting from an high level. The main process, which is constantly running, float between a small number of determined states: initialising, still, moving and error handling. Together with the Ninux Florence developers community I built the following state machine graph:

This was realized with the GraphMachine module of the “transmissions” library. Now I’m working on the full representation of this map in code lines. But there is something more.. In fact, at this point, multiprocessing became necessary to provide a safe environment: when the engine is in the MOVING state, for example, and a new command is sent to make a new different rotation, the main process should have the possibility to manage simultaneously the ongoing movement and newer requests.

That’s why we choose to keep the main process always active and make it decide when to run the movement procedure in a dedicated process, like a traffic light.

Turnantenna.me

The greatest effort, this month, was done writing down a full, detailed documentation of the project. 80 pages on what is the Turnantenna, how it works, and when and why to use it.

Many people expressed their interest in the project, someone has offered to support us but, without a complete documentation available, it is difficult to provide a starting point.

The whole doc will be soon available, and this post will be updated with the dedicated link. So, if you are interested in the project, let us know! For the moment, GitHub repository is available here.

See you next month!

— UPDATE —

The link of the full documentation of the project is provided below.
turnantenna.readthedocs.io

Implementing Pop-Routing in OSPF – July Updates

In the last months some issues emerged with the original plan of my project.
After implementing topology export in NetJSON format for BIRD, I started investigating how to modify the timers of OSPF, and finally found that OSPF has a limitation regarding the timers, which states that all the routers on the same network must have the same timers. This is surprising, since the RFC supports specifying the timer in the messages, and thus it would be trivial to have nodes with differentiated timers (as OLSR and other protocols do, for instance). Actually somewhere else in the RFC it is stated that even if the timers could be diversified, they must be used as a network-wide parameter, otherwise routers wont pair as neighbor. This issue is not easy to solve, and would break the compatibility with other OSPF implementations, thus, together with my mentor we decided to change the project on something that we could accomplish in the remaining time and that was still helpful for Wireless Communities.

Since many WCNs in Europe are still using OLSRv1 we’ve decided to implement PopRouting on it. The NetJSON plugin[1] is already available so we only had to implement the Timer’s plugin and the PRINCE plugin to communicate with it.

During this mont I’ve implemented and alpha version of the Pop-Routing plugin[2] for OLSRd. This plugin is quite simple, it listens to a specified port and it parse two commands : “/HelloTimer=xx.yy” and “/TcTimer=xx.yy”, where xx.yy are floating point numbers.

I’ve also implemented the Prince plugin[3] that communicate with OLSRd. It fetches the topology from the NetJSON plugin and it pushes the updated timers to the PopRouting Plugin.

The next month I am going to fix all the bugs I found, document the plugins and test Pop-Routing with NEPA test.

Despite those problems the implementation of NetJSON for OSPF is working and we hope it will be merged in the BIRD codebase soon.

Cheers, Gabriel

[1]: https://github.com/OLSR/olsrd/tree/master/lib/netjson
[2]: https://github.com/AdvancedNetworkingSystems/olsrd/tree/poprouting/lib/poprouting
[3]: https://github.com/AdvancedNetworkingSystems/poprouting/blob/refactor_ospf/prince/src/olsr/olsr.c

Implementing Pop-Routing in OSPF – June Updates

This is a continuation of the previous post [1].

During this month I have implemented the NetJSON plugin for BIRD. It exposes the topology of an OSPF Area using the network-graph format and thus allows Prince to fetch the topology and calculate the timer’s value.
I deployed a small testbed to debug my code using the network emulator called CORE [2]
Here you can see the testbed:

I’m currently working on this repository [3] and I’m looking forward to send a PR to BIRD.
I defined a new command in the bird’s cli: “show ospf topology netjson”. It returns a network-graph output that can be used by prince or by any other NetJSON[4] compatible software.
Here you can see the topology of the testbed using d3.js [5][6].

In this next coding period I will implement a plugin for Prince that interacts with BIRD. Unfortunately it uses a UNIX Domain socket instead of a network socket, so I’ll need to code the communication routines from scratch.

Cheers, Gabriel

[1]: https://blog.freifunk.net/2017/05/30/implementing-pop-routing-ospf
[2]: https://www.nrl.navy.mil/itd/ncs/products/core
[3]: https://github.com/AdvancedNetworkingSystems/bird/tree/origin/int-new
[4]: http://netjson.org
[5]: http://ninux-graph.netjson.org/topology/49aecbcf-a639-47a7-9f58-e39de5d57161/
[6]: https://github.com/netjson/django-netjsongraph

Implementing Pop-Routing in OSPF

Hello everyone.

I’m Gabriele Gemmi, you may remeber me for… Implementing Pop-Routing[1]
This is the second time I participate in GSoC and first of all I’d like to thanks the organization for giving me this opportunity.
Last year I implemented PR for OLSR2. The daemon, called Prince [2], is now available in the LEDE and the OpenWRT feeds.

What is Pop-Routing

PR is an algorithm that calculate the betwenness centrality [3] of every nodes of a network and then uses this values to calculate the optimal message timers for the routing protocol on each node. In this way a central node will send messages more frequently and an outer one less frequently.
At the end the overall overhead of the network doesn’t change, but the convergence gets faster.

Objectives

My project focuses on extending Prince functionalities to use Pop-Routing with OSPF. I decided to work with BIRD, since it’s written in C and it’s already available for OpenWRT/LEDE
In order to do this I need to develop 2 components:
— A plugin for BIRD that expose the OSPF topology in NetJSON and allows to update the timers
— A plugin for Prince that communicate with the BIRD plugin

I already started developing the former [4], and I’m looking forward to implement the latter.
I’ll keep reporting my updates here, so stay tuned if you wanna hear more.

Cheers, Gabriele

[1]: https://blog.freifunk.net/2016/05/23/implementing-poprouting/
[2]: https://github.com/AdvancedNetworkingSystems/poprouting/
[3]: https://en.wikipedia.org/wiki/Betweenness_centrality
[4]: https://github.com/AdvancedNetworkingSystems/bird

Implementing Pop-Routing – Midterm Updates

 

Hi Everyone!

Today has started the midterm evaluation, the deadline Is next Monday, so I have to show the work I have done ‘till now. It can be resumed in the following parts:


1) Refactoring of graph-parser and C Bindings

During the community bonding period I started working on the code of Quynh Nguyen’s M.Sc. Thesis. She wrote a C++ program capable of calculating the BC of every node of a topology [1]. I re-factored the code, and now it is a C/C++ shared Library [2]. I’ve also applied some OOP principles (Single responsibility and inheritance) and unit tests to make it more maintainable.

The interface of the library Is well defined and it can be re-used to implement another library to perform the same tasks (parsing the json and calculating the BC).


2)Prince Basic functionalities

After I completed the library a started working on the main part of the project. the daemon. We decided to call it Prince in memory of the Popstar.

This daemon connect to the routing protocol using the specific plugin (see below), calculate the BC using graph-parser, computes the timers and then it push them back using again the specific plugin. With this architecture it can be used with any routing protocol.I wrote the specific plugin for OONF and OLSRd. At the moment it has been tested with both, but I need to write a plugin for OLSRd to change the timers at runtime. For OONF I used the RemoteControl Plugin.

With these feature Prince is capable of pulling the topology, calculate the BC and Timers and push them back to the routing protocol daemon.

 

3) Additional Features: Configuration file, Dynamic plugins,

I wrote a very simple reader for a configuration file. Using the configuration file the user can specify: routing protocol host and port, routing protocol (olsr/oonf), heuristic, (un)weighted graphs.

As you can see from this Issue [3], I’m going to use INI instead of this home-made format.

As I said before I moved to a specific plugin all the protocol specific methods (pulling the topology and pushing the timers), to keep the daemon light I decided to load this plugin dynamically at runtime. So if you specify “olsr” in the configuration file just the OLSRd specific plugin will be loaded.

 

 

At the moment I consider this an “alpha” version of Prince. In the next 2 months I’ll be working on it to make it stable and well tested. The next steps will be:

 

  • Close all the Issues [4]
  • Write tests and documentation for Prince.
  • Write a plugin for OLSRd

 

Cheers, Gabriel

 

[1]: https://ans.disi.unitn.it/redmine/projects/quynhnguyen-ms

[2]: https://github.com/gabri94/poprouting/tree/master/graph-parser

[3]: https://github.com/gabri94/poprouting/issues/4

[4]: https://github.com/gabri94/poprouting/issues

Implementing Pop-Routing

Hi everyone!

I am Gabriele from the Ninux community. I am participating in GSoC 2016 for the first time and I am very glad I have been accepted as a Student for Freifunk. I am from Florence, Italy. Here I’m studying Computer Science, soon I will graduate and I hope to use the results of this project to write my bachelor thesis.

Four years ago, with others community networks’ enthusiasts we have funded Ninux Firenze[1], the fist Wireless Community Network in Florence where I had the chance to learn how these networks work and to meet many others people interested in this field. The network is constantly growing, and now it counts almost 20 nodes. In May ’14 I have been for the first time to Wireless Battle of the Mesh in Leipzig where I met the Freifunk community. For this GSoC I will work on a project called PopRouting[2]:

OONF (OLSRv2) is a link state routing protocol. It works sending periodical messages to his neighbors with the aim of transmitting information about topology changes. With these information each node of the network is able to calculate the paths to reach any other destination. These messages are periodically generated, based on the configuration parameter that regulates the sending interval. A short period will make the network react rapidly but it will also cause a large overhead due to control messages. Pop Routing is a recent technique that takes advantage of the knowledge of the network topology to find the optimal value for the OONF’s timers. Using Pop Routing every node computes the “betweenness centrality” of every other node and uses it to calculate the optimal trade-off between convergence and overhead for its timers. The algorithm has been developed at the UniTN and the algorithm to compute the BC in C++ is available as free software. My goal is to code a daemon (in C) that is able to calculate autonomously the BC of the network and push it to OONF using the telnet plugin.

In this month of community bonding I have been to Wireless Battle of the Mesh v9 in Oporto(PT). There I met the OONF developers and we discussed how to implement this inside OONF. I also gave a presentation on the project. After the Battlemesh I started working on the algorithm developed by UniTN and I made a C/C++ library out of it [3].

Today I will start coding for the GSoC, stay tuned and I will give you more updates soon.

 

Gabriel

 

[1]: http://firenze.ninux.org/

[2]: https://ans.disi.unitn.it/users/maccari/assets/files/bibliography/INFOCOM2016.pdf

[3]: https://github.com/gabri94/poprouting/tree/master/graph-parser

Linus Torvalds and Dirk Hohndel meetup with Freifunk Community at Google Reunion

Freifunk attendees had the chance to discuss Community Networks with Linus Torvalds and Dirk Hohndel from Intel at the Google Mentor Summit. Linus said, it was impressive to see the growth of community networks around the world and it is exciting to see so many people working on Linux for embedded devices.

Linus Torvalds, Mario Behling, Federico Capoano, Freifunk, Google Summer of Code

Netengine Google Sumer of Code Project

Hi everyone, I’m Alessandro Bucciarelli and I am participating for the first time to Google Summer of Code.
I chose to apply to work on Netengine, a project by Freifunk/Ninux. Netengine is a Python abstraction layer thought to retrieve informations about network configurations, and not only, from multiple couples of network protocols/device firmware.

Actually the main network protocols we are working on are: SNMP, SSH; with HTTP which is an idea for the immediate future.
By the firmware side there are AirOS and OpenWRT which are the most used firmwares among network devices (antennas and other) deployed inside the Ninux network.

Many of the readers, if experienced in the network field, will agree with me in saying that the retrieval of network informations (e.g IP addresses of the interface/s, MAC addresses, routing configurations) is vital.

This aspect is more than vital when you are interacting with remote devices, geographically widespread and sometimes accessible by only unskilled persons, to have a timely diagnostics of the  deployed hardware.

The module we are developing tries to solve, and I am sure it will be so, the problem of having informations from the devices REMOTELY, without any kind of further configurations and without any kind of physical interaction with the device.

For further details and code please visit https://github.com/ninuxorg/netengine or email us at ninux-dev@ml.ninux.org

Italienische Behörden fordern in Erdbebengebieten Bürger zur Öffnung ihres WLAN auf

Wie lebensrettend offene Infrastrukturen sein können, zeigt sich vor allem in Katastrophenszenarien, wie zum Beispiel momentan im Erdbebengebiet in Norditalien. Jetzt berichtet auch Spiegel Online über einen Aufruf der Behörden an die Bürger ihren WLAN-Zugang zum Internet zu öffnen, um die Kommunikation von Rettungskräften zu erleichtern.

Italiens Norden wird von Erdbeben erschüttert, die Rettungskräfte arbeiten pausenlos. Die Kommunikation ist schwierig, weil Telefon- und Handynetze teilweise zusammengebrochen sind. Nun rufen Städte und Gemeinden der betroffenen Region die Bürger auf, ihre heimischen W-Lan-Anschlüsse in freie Hotspots zu verwandeln und dafür den Passwortschutz kurzfristig aufzuheben. (http://www.spiegel.de/netzwelt/web/.…)

Auf Republicca können die Italiener nun nachlesen, wie der Zugang geöffnet wird:

PER CONSENTIRE a tutti coloro che non riescono a comunicare via cellulare di collegarsi ad internet, molti comuni invitano i cittadini dei paesi colpiti dal terremoto di oggi ad aprire la propria rete wi-fi domestica. (http://www.repubblica.it/cronaca/2012/05/29/news/i...)

Der Kommentar von Reto Mantz dazu:

Der Fall wirft ein deutliches Schlaglicht auf die Relevanz des Zugangs zum Internet und allgemein des Zugangs zu Kommunikationssystemen. Und letztlich ist dieser Punkt auch in rechtlicher Hinsicht beachtlich: Wenn eine Kommunikationsstruktur zur Begehung von Rechtsverletzungen genutzt wird, greift potentiell das deutsche Konstrukt der Störerhaftung: Der Anschlussinhaber soll als Mitwirkender an der Rechtsverletzung des (möglicherweise unbekannten) Dritten wenigstens auf zukünftige Unterlassung haften. Nun zeigt das Beispiel der italienischen Städte und Gemeinden, dass ein offenes WLAN nicht Gefahrenquelle ist …, sondern eine wichtige gesellschaftliche Funktion erfüllt. … Auch wenn der Aufruf der italienischen Städte nur der vorübergehenden Öffnung von WLANs dienen soll, zeigt er doch, wie wichtig heutzutage der freie Zugang zu Kommunikationsstruktur ist, nicht nur zur Überbrückung des sog. Digital Divide. (http://www.retosphere.de/offenenetze/2012/06/02/funktion-und-bedeutung-des-wlan-acess…)

Zu hoffen bleibt, dass die Erkenntnis, wie wichtig und lebensrettend offene Netze sein können sich auch längerfristig in Italien durchsetzt. Bei vielen Bürgern ist dies bereits vor Langem angekommen. Die Bürgernetze von Ninux.org in Italien wachsen beständig. Wann wird sich diese Erkenntniss auch in der Politik in Deutschland durchsetzen und wann werden die gesetzlichen Beschränkungen und Abmahnfallen für Betreiber freier Netze endlich abgebaut?