Routing and WiFi experimentation

Posted on by spacekookie

The beginning of my work period was pretty busy, not always with Summer of Code things. My mentor math and I had already talked about a lot of the things that needed to happen in order to move qaul.net away from an OLSR based routing protocol and make it extendable as well.

As previously hinted, we are using Rust for the protocol implementation, allowing for easy integrating into the existing C code as well as giving us the option to bit-by-bit rewrite the entire software in Rust, a much more modern and forgiving language. The first thing I tackled was to design a common API for the qaul.net library (libqaul) to use to talk to any networking backend. The routing code holds the state of the network and allows the sending of direct and flooded messages into a network (regardless of implementation under the hood). But to do that we also had to define some common characteristics for nodes and messages.

In the end, a lot of the work was sitting down, going through old notes and determining what our protocol was supposed to do. We looked at existing protocols a lot, thinking about extendability and backwards compatibility. The protocol itself will be binary encoded, although not yet sure which format. There is msgpack, cpnproto/ protobuf as well as some Rust specifics (Rust Object Notation to mention one) to look at. But that shouldn’t actually matter for now. All the versioning and extend-ability are being done in the struct level of the protocol, meaning that we could even switch binary encoding half way through. Keeping the encoding and decoding written in Rust, this is actually incredibly easy with the `From` traits. But I digress…

The protocl we ended up designing can handle any type that already exists in qaul.net, as well as allows for custom user extentions – messages that have a type field and a random binary message payload which allows plugins on both sides to interact with it.

 

So…so far the routing core isn’t doing much routing. But that’s okay, that comes later 🙂 With the networking API in place, we actually have something what we’ve wanted for the last 2 years: a hardware abstraction over any networking backend. The API is implemented in Rust as a trait (think like a Java interface), which makes implementations and even implementation specific code very easy.

The next thing on our todo list is working out how WiFi direct behaves. This is kinda disconnected from the rest of the project, but it’s something that has to happen. For this purpose I’ve written a small demo app (still WIP at the time of this writing), which will let us explore the way that WiFi direct works, how to build mesh groups, etc. These experiments are still ongoing, and we hope to have something to show until the end of the week. I will probably publish a small article on my blog about it – check it out (if you’re reading this in the future 😉 )

All in all, the amount of code written in the first section of GSoC2018 is medium. We have however answered a lot of open questions, have a good plan on how to continue and hope to have more to show off by the time of the next evaluation.

If you’re curious about the progress being made, check out the github repository.

 

Until next time,
Katharina

nodewatcher: Build system rework and package upstreaming – First update

Posted on by Robert Marko

Since last update I spent most of time on fully understanding current build system and nodewatcher internals.

Build system

During the time spent into looking how the whole system works I belive that I was able to figure out what every step and script does in the current build system.
During that I found a lot of relatively simple improvements that can really reduce custom stuff that we have. Most of it was added 3-5 years ago when OpenWrt wasnt really in the state it is now. Custom mirror for source was added,which is now useless as it was not updated and on some really old custom packages it is really slow. Also,building all packages that are added to the package list during build time is not really efficient as during its building OpenWrt default feeds are replaced with our custom package feeds.
This causes users to be stuck on really old and quite limited number of packages. This will be revorked to only replace the target feed as it contains all of the kernel mods that are tied to specific kernel version.
Other packages have no such requirments and versions in upstream can be used.
Also,wget was used during the build time to pull dependecies instead of curl which is recommended tool. Wget is fine for simple downloads but a lof of packages are pulled from behind CDNs and with lot of redirects from mirrors such as Sourcefourge,those curl can handle fine but wget cant.

I have started dropping unused and old packages as well as those that had custom patches that were upstreamed a long time ago.
Stuff like iwinfo from 2015 and old curl.

Also,I have started to move both the buildsystem docker image and runtime docker image from Ubuntu 14.04 to 18.04 Bionic.
This does not yet fully work as Imagebuilder does not detect GCC and Ncurses in the runtime image as working despite the fact that GCC works fine. This will be hard to diagnose,but I feel that it is quite simple.

Custom Wlan Slovenija packages are being prepared for upstreaming.

This is all for now,next two weeks should be bring solutions to most issues.

Robert Marko

VRConfig Update

Posted on by Tobias Ilte

Hi,

I have some quick updates about VRConfig for you.
Short recap: VRConfig aims to introduce a graphical configuration mode for OpenWrt’s Webinterface LuCI.
For that need to collect pictures of the backside of all supported routers. The idea is to do this in a crowdsourcing manner. The community can submit pictures of their routers together with a metadata file which contains the locations of the components on the picture.

I spent the last weeks developing a web application to provide the annotation functionality of the router components.
A working prototype is now ready and can be tested at the following URL: https://vrconfig.gitlab.io/annotator/
Source code: https://gitlab.com/vrconfig/annotator

The annotator produces a JSON file which in turn can be parsed by the LuCI Application to provide the graphical configuration mode.

The LuCI application is being developed right now and will be provided shorty under the following URL: https://gitlab.com/vrconfig/luci-app-vrconfig

More info about that in the next blog post.

GSoC 2018 – DAWN a decentralized WiFi controller (1st update)

Posted on by nick.power

DAWN is using the ubus bindings of the hostapd. Ubus is a messaging system in OpenWrt to which processes can subscribe and publish information or services. The hostapd ubus bindings allow to collect probe-, auth- and assoc-requests. Furthermore, it is possible to deny these requests. Additionally, we can gather client information or deauthenticate clients.
I made my life easy by just extending the hostapd ubus calls with all information I need. I wanted to get these changes upstream but some of my pull request were rejected. I added the bssid, the essid and stuff like this to the hostapd notifications. The pull requests were rejected because I can gather these information through the netlink socket. The ubus bindings of the hostapd should only spread information that can not be gathered in other ways. Now I only have two pull requests left:

Already accepted pull requests:

Added channel survey data in libiwinfo

I had to made the decision, if I want to directly use nl80211 or some library. I already used libiwinfo to contentiously update the rssi of the connected clients. Furthermore, the libiwinfo library is often installed on OpenWrt devices. With the libiwinfo it was possible to gather the essid and the bssid of the WiFi interface. The only information I missed is the channel utilization. The channel utilization is a value between 0 and 255. It is a measure how much a channel is used and what capacity is left.
The channel utilization can be calculated:

Unfortunately, the needed information is not contained in the libiwinfo. So I extended the lib by the necessary information: https://github.com/PolynomialDivision/iwinfo/tree/feature/channel_util
There is some weird behavior of the ath10k driver that I tried to debug. The ath9k driver is working very smooth. If I try to obtain channel survey data without waiting a short time, the survey results become 0. Just waiting between 2 calls fixes the problem.
When I had to figure out how to contribute to the OpenWrt projects. (https://git.openwrt.org/project/iwinfo.git) This can be done via the mailing list. There is a nice tutorial how to send patches using git (https://burzalodowa.wordpress.com/2013/10/05/how-to-send-patches-with-git-send-email/) I’m still waiting that the patch will be merged.

Now I can calculate the channel utilization. Instead of always updating this value, the channel utilization should be averaged. (channel utilization value can be very dynamic)

That’s it. Now I had to rewrite the daemon to gather the informations from the libiwinfo.

Bootstrapping

I want to implement bootstrapping. If a router joins the decentralized controller, it should automatically get the configuration from one router of the decentralized group. Different solutions are possible. I could use scp, rsync to get a configuration from another node. I wanted a different solution. With uci (Unified Configuration Interface) you can configure daemons. I use uci to read my configuration into the daemon. My idea was to send the daemon configuration via the network as a string and use uci to configure the daemon configurations file. Unfortunately, I had some troubles with the uci lib. This approach is not finished.

Lesson Learned – Use calloc instead of malloc!

I spent a lot of time trying to fix some stupid mistake.
The ubus c-library has a function called ubus_add_subscriber which expects a ubus_subscriber. Everything was fine in my old implementation because I used a global variable. Now I wanted to add more subscriber using an array of pointers. What I did was:
struct ubus_subscriber *sub = malloc(sizeof(struct ubus_subscriber));
ubus_add_subscriber(ctx, sub);

This crashed all the time and I was very confused. Finally, a friend of mine said that I should try calloc. It worked! The function ubus_add_subscriber goes through the existing pointers in this struct if they are not null!
Lesson learned: Use calloc. 😉

Lesson Learned 2 – Read header file comments carefully if they exist!

If you use uci_lookup_ptr(ptr,”bla.@bla.bla=blavalue”,true) it will not work!
uci_lookup_ptr needs a string that can be edited and that is not constant! 😉

GSoC 2018 – Better map for nodewatcher (1st update)

Posted on by Marin Stević

Since my last update I made a lot of progress in understanding how nodewatcher works, mostly around Django, and implementing some of the elements I stated in my last post.

My progress in the beginning was very slow because I hadn’t used Django in such capacity used in nodewatcher. But after a couple of trial and error moments and a lot of help from Django forums and wlan-si members I was able to get a grip of the things I needed. There should be a more detailed description as to how to some parts of the nodewatcher system work. Currently only a handful of people know how the whole system work and that shouldn’t be the case, I will try to document most of my findings and contribute them to the project to help others later on.

I started my own leaflet map in order to begin progress on the map while I learned everything around the current nodewatcher schema. I tried to implement the basic functionalities first to see how the whole code is layed out. As you can see from the picture below I tested out the fullscreen option of the map and also the color representation of the different nodes. In the top-right corner I also added the support for selecting which nodes to show.

This is just a test example and there is a lot of work implementing this into nodewatcher because here I wrote my own script and added the markers by hand. The biggest part is that I need to figure out where to add this code to nodewatcher and make it work with real nodes. I hope to have this figured out until the next update so that some of the features get added. These features are subject to changes and will most likely change in appearance.

I also had some problems with implementing new scripts but that shouldn’t pose a problem in the future. As I said earlier the main problem is adapting to the current code and maintaining its structure so that there isn’t any confusion when someone else takes over. But as I said this involves a lot of asking around for help and will be helpful later on because I hope to continue with this project after GSoC.

After that I will start working on the side menu and anything else that shows as a good addition to the map but for now it is important to learn how the system works and add basic upgrades so that later on it is easier to focus just on adding new elements not wasting time with learning how everything works again.

Tout médicament non utilisé ou déchet doit être éliminé conformément à la réglementation en vigueur. Les comprimés pelliculé sont jaune clair en forme d’amande, avec l’inscription. Imprimer Theme par defaut Theme doc avenue Theme dexther web. online casino Forme et présentation Composition Indications thérapeutiques Contre-indications Mises en garde spéciales et précautions d’emploi Effets indésirables Recommandations médecins Recommandations patients Fertilité, grossesse et allaitement Interactions avec d’autres médicaments et autres formes d’interactions Posologie et mode d’administration Durée et précautions particulières de conservation Incompatibilités Surdosage Propriétés pharmacodynamiques Propriétés pharmacocinétiques Effets sur l’aptitude à conduire des véhicules et à utiliser des machines Classes thérapeutiques Classes ATC Données de sécurité préclinique Précautions particulières d’élimination et de manipulation Conditions de prescription et de délivrance Forme pharmaceutique Nature et contenu de l’emballage extérieur Données technico-réglementaires.

GSoC 2018 – Kernel-space SOCKS proxy for Linux – June progress

Posted on by Fejes Ferenc

Assembling the testbed

I decided to give you a brief intorduction to the development of my testbed. In the past month most of the time I experimented with different virtual environments for kernel development. The pros of virtualization:

  • Fast test cycles: multiple virtual machine (VM) can use the same, freshly compiled kernel
  • No physical devices, you dont have to reboot your machine every time when you want to test your recent kernel changes. VMs reboots very fast (about 6-7 sec in my current setup)
  • Flexible network virtualization: you can connect your VMs with virtual ethernet links to virtual switches

My current worflow looks like this:
1. Make changes in the kernel code or configuration (make menuconfig or .config file)
2. Compile the modified kernel
3. Boot the virtual machines with the new kernel
4. Test if works, debug, etc.
5. Goto 1.

In the following you can find a detailed intro how to setup the kernel development and test environment with QEMU and virtual networking

Continue reading “GSoC 2018 – Kernel-space SOCKS proxy for Linux – June progress” →

GSoC 2018 – Ground Routing in LimeApp – 1st update

Posted on by Marcos Gutierrez

Overview

In this past month I was working on the update of the lime-app dependencies (it was quite outdated). I also worked on the view and the ubus module that reads and saves ground routing settings in the LiMe config file.

The view: (Github LimeApp branch)

It is the minimum configuration of a plugin for lime-app. It has defined the constants, the store, actions (set and get) and basic epics to obtain the data using uhttp-mod-ubus.

Lime-app uses Preact for rendering the views, redux for state management and rxjs-observable as middleware for asynchronous events. For now you only get the setting as a json and expose it to the user.

Ubus (Github lime-package-ui branch)

Create the lime-groundrouting package that exposes and sets the graound rotuing configuration to lime. For the time being, just expose the settings.

ubus call lime-groundrouting get

To do this I use the LUA library lime.config.

Next step: Save changes.

In the coming weeks I will mount the form and the validation scheme in both the app and the ubus module.

OpenWLANMap App: Update 1

Posted on by Thi Huyen (Lilli) Cao

Hi,

As I mentioned in the last blog post [0], the first step I did is defining the app’s functionalities[1] and designing the app architecture.

Basically the app contains 1 service, which runs in the background and communicates with UI thread per broadcast (public-subscriber pattern). Since as default, the service will run in the main thread, which is not wanted, I created a ScanThread to handle the scanning. It sends every 2s as default (and should be adapted with user’s speed etc. later) a scan request to the WifiLocator and gets a scan result back from it asynchronous. The WifiValidator then validates the scan result as well as the returned location, and puts the valid wifi access point in a WifiQueue. The WifiStorer will take anything from the WifiQueue and writes it to local disk (simple Consumer-Producer pattern). Based on the user’s upload mode setting, the WifiReader will be triggered if uploading is wanted, reads the local disk files in wanted format for uploading and passes it to the WifiUploader. It then uploads the data to any supported project api and as soon as the uploading is successful, the data will be deleted and ranking will be updated.

The next step I did is designing the new UI, got some feedback from mentor and changed it appropriately. Also in the process I defined all the user’s setting options. I spent a lot of time reading the android documentation for parallel processing and made decision for each functionalities, which is relevant for the next part. (WifiStorer, WifiReader: normal Thread, WifiUploader: AsynTaskLoader etc.) I write more about it in the next post.

Finally I jumped into implementing. I started with the demo mockup and then slowly implemented the logic part. I have finished the scan service and a part of the WifiValidator.The WifiLocator uses gps for defining location if available, otherwise it makes a request to openwifi.su with the surrounding wifis. I provided methods to do it with both new and old openwifi.su api in case we want to use any of them in the future. I ran into an android bug, where the wifi scan result is always 0 if the user disables GPS, even the location permission is granted.(Tested on Android 6). It is kind of weird because scanning wifi does not have anything to do with the gps and turning on gps the whole time will cost phone a lot of energy. Still it’s kind of wanted feature from Android to make users aware that their location information is being accessed when they use kind of app. Because the location of user’s phone could be defined based on the collected wifis. Since it is OS design, I pop up users a message with those information to ask them to turn on their GPS if they turn it off. I also implemented a part of WifiValidator, the WifiFilterer to check if an access point is openwifi, from freifunk or mobil hotspot or marked with _nomap (which should not be collected).

What’s now?

If you want to check the app, feel free to download install file .apk from [2].

If you as usual do not want to install an unknown source, I also provide a short demo video

 

What’s next?

In the next time, I will finish the WifiValidator, which should not only filter the access point but also validate the location to provide scan service a better scan period to save energy (in case the location is not changed for a long time, the scan service should be stopped etc.) and then other parts as shown in architecture image above.

Links

[0] https://blog.freifunk.net/2018/05/14/introduction-openwlanmap-app/

[1] https://github.com/openwifi-su/OpenWLANMap-App

[2] https://androidsmyadventure.wordpress.com/2018/06/03/openwlanmap/

 

The Turnantenna – First evaluation update

Posted on by Marco Musumeci

After a month of work on the project, the Turnantenna’s driver is evolving to the definitive version.

During this month I worked hard in order to write a good driver for the stepper motors. If you’re looking for more details and want to understand the basic functioning of the Turnantenna system look at my first blog. My work could be find on GitHub.

Overview

Image found at http://abhieeeprojects.blogspot.com

From wikipedia:

a unipolar stepper motor has one winding with center tap per phase. Each section of windings is switched on for each direction of magnetic field.

To control the position of the rotor, we have to play with the stator’s windings, following a proper pattern in order to make the first move smoothly.

The driver does exactly this, it turns ON and OFF the pins of the logical board (the orange pi) following the correct pattern; doing so the coils of the engine are powered properly, and the rotor turns.

The older version of the driver was a good prototype, and I based my work on it. As a good starting point, I didn’t change the overall scheme, but I found some problems with my mentor and I worked to solve them. The most important issues were the following.

Problem #1 : A poor control of inputs

We used python 3 to write the code. The driver is a class, called Stepper, that has some methods. Calling those methods, the object controls the real engine. To do so we needed to use the python wiringpi library.

The old code was something like this:

import wiringpi as w

class stepper():
    def __init__(self, pin1, pin2, pin3, pin4):
        """Set up the GPIO pins and all the attributes"""
        w.some_method_to_initialize_the_board()
        w.some_method_to_configure_the_pins(pin#)
        self.initial_attributes = XYZ
        ...

    def stop(self):
        """make the engine stops"""
        w.some_method_to_turn_off_the_pin(pin1)
        w.some_method_to_turn_off_the_pin(pin2)
        ...

    def move(self,speed,rel=1,dir=1):
        """make the engine run somehow"""
        ...

engine = Stepper(0,1,5,12)  # Random pin numbers

engine.move(250, 100)

To make sure that everything works, inputs should be controlled properly:

  • Pins choice should be controlled, different coils can’t use the same pin.
  • Values should be integers, not chars, lists, tuples or others
  • At this time, I don’t know what happen if I run the engine with negative speed. Will it go back?

This kind of control was lacking in the older version. This is our first problem.

Problem #2 : Too many input variable

Look at the “move” method, could it be simplified? The answer is yes.

dir (direction) parameter is not so useful. It was intended to switch between “go ahead” and “go backward”. But if I want to go in the opposite direction, I could say I want to do a movement with a negative speed. In the newer version I adopted this scheme allowing negative values for the speed, and removing the dir parameter.

rel (relative) could be removed too. Its function is to keep in memory the last position of the engine, but it could be better done using an object’s attribute.

Problem #3 : Wrong speed management

Thanks to Leonardo (my mentor) who discovered this problem, I had to plot the following graph that demonstrates the presence of an error in the algorithm used to accelerate the engine. To understand the problem, you need to see that the driver was intended to make the engine accelerate with a constant rate. In other words, acceleration has to be constant [1], and speed has to increase linearly [2].

Note: in the real code I added an acceleration factor which allows to manage the magnitude of this acceleration. That doesn’t impact on the constant acceleration hypothesis.

The simplified version of the algorithm is something like this:

# inputs: num_step, final_speed, speed = 0

step_count = 0
while step_count <= num_step
    speed += 1
    t = 1 / speed
    self.do_one_step()
    time.sleep(t)
    step_count += 1

Where:

  • num_step stands for the total number of steps that should be done in the acceleration phase;
  • final_speed is the goal, the speed wanted by the user;
  • speed is the actual speed value. Here we are in acceleration phase, and that’s why we’re starting from speed=0

It’s clear that the algorithm increase the speed constantly (and linearly). So the condition [2] seems to be satisfied.

However, speed is not directly used. To manage the step frequency, the time delay is used instead: this simplify the algorithm. But there is a problem, and this graph demonstrate that hypothesis [1] is not verified:

The graph shows a first interval where speed increases (acceleration), another one where speed is constant, and the last one where the engines slows down (deceleration). We can say that deceleration is a negative acceleration, in fact the two external intervals are specular.

In the graph we can note an hyperbolic shape of the acceleration phases. That’s a proof of the non-constant acceleration. The problem is that we expected a different graph, and wanted to see a linear shape of the speed function. The driver works, but not as we like.

The bug here is in time managing, since it don’t decrease linearly. Time is defined as t=1/speed, and that’s an equilateral hyperbola’s equation. Right now I’m working on a solution to obtain a linear, simple equation.

Solving problems : Make it clear

As this code will be published, I added a lot of documentation like comments, docstrings, and more verbose and specific error handlers. I rebuilt the entire move method to make it more clear and readable, and to solve the problems listed before. I wrote it down simultaneously with the tests, and it’s still work in progress just because it has to be bullet proof.

Tests

I can say that tests was the core activity of this month. I wrote tests, done tests, tests the tests, delete some tests, write new tests, correct other tests, added new tests, and still testing.

All the improvements done to the original code, was born from tests. I’m a newbie with python, and testing opened my mind to a tons of things about coding. Now I’m still testing and deepen the code, and how to improve it.

Conclusion

Now this first period is almost finished, and it’s time to conclude the last things. After that I’ll starting develop the web interface to control the driver from remote.

I make no secret of the fact that this project is a very challenge for me. This is my first serious coding experience. Almost everything is new, but the Ninux community is really supporting me. A special thanks goes to Leonardo and Edoardo, and their patient.

See you at the next post!

Meshenger – P2P local network messenger – Update 1

Posted on by Daniel Dakhno

 

In the recent weeks i have made quite some improvements on my initial demo application:

  • share information using channels like Telegram instead of a QR-Code
  • turn off the screen when the earpiece is held to the ear during a call,
  • use IPv6 link-local adress when possible, IPv4 as fallback.

 

I have jumped into WebRTC development, searched and evaluated different projects with similar goals:

  • serverless-webrtc-android
    • barebone example of WebRTC without signalling servers
    • I ported the code from Kotlin to Java for a better understanding and integration into the project
  • webrtc-android-codelab
    • loopback example: WebRTC PeerConnection through localhost, providing insight into how to connect two nodes running on the same device
  • android-webrtc-tutorial
    • insight into how signalling works through an external server

The given examples helped me to collect a knowledge- as well as a codebase which I will further use to implement video and audio transmission over WebRTC.

Evaluation, hacking and testing of those examples helped me to get a understanding of the inner-workings of WebRTC and will surely support me in the Integration of WebRTC into Meshenger.

 

Here are some screenshots of the current state of the application:

Contact list
Scannable QR-Code
QR-Scanner
Manual information exchange

 

 

 

 

 

 

 

 

My next step will be the adaptation of said projects to the newest WebRTC-version

as well as further dealing with the fundamentals of WebRTC and finding a way to circumvent a central server.