Try LibreMesh without having a router-GSoC 2022

Introduction

Hello everyone, my name is Irina and I am working in this edition of GSoC 2022 on the project “Try LibreMesh without having a router” with Altermundi. I will start this post by explaining what LibreMesh is, what are some of the obstacles that appeared when we were testing it, some solutions that we did to those problems, the knowledge acquired in community bonding, and what  objectives are in the next weeks. 

What is LibreMesh?

LibreMesh is a framework for creating OpenWrt-based firmware for wireless mesh nodes. LibreMesh works in a decentralized way and is used as a base for community networks. These mesh networks allow the connected nodes to route each other’s traffic.

Problems when testing LibreMesh

In the first steps, it was necessary to install QEMU, download the LibreRouterOS images and its dependencies,. LibreRouterOS has an automatic configuration  to be able to run virtual mesh networks on any computer. Then the LibreMesh test was based on the following steps:

1. Run a node:  this has been made without major issues. 

However, we discovered a bug when we were running the node: if you want to close the virtual node the network interface isn’t removed from the network configuration of the host. This is because the script that closes it does not eliminate that network interface.

2. Set up a node with internet access: for this it was necessary to install dnsmasq. In this case there were some inconveniences since giving this access required to use a port that was occupied by a dnsmasq process and a systemd-resolve, so it was necessary to free that port by killing those processes. By doing that, the host loses its DNS capabilities so we’re planning to solve this bug by allowing the user to give the node internet access without killing the processes.

3. Set up a cloud of nodes: it was necessary to install ansible and clusterssh.

When running the node cloud, it threw an error because it couldn’t find the location where the image files should be. To solve this, we modified some lines of the qemu_cloud_start.yml script:

  • rootfs: files/generic-rootfs.tar.gz
  • ramfs: files/ramfs.bzImage

placing as suffix “./” to each one

  • rootfs: ./files/generic-rootfs.tar.gz
  • ramfs: ./files/ramfs.bzImage

also in the same script it was necessary to change the following lines to use the LibreRouterOS Images:   

Another problem that appeared was that in the script there were calls to ifconfig

def if config (self, cmd):

return self.module.run_command ([ ‘ifconfig’] + cmd)

Since ifconfig is not installed in versions later than Ubuntu 18.04, we have to install it through this commands:

  • sudo apt-get update
  • sudo apt-get install net-tools

Period Community bonding

During this period I had a first approach to my mentor Germán Ferrero and an accompaniment from my advisor Tomás Assenza, who helped me in understanding and solving the different problems mentioned.

Each meeting has been very effective and useful since I was able to familiarize myself and learn about virtualization. The goals to be met in the following weeks were very clearly set out in the plan to achieve all the goals by the end of the project.

That is why I appreciate the kindness, predisposition, accompaniment, and pleasant help of both of them.

Goals for next weeks

– Explore different virtualization and containerization technologies such as Linux Containers, Docker, Virtual Box.

– Run some node on these tools in order to compare which of them turns out to be easier for such a fact.

Minstrel TX Rate Control in User space – GSoC ’22

Introduction

Hi everyone! I’m Prashiddha. I have recently graduated from Jacobs University Bremen with a BSc. Computer Science and minors in Robotics and, Global Economics and Management. For the past year, I have been involved in the research and development of open-source software at SupraCoNeX, primarily focusing on facilitating rate control in user space, which will soon be public.

For GSoC’22, I’ll be working on implementing and testing Minstrel HT, the default WiFi rate control for Linux-based OpenWRT OS, in user space. This first blog post intends to cover details on the necessary background to understand the project and its implementation.

What is WiFi Rate Control?

A typical WiFi network consists of at least a sender and a receiver that communicate through the propagation of radio frequencies within the license-free ISM band. The radio waves carry information in binary as an encoding, and the sender devices can choose from several modulation and transmission parameters such as coding rate, bandwidth, and guard interval. The choice of a transmission scheme between the WiFi devices determines the theoretical network throughput or data rate. A metric called the Modulation Coding Scheme (MCS) Index has been defined to help better understand the WiFi data rates and the RF environment of the network. The MCS index is based on the parameters of the transmission schemes mentioned above.

With newer IEEE 802.11 standards such as IEEE 802.11ax, there are hundreds of available MCS rates for transmission. At first glance, it may seem like maximum data transmission could be easily attained using only those rates which yield the highest theoretical throughput. However, the modulations which achieve high data rates only work best when the link between the WiFi devices is robust. Furthermore, compared to wired-based communication, the wireless communication channel demonstrates higher dynamics and is prone to interference, especially if multiple wireless devices share the medium uncoordinated. As such, the performance of WiFi networks is far from optimal, and there have been significant efforts to develop WiFi rate control algorithms that dynamically adapt transmission data rates in response to the varying wireless channel conditions.

Motivation

In Linux-based OpenWRT WiFi devices, the mac80211 subsystem in the kernel space is responsible for rate control. This includes the implementation of rate control algorithms like the Minstrel HT in the kernel space. The kernel space provides full access to the device’s hardware and memory. Development of modules is hence subject to the risk of complete system failure due to bugs or failure in particular modules and submodules. Additionally, development in kernel space is restricted to the use of integer value operations. Due to the instability and risk involved in accessing the floating-point unit, floating-point operations are avoided in kernel space. Lastly, capabilities for prototyping and debugging required research and development are highly restricted in this space.

Given the limitations and lack of ease of development in kernel space, the need for a user space rate control algorithm development is apparent. With this, my GSoC’22 project will focus on implementing a user space variant of Minstrel HT with experiments designed to compare the performance with its kernel space counterpart.

Deliverables of the project

The end goals of my GSoC ’22 projects are as follows:

  • Software Architecture of the user space Minstrel HT implementation in Python.
  • Proper Documentation and Guide on working with the Minstrel HT package.
  • Ready to run demo script to showcase the potential of user space Minstrel HT.
  • Detailed analysis of WiFi rate control experiments for performance comparison between kernel and user space Minstrel HT.

What’s been already done?

Previously, I’ve been heavily involved in the implementation of Minstrel HT for OpenWRT access points in user space. The implementation has been done in Python using a user space rate control framework called Rateman which is under development and will soon be released as open-source. Since the rate control algorithm in user space needs to be designed to work on multiple access points simultaneously, initially, multiple experiments were conducted to evaluate various methods of parallelizing rate control from the asyncio framework such as async task, thread pool, and process pool. The results indicated executing user space rate control as an async task from Rateman to be the best scheme for parallelizing.

With this, a user space Minstrel HT in Python has been developed to be executed as an async task and the first results seem to indicate towards a comparable performance with its kernel counterpart. However, the first experiments were far from foolproof and requires better-designed experiments to yield a concrete result.

What’s next?

The user space Minstrel HT is not complete and still requires implementation of additional features in order to be identical to the kernel Minstrel HT in terms of functionality.

  • Changing the output of user space Minstrel HT to a live printout of the rate statistics table.
  • Adding option to store the output in as CSV file to aid in comparision with the kernel Minstrel HT.
  • Extending user space Minstrel HT with functionalities from the kernel variant such as calculating number of retransmission, random sample table, and reducing the number of spatial streams.
  • Adding a new estimator called Butterworth Filter which is currently used by the kernel Minstrel HT.
  • Modifying user space Minstrel HT to accomodate for various WiFi rate control experiments if needed.

Concluding Thoughts

With the start of GSoC ’22 coding period, I’ll begin with modifying the output of user space Minstrel HT to printout of live rate statistics along with option to save it in a CSV file, identical to the kernel Minstrel HT. This will be followed by extending the user space Minstrel HT with remaining functionalities from the kenrel variant. In the end, the project will mainly focus on experimentations for performance analysis of Minstrel HT in user space.

With this, I’d like to conclude the first blog on the user space Minstrel HT GSoC ’22 project with freifunk. Please feel free to reach out and connect with me. 🙂

Thanks for the read!

Traffic Monitoring and Classification via XDP/eBPF – GSoC2022

Hi everyone!

I’m Qijia Zheng, my major is Cyberspace Security, a graduate student now studying in University of Science and Technology of China.

About me

My name is Qijia Zheng. Recently, I have conducted research in the field of wireless sensing. It is a interdisciplinary subject of wireless communication and computer vision(deep learning). Now I come to Freifunk to learn the knowledge of wifi to gain a deeper understanding of wifi communication. Also, I was attracted by the concepts of P2P and ad-hoc when I began to learn computer network. I am glad to see and be a part of such a community who brings them to reality, not just on the Internet.

About the project

Why XDP/eBPF?

eBPF can be seen as a way to bypass most kernel stacks like network stack, I/O stack and etc. Using eBPF to get information in kernel could bring less overhead and run customized programs. Also, existing eBPF projects like BCC, bpftrace provide a higher-level framework to write eBPF code, which make life easier.

What I have done?

With my mentors’ advises, I got my hands dirty with the project from a survey.

After surveying tens of wifi analysis projects, I got a list of wifi statistics and created a repo to record the survey.

I evaluated XDP and BCC on my x86 laptop equipped with AX200 NIC, but I encountered lots of barriers when implementing it.(see next section for detailed)

In addition to do a survey of some open source projects, I checked some papers about Rate adaption for mac802.11 wireless networks. The statistics of wifi traffic for transmission rate control could be classified into three groups ACK, SNR and BER, specifically they are packet loss ratio, transmission time, frame receptions, SNR, bit errors and etc.

As I got nothing from mac80211 based on AX200, I focused on collecting statistics from higher layers which basically provided by BCC and XDP project.

Official XDP repo provides a xdp_loader program. By leveraging xdp_loader, I succeded to load my XDP program on my wireless adapter to get the receiving bytes per second and other basic statistics of ethernet frames.

About MAC80211 subsystem

For some reasons, I just test XDP programs on AX200 NIC.

Obviously, we don’t have native XDP support for the driver of AX200.

I put my attention on generic XDP. After runing XDP-TCPDUMP tool provided by XDP-PROJECT repo, I got a “.pcap” file containing packets of wireless traffic captured by the tool. And the payload recorded in the file are all from struct xdp_md *ctx. However, the packets show nothing about IEEE80211 header even the wireless NIC working at monitor mode, which probably means AX200 NIC removed IEEE80211 header before XDP staff getting involved. As a result, we will not get any information about mac80211 header by implementing generic XDP on AX200.

Also, I tried bare tracing tool like kprobe, function graph to trace functions of mac80211 like ieee80211_xmit. Unluckily, I still got no output in TRACE file. That is to say, running eBPF programs attaching to kprobe to on AX200 is infeasible. But we still have some tracepoints of cfg80211.

Next step
  • Switch AX200 to Atheros series to evaluate XDP on ath9k driver to unlock the potential of eBPF attaching to dynamic tracing
  • Keep surveying papers related to transmission rate control topic
  • Select proper machine learning algorithm to implement features and measure the model based on Precision, Recall and F1 score
Goals of the project
  • By leveraging the statistics collected using eBPF, we could create a fingerprint for STAs to identify different types of devices.
  • Implementing a dedicated use case that demonstrates the benefits of eBPF/XDP.
  • Providing the foundation for further research in the direction of network research.
Finally

I would like to thank my mentors for offering the helps for me till now.

Videoodyssee System Refactoring – GSoC’22

Introduction

Hello folks 👋,

I’m Vijay. In this summer I will be working on the project ‘Videoodyssee System Refactoring’.In this first blog post I would like about my project and little bit about me.

About me

I’m Vijay Reddy , a Computer Science undergrad from India. I’m interested in Web application security, Computer Architecture and Blockchain Security. I do play CTF’s in my free time to learn about the latest security vulnerabilities and to learn new skills.

I’m new to the open source world and I found it really fascinating to work with communities around the world which are making the world a better place.I’m also fascinated by the idea of writing code which will be used by thousands of people around the world .Open source contributions will allow me to work with new people and make connections with cool people around the world which will be helpful for my career.

About Videoodyssee

In 2017 we started our video portal for freifunk at media.freifunk.net. To get videos online we have a more or less complicated upload workflow:

  1. Upload video file to a processing server.
  2. Correct video file settings.
  3. Convert video to target formats.
  4. upload to CDN and Youtube.
  5. Publish the video to the media portal.

The following systems are involved:

  1. Videoodyssee Uploader – a web portal to upload video files built with Spring Boot, Thymeleaf, CSS, JS, Ansible, and Maven
  2. Videoodyssee – the processing server, built with bash scripts and python
  3. lambdacd pipeline – controls the workflow, built with Clojure, Leiningen, and LambdaCD
  4. mirrorbits – the CDN, built with Go
  5. voctoweb – the media portal, built with Ruby, HTML, CSS, JS and puma

As of now, it is possible to upload new videos, but the process is not comfortable. So we need to improve the software.

Problems with the current Videoodyssee System:

  1. With current lambdacd pipeline we can’t have multiple workers to enable parallel video processing.
  2. The current lambdacd pipeline isn’t maintainable anymore and it has got a lot of dependency issues.
  3. Currently we don’t have any video verification steps in our publishing workflow which leads to publishing of unwanted videos.
  4. Currently there isn’t any feedback mechanism implemented in the front-end part so whenever a user uploads a video using Videoodyssee uploader, the user will not get any feedback on whether the video is uploaded , processed and published or not.

Project Milestones

  • Reimplement the video processing pipeline using a better and maintainable CI/CD tool which supports multiple workers for parallel processing:
    • Replace lambdacd pipeline with a better-maintained work flow system.
    • Update the processing pipeline scripts to make them work with the newest versions of tools like FFmpeg.
    • Reimplement the video processing pipeline with those updated pipeline scripts using the new CI/CD tool.
  • Add publishing workflow to prevent the publishing of unwanted videos.
  • Implement video processing feedback mechanism in the Videoodyssee uploader so that the user will get notified when each step of the video processing pipeline is finished.

Community Bonding Period:

In the community Bonding Period I have familiarised myself with my mentor Andi Bräu and discussed a lot about the project.

In this 3-week community bonding period , I evaluated different CI/CD tools with my mentor to find out a perfect CI/CD tool that fits best for our video processing pipeline and found out that GoCD and Jenkins will be perfect for our usage.

In the first week of the coding period, I will implement a sample pipeline using both of these tools to evaluate the performance and convenience of these tools , then I’ll choose one for implementing our video processing pipeline.

And finally , I would like to thank my mentor Andi Bräu for his incredible support and feedback.

Implement elRepo.io unit testing – GSoC’2022

Hi all! 

My name is Andrea and I am participating implementing elRepo.io unit testing project for GSoC 2022 edition!

I just finished a full stack developer bootcamp, and I’m now inmersed on the never-ending world of opensource programming! I would like to participate acitvely with communities that build this world to learn a lot, have fun, and get involved on a big opensource project!

I feel really lucky to have been selected and start coding as soon as I can!

What is ElRepo.io?

ElRepo.io is an Android app that combines the RetroShare JSON API with a Flutter frontend aimed to share content into the RetroShare F2F network.  Actually elRepo.io is on alpha stage and need a series of features implementation and unit testing to improve the both developers and users experience and scalability of the project. With the help of my mentors I expect to implement desired features and unit testing, learning how Flutter works together with a first landing into RetroShare decentralized world. 

And, how I will do that?

Before all, I did a series of crash courses about Dart and Flutter, just to don’t get stuck on the first days!

On this first period the objective is to code a set of unit tests for elRepo.io, but, according with my mentors, additional jobs may be necessary done:

    – Library refactoring to make them more testable

    – Null safety migration to bring compatibility with new versions 

    – Bug fixing and other todo’s fixing

Together with my mentors and their recommendations, I will check how to fix and go forward to the objective… So, let’s start coding!

TX-power control in WiFi networks – GSoC’22

Hello everyone!
My name is Jonas and I have the opportunity to be part of GSoC’22 cycle. With this first post I would like to introduce me and my project. The essence of the project is to provide and evaluate an implementation for TX-power control in the WiFi subsystem of the Linux kernel.

About me

I’m Jonas Jelonek and I recently finished my bachelor degree in ‘Automation and Electronics Engineering’ at the University of Applied Sciences Nordhausen in Germany. This summer I have some free time before I will begin my master studies in October, which I will spend on realizing my GSoC’22 project.

Coding has been a passion of mine for a long time and I have always been a fan of open source software (OSS) and hardware as they usually provide more possibilities and benefits as their proprietary counterparts. OSS is ubiquitous in my environment as I love both to use and to contribute to OSS, and also to share my code with others. During my studies, I had the opportunity to extend my knowledge and coding skills, and also to get more in touch with open source projects. I am involved in projects targeting research whose achievements are later going to be available open source.
Besides, I also had the honor to visit conferences like MECO, SSEA and ESA PAC which aim at sharing knowledge and expertise in fields of hardware and software engineering and offer the possibility to get in touch with scientists and engineers from other countries.

My motivation

My motivation mainly comes from a research project I have been part of since 2020. This project aims at developing, implementing and evaluating new and improved resource allocation algorithms for WiFi networks. As WiFi networks are becoming more and more important, the same applies to their reliability and performance and the efficiency of the underlying hardware. TX-power control (TPC) is a way to improve this by adjusting the transmit power per packet or per multi-rate retry (MRR), resulting in overall higher throughput in wireless networks and/or higher efficiency by optimized resource allocation.

Beginning in 2007, Atheros 802.11a/b/g/n chipsets (driven by ath5k and ath9k Linux drivers) started to support TPC in hardware, followed by Mediatek recently. They allow to set the TX-power per packet, but yet do not provide a driver interface for this in the corresponding Linux drivers. Unfortunately, also the mac80211 subsystem, which is responsible for WiFi communication in the Linux kernel, does not provide an appropriate API to support TPC and per-packet/per-multi-rate-retry annotation of TX-power.

I want to change that! I want to provide a proper TPC API for the Linux kernel to extend its support in several devices and to allow several projects to research the improvements and opportunities of TPC for wireless networks.

What we want to achieve …

Main goal of this project is to implement the missing three-tier software parts that make TPC for WiFi hardware in the Linux kernel possible:

  • (1) mac80211 structs to annotate TX-power levels per packet
  • (2) mac80211 structs to account TX-power status information once transmitted
  • (3) a TX-power control (TPC) API for TPC algorithms

This is achieved by developing and implementing the required structs in the mac80211 subsystem. These steps are subject of the first milestones and will be worked on in phase 1 of the coding period.

In addition to this, I will modify several Linux WiFi drivers to make use of this API. In the beginning we target to use ath9k and mt76 WiFi drivers for Atheros and Mediatek hardware. This hardware will then be used to test and validate the implementation in several experiments. These further changes and the evaluations are part of phase 2 of the coding period.

What we have done so far …

Usually, some time passes between the submission of a patch to the kernel mailing list and its final integration in a kernel release. We have already been preparing the implementation of TPC in the Linux kernel for some time with some patches .
Before GSoC I already worked on ath9k and ath5k Linux drivers, changing the way those two drivers receive TX rates from rate control algorithm to use the latest mac80211 structs instead of legacy ones. This simplifies further adjustments mandatory for TPC during the GSoC project. The corresponding patches were posted on the kernel mailing list in Nov 2021, and were included in the Linux kernel release 5.16 in Dec 2021. (commits 7f3a6f5dd207ecd582e1f02ebdb498493770a490 and a5d862da91058a855ab00bf46b5383543bbf3979)

In May 2022, I implemented changes in the TX status path of the mac80211 subsystem, minstrel_ht rate control (RC) and two WiFi drivers, and proposed them as patches on the kernel mailing list. They already include a new struct that allows the usage of all MCS rates (including 802.11ax and above, before only up to 802.11ac) and TX power annotation in status path, changes in minstrel_ht to make use of these changes and resulting changes in WiFi drivers mt76 and ath11k. These changes were approved within short time and will be included in the upcoming release 5.19 of the Linux kernel. (commits 44fa75f207d8a106bc75e6230db61e961fdbf8a8 and 569cf386ec5f4619388ae0c62169175dc2804f32)

In the community bonding period, I discussed the project objectives with my mentor and we made a plan for the upcoming steps of the project. After that, I started to work on the structs and their implementation that are part of the first milestones of the project.

What’s next?

As already mentioned in the text, there are several steps to work on and it’s a tight schedule. I already started to work in the structs and their implementation, and this will also be the task for the following weeks. This usually takes some time to develop a proper and well-thought design.
But I also the integration of this API into the already mentioned drivers will be begun in by the end of this period.

That’s may project!
I will keep you up to date with the progress. Keep watching!

Thanks for reading!
Jonas

Call A Friend – GSOC’22

Introduction

The existing version of Messenger app has some bugs related to connectivity and exit, the UI is a bit old and the codes are written in Java. This GSOC project aims at converting the whole source code to kotlin, improving the UI, fixing bugs and making a stable release for the app.

About Me

Hello everyone

I am Aditya Khandelwal, First year college student pursuing B. com hons in Accountancy, programming is my hobby, I am good at developing android apps with kotlin. I love to code, being a part of GSOC’22 would help me get experience with open source Development.

Project Goals

My goal is to fix bugs in the existing version of Meshenger app, convert the codes to kotlin and make a stable release for the app.

Decision after Community bonding period

I would start with making a cleaner UI for the app then I will convert the whole codes to Kotlin, implement the new UI and work on fixing the bugs along with frequent testing.

My experience

In the Community bonding period I familiarised myself with my mentors, got insight about how the app was initially developed, what we are looking forward with the app and so on.

I also got valuable feedback and suggestions on planning the working model.

I am really excited to move forward with the coding phase.

Completing the Retroshare Web Interface – GSOC’22

Introduction

The Web interface for Retroshare Application is under process. Many features are functional but there is still a lot of work to do and fix. This GSOC project aims at developing the web interface to a certain extent and include as many missing features as there are in the Retroshare QT interface.

About Me

Hello Everyone,

I am Sukhamjot Singh, a 4th-year undergraduate in Computer Science at International Institute of Information Technology Bangalore. I love 2 things – Table Tennis and Open Source development. The fact that my contribution, however small or big, will be a part of something huge and useful, motivates me to work further.

Project – Goals and Deliverables

Homepage of Web Interface.

The Web interface for Retroshare is made using MithrilJS ( a lightweight Javascript framework ) and the backend is mainly supported through the use of Retroshare QT interface and JSON API that is already available. Frontend design will be consistent with the already present features.

Major deliverables: The major deliverables will be in form of tabs which mainly include:

  • Files
    • File Manager
    • Uploads
    • Search
  • Forums
    • Navigation Bar for different categories of forum.
    • Displaying forums info
    • Search
  • Channels
    • The layout is same as forums.
    • Display posts, files, info of the channel
  • Configuration
    • This tab helps to configure Retroshare App
    • Config options for Network, People, Mail, Files etc.
    • This needs further discussion with the mentors.

Further objectives also include solving some of the issues in the features that are already present. Working on existing Github issues is also a priority.

Community Bonding period – First two weeks.

In this period, I have familiarized myself with the mentors ( Cyril Soler and Gio ) and discussed the goals and objectives of the project. I have started developing the Channels and Forums tab and have completed the basic layout. All of this work is done in a separate branch on my fork of the repository.

Ongoing Pull Request : https://github.com/RetroShare/RSNewWebUI/pull/51

My fork : https://github.com/Sukhamjot-Singh/RSNewWebUI

The updates about the development are regularly provided on Github and to the mentors. There are also weekly meetings being scheduled to report the progress of the project with the mentors. I will also post more blogs for further developments.

The process of contributing and getting selected for Google Summer of Code’22 has been thrilling and I wish to have an exciting summer with Freifunk – Retroshare and my mentors.