I’ve set up a lot of services since planning out the data pipeline in June; the new infrastructure diagram looks like this:

So, I wrote a little Rust utility called json_adder which takes the JSON files and reads the data into a MongoDB collection. We also have a GraphQL server, running through Node.js, which can handle some (very) simple queries and return data. Many of these services are individually easy to set up, the tricky part is making sure everything works together correctly. This is what your favourite technical consultancies call “fullstack development”.

Data loading

The first change from the initial plan was to use MongoDB instead of MySQL as a database. MongoDB has a built-in feature for handling time-series data, which is what we’re working with. It integrates well enough with GraphQL, and since one of my mentors (Andi Bräu) works with MongoDB on a daily basis, there’s a lot of experience to draw upon.

Here is the Rust code for the utility which adds the data to the database, and here’s what it does:

• Read over each JSON file in the directory in a loop.
• Extract the last-modified time from each community in the file and do some conversion to BSON.
• Read the data into a struct, and add each object to a vector.
• Insert the vector into the MongoDB collection.
• GOTO next file.

When setting up the connection to MongoDB, the setup_db module will also check to create the collection if it doesn’t exist. This extra bit of logic is ready for when this has to be run in an automated process, which might involve setting up the database and filling it with data in one go.

I don’t have a binary to link here, and will write build instructions in future.¹ On each commit which changes Rust code, Github Actions runs cargo check on the repository. When it comes to deploying this in reality, we can change that workflow to provide release builds.

At the moment it is a one-way process. You point json_adder at the data directory and it adds everything to the database. For future work, I’ll need a way to keep the database in sync, only add the files which have been modified, run it on a schedule. For now, it works fine.

GraphQL

Here is the GraphQL server code. Fetch the dependencies, in this order:

npm install express express-graphql graphql mongodb

The ordering of these is important.

Then, run npm start, and open http://localhost:4000/api in your browser, you should see the GraphQL IDE.

At the moment, the GraphQL can handle a query with an argument for metadata. I’ll build out the schema, and the resolver, to handle more arguments / more complicated queries. A lot of the documentation for GraphQL assumes a fairly simple schema, built for handling calls to a web app, which is slightly different from our case. The JSON Schema for the Freifunk API is not only quite long, it is also versioned, and newer versions are not necessarily backwards-compatible. I am going to sidestep this complexity by writing a GraphQL schema which only includes fields available in all versions. My first working example to try is a query which counts the number of nodes per community.

You’ll notice that the last step in in the pipeline is yet to be completed, I don’t have a visualisation to show. But, now that we can reliably pull data all the way through the pipeline, we’re almost ready to make some graphs.

1. In the meantime, you only need to run cargo build in the json_adder directory. ↩︎

Hello everyone! This blog post is meant as a mid-point update on the Cable Purpose Autodetection project.

This first part of the project was the occasion for a lot of discussions on how the project will take place to occur, and discuss how it will materialize for a LibreMesh user. These discussions mostly happened in the LibreMesh mailing list, and in the public matrix channel.

What is the goal of the project again?

In some scenarios, routers running LibreMesh benefit greatly from having specific configurations applied, which are not yet natively integrated — As of now, an expert needs to step in and setup these configurations manually, which is not always an option for communities. For this project, I am investigating scenarios that would benefit from automatically applied configurations.

What was discussed

There was a lot of discussions in the LibreMesh mailing list and in private messages, talking about the implementation and the scope of the project

Here is a sample of configurations where it would be useful to have such a system:

• A LibreMesh router is connected to the internet and is serving internet access to its clients.
• A LibreMesh router is connected using Ethernet to another LibreMesh router in the same network, to extend coverage.
• A LibreMesh router is connected to another router not running LibreMesh, but working as an access point (in a point-to-point link) for client connecting to that router, and transferring data to the LibreMesh router.

Multiple solutions where discussed in the mailing list, for example, it would be useful to detect if a neighbor router is running LibreMesh or another firmware. A solution consisting of LibreMesh having its own ipv6 multicast has been discussed. This would allow a LibreMesh-running router to be more aware of what is around itself, and it could also detect if another LibreMesh router is part of the same network or a different one.

Implementation Solutions

One main point of discussion was “how to apply the configurations”. There was a lot of back-and-forth on that subject. The final software would take the shape of multiple scripts, each dedicated to tackle a single configuration setting.

• The configuration scripts could be ran automatically at runtime. For example, a script could run every time an interface goes up or down. This means that if a link between two routers is broken, and somehow goes on and offline regularly, the configuration setting would be applied repeatedly, and in the process, could break some other part of the configuration. In that situation, potential problems would be very hard to troubleshoot if the user is not warned that this auto-configuration is occurring.
• The configuration scripts could be ran when the user runs “lime-config”, which is an existing configuration tool for LibreMesh. This would avoid the problem of a script running repeatedly, because the user would stay in control, but it kind of ruins the whole “running configurations automatically” aspect of the project.
• Another solution would be to implement buttons & switches in the Lime-app (that currently allows you to manage a node and its network), and add a new page or field where a user could choose if a specific configuration should be automatically applied, apply it manually or not at all. The problem is that the name of these settings might be obscure to less tech-savvy users.

So, we have different possibilities, each with their advantages and drawbacks. The solution will be to have some auto-configuration applied in a way, and other auto-configurations applied in another way.

Virtualization and problems encountered

One of my biggest roadblock in starting this project was running virtualization software. A previous Google Summer of Code participant made modifications to the LibreMesh code to allow virtualization using Qemu or Ansible (available here), but this code wasn’t pulled into the LibreMesh repository because it is not fully ready yet. This implementations contains instructions and modifications that allows LibreMesh to virtualize with Qemu and Ansible, and both of those software have different working and breaking points, and I had to experiment a lot to learn how to solve or work around them.
Using Qemu, it is possible to run up to 100 routers that act as nodes in a network. You can access each node using ssh to ping, create interfaces or check connection from one to the other.
Using Ansible, you can use the qemu_cloud_start.yml file to pre-configure and fine-tune the layout of your network before starting the virtualization in batches. (This makes it easier to specify which node should be an edge node, connected to the network, etc…). This .yml file is working as a blueprint for Ansible, and it will build your interfaces and network of nodes from the instructions contained. You can then run the nodes as a single batch, and connect to each of them using ssh.

However, virtualization is limiting when trying to understand a system. Simulating the cables connecting each node of a network is harder to visualize, in lieu of plugging everything by hand, and you often encounter problems that would not happen if you were working on actual hardware.
For example, connecting the virtual machine to the Internet is challenging (and not working 100% of the time), because you need libraries that are not checked from starting the virtual machine software, and the error messages you get from the machine are not clearly related to what the actual problem is.

Alternative: Working on actual hardware

I am currently waiting to receive a few routers that a community member (Aparcar) sent to me. I plan on using these routers to setup a LibreMesh network, and implement the changes I made in a more physical way than my current virtualization setups.
This setup will allow me to bypass some of the problems I am regularly encountering specifically because of the virtualization, at the cost of a slower iteration cycle. I’m looking forward to setting up and tinkering with actual hardware!

Next steps

• The biggest next step is implementing code that can actually detect the specified configurations. The foundation for this is done, but more work is needed to make code that can fit the use cases.
• From the mailing list, a lot of different solutions were discussed. We seemingly have more solutions than problems, so the next steps are implementing some of those solutions, seeing how they fit together with the current LibreMesh code, and choosing which are the best candidates to solve the auto-configuration problems.
• Another big next step is transferring my changes to an actual router running LibreMesh code, to be able to test and code in a real-world setting

Bonus learning material

Q&A video

A big help in this project was a Questions & Answers session we organized with seasoned LibreMesh developer. We recorded the session, thinking that it would be good to keep track of some ‘newbie questions’ and their answers, for future onboarding. I uploaded the video to guifi-net’s peertube, using LibreMesh’s account. Thanks again to everyone who participated!
The video is available here

Vocabulary list of technologies

Another help in my learning process was a vocabulary list of the technologies I learned of, and how they were used within LibreMesh. This list gets extended every time I lean about a new part of a technology.
There was a lot more abbreviations I had to learn than I anticipated, and I’m still encountering more new ones every time!

Closing words

And that’s about it for a mid-project update! Thank you for reading, and don’t hesitate to send me a message if you’d like to know more.

Hello everyone! With the first evaluation of GSoC 2024 approaching, I am sharing this blog post to give a comprehensive update on the progress of the Throughput-based dynamic Power Control Approach. Please check out my initial blog post for more information in case you need to become more familiar with WiFi power control. 

New Developments

1. Basic Python Package Creation

A basic Python package for the power control algorithm was successfully created. Let’s refer to this package as ‘power_controller’. The power controller includes a main class and exposes two primary functions: ‘configure’ and ‘run’.

The ‘configure’ function is designed to facilitate the setup of the Transmit Power Control (TPC). It allows for manual activation of the Transmit Power Control (TPC) mode. It initializes the modes of allowed power and parses the incoming control parameters. These parsed parameters are then returned to the ‘run’ function to execute the main logic.

The ‘run’ function implements the core logic of the power controller. This function is responsible for dynamically adjusting the transmission power at configurable intervals. It supports various power selections ensuring the power control algorithm can adapt to different network conditions in real time.

2. Integration with Python-WiFi-Manager

The power_controller has been connected with the existing Python-WiFi-Manager package from SupraCoNex. The WiFi-Manager package provides an interface to monitor each transmitted frame and control the transmission rate and power.

We utilize this interface by linking the power_control package and exposing the  ‘configure’ and ‘run’ functions for the WiFi-manager package for further power control. 

3. Development of Power Controller Logic

The ‘run’ command now includes a simple yet effective power controller logic. Key features of this logic are:

a) Configurable Update Interval: A customizable update interval has been added, allowing the system to adjust transmission power at defined intervals.

b) Dynamic Power Selection: At each update interval, a random supported power level is set as the transmission power which helps in dynamically exploring different power settings.

4. Power Setting Modes

Talking about dynamic power selection, various modes of power setting are introduced. These modes define how the transmission power is selected during each update interval:

1. Highest: Always selects the highest supported power level.
2. Lowest: Always selects the lowest supported power level.
3. Random: Selects a random power level from the supported range.
4. Round Robin: Cycles through the supported power levels in a round-robin manner.
5. Fixed: Uses a predefined, fixed power level for all intervals.

5. Experimentation and Observation

A settings file was created to run the power controller with different power modes using the Experimentation Python package from SupraCoNex. The observations from the trace files were promising

• Power Control Verification: It was confirmed that the power levels were being dynamically controlled as per the logic defined.
• Rate Adaptation: The packet transmission rate was observed to change and adapt in response to the varying power levels, demonstrating the algorithm’s effectiveness in real-time conditions.

Next Steps

The next phase of the project will focus on advancement of the power_controller as well as data analysis and visualization to gain deeper insights into the performance of the developed power control algorithm. We will also advance the power_controller package.

Here is the outline:

1. Advancement of Power Controller

The major step would be extracting real-time throughput information for the power controller. Another step will be to implement the strategy to lower transmit power when the transmission is optimal. If in case transmission fails in this low power transmission, we stratetigize a method to fallback to higher transmission again.

2. Data Extraction and Plotting

• Data Collection: Write a Python script to extract data on rate-time and power-time relationships from the trace files generated during the experimentation phase.
• Visualization Setup: Utilization of visualization libraries such as ‘’matplotlib’ or ‘seaborn’ to create plots that can illustrate:
  1. Rate vs Power Selection Over Time: Plot how the selected transmission rate varies with different power levels over time.
  2. Throughput Analysis: Visualize throughput variations based on the chosen power settings and rate adaptations.
  3. Frame Aggregation: Analyze frame aggregation patterns and efficiencies under different power control strategies.

3. Analysis and Insights

Graphical Analysis: Interpret the plotted graphs to derive insights into:

a. Optimal Power Settings: Identify trends where certain power levels consistently lead to higher transmission rates and better throughput.

b. Impact of Rate Adaption: Understand how dynamic rate adjustments influence overall network performance.

c. Efficiency of Frame Aggregation: Evaluate the effectiveness of frame aggregation techniques in optimizing data transmission.

Based on this analysis, we can further iterate the power control algorithm to fine-tune parameters and enhance performance.

Conclusion

With the successful creation of the Python package, integration with SupraCoNex tools, and promising initial experiments, the second half of the coding period will be focused on power control in a real changing environment through data analysis and visualization. Please feel free to reach out to me if you would like to know more about the ongoing project. Thank you for reading. 

Hello everyone, I’m Pierre, I was first introduced to Freifunk when searching for free WiFi in a cafe in Berlin, and I’m inspired by the radical visions of people attaching antennas to apartment buildings. Now I’m here to work on the community data API.

Since 2014, Freifunk has been collecting a snapshot of information about all communities, every hour, in a big directory available here. At the moment the data is being used for a calendar, a news feed,¹ and a map of communities, among other services. Maybe you can think of lots of other things which could be done with that data, but first you would have to parse some 80,000+ compressed JSON files.

The goal of this Summer of Code project is to overcome that hurdle up front; make this community data more easily accessible, and uncover some insights about the network along the way. This is a typical situation for large archives, where one of the key challenges is making data readable and useful.

I intend to build a pipeline from the JSON directory through to some visualisations on a web page. One step behind that, I want to expose a GraphQL interface so that anyone else can query the data for their own purposes. The main part of this project revolves around setting up infrastructure to handle each section of this pipeline.

Although this system is conceptually very simple, there are lots of moving parts involved. The JSON files are loaded into a SQL database, the database talks to GraphQL, which itself talks to some browser-based visualisation library.

I’ve tried to make choices which prioritise boring technologies, reduce the fragility of this process, and hopefully make it easy to maintain in future. I will build it from the top down; starting with the database, and finishing with the visualisations themselves.

I’ll be working in this repository. There’s not much to show at the moment, but it should rapidly take shape in the coming weeks. My next post will be about the process of loading these files into a database.

1. On which this blog appears, hello! 👋 ↩︎

1. Introduction

Hi everyone. I am Raul Shahi. I am currently pursuing a Master’s degree in Computer Engineering for IoT Systems from Nordhausen University of Applied Sciences, Nordhausen, Germany. For the past couple of months, I have undertaken the position of Research Assistant at my university, which involves researching and developing open-source software for IEEE 802.11 resource allocation.

For GSoC 2024, I will work on a novel Power Control algorithm to reduce interference and increase aggregate network throughput. This introductory blog post intends to summarize the necessary background to understand the details of the project.

2. Leveraging TPC performance for enhanced WiFi Performance

Regarding WiFi networks, ensuring everything runs smoothly is all about how we manage the available resources. As the concept of WiFi resource allocation has been covered by Prashiddha during his GSoC projects before, I would suggest the readers refer to his blog posts [3][4]. For IEEE 802.11 devices, deep diving into the system internals is required which can be overwhelming for developers as they would have to be familiar with kernel subsystems and Ansi C. This technical barrier makes it tough to adapt and innovate with new algorithms.

On top of that, many of the latest WiFi chips are becoming closed-source which means the open-source community can have a harder time developing new and advanced algorithms. However, chips like Mediatek, still share throughput information which can be utilized for the optimization of WiFi networks.

Active Transmit Power Control (TPC) is an exciting development area. By adjusting the transmit power for different devices, TPC algorithm extensions like Minstrel-Blues which is built on top of the Minstrel rate control algorithm are promising. They can reduce interference, boost overall network throughput, and improve how the network handles multiple devices in crowded spaces.

3. Motivation

To solve the challenges in WiFi resource allocation, the SupraCoNex project introduced the user-space WiFi Parameter Control API and an interface called Python-WiFi-Manger to enable the development of algorithms using Python. Developers can use these tools to create and test their algorithms on real WiFi chips. On top of the API, a remote socket extension has been realized to facilitate the remote management of access points. The API allows users to monitor transmission data and adjust rates and power on access points without technical knowledge of the kernel. 

I propose developing a new transmit power control algorithm using the WiFi-Manager for IEEE 802.11 ac chips, based on throughput estimates. This approach could potentially extend to Mediatek ax chips, paving the way for smarter, more efficient WiFi networks. However, the approach can be easily extended to any chips that expose the estimated throughput information and provide an interface to fine-tune power settings in real-time.

4. Methodology

The WiFi-Manager package in Python allows the user to monitor each transmitted frame while leveraging the options to control the rate and power for each connected access point. The first task will be to create a new Python package for the power controller and integrate it with WiFi-Manager. Since WiFi-Manager does not yet include throughput information, we will either add it to WiFi-Manager for broader use or directly into the power controller.

The power controller will track throughput at different power levels and average these observations using techniques like the Butterworth filter on EWMA. This can be the basis for determining the optimal transmission power. If throughput suddenly drops, the controller will have a fallback strategy, such as increasing power aggressively to restore the link quickly.

Like the Minstrel-HT [1] and Minstrel-Blues [2] algorithms, our power controller will have update and sample intervals. At the end of each update interval, it will adjust the power level based on recent observations. During sample intervals, it will explore different power levels to find the best settings.

The existing solutions from the Experimentation Python package from SupraConeX can be used to compare the proposed approach, evaluating performance based on rate selection, aggregation, throughput, and power levels.

5. Deliverables of The Project

The end goals of my GSoC 2024 project are as follows:

a. Implementation of a dynamic power control algorithm to adjust power levels
based on throughput information with the ready-to-run demo.

b. A detailed report documenting the research methodology, experimental setup,
results, analysis, and conclusions drawn from the study.

c. A study conducted using the Experimentation Python package that compares
the performance of the proposed approach with the existing solutions on
productive Freifunk Mesh and Access deployments.

6. What has been done until now?

a. Developed a foundational understanding of the importance of resource allocation in wireless transmission.

b. Familiarization with the ‘WiFi-Manager’ package as well as other packages from ‘SupraConeX’.

c. Remote access point setup for monitoring of data transmission and power, rate control.

7. What’s next?

a. Create a Python package for the power controller.

b. Preliminary performance comparison and evaluation of the developed algorithm.

8. Conclusion

I am incredibly grateful to be part of GSoC 2024 as we embark on this exciting journey. Developing and implementing a new power control algorithm with WiFi-Manager is just the beginning, and I am thrilled to contribute to innovations that could significantly improve WiFi performance. This project promises to push the boundaries of what’s possible in wireless communication, and I look forward to sharing our progress and successes with you. 

Please feel free to reach out and connect with me.

References

[1] Minstrel-HT Source Code in the Linux Kernel 

[2] A Measurement-Based Joint Power and Rate Controller for IEEE 802.11 Networks

[3] Minstrel TX Rate Control in User Space, Prashiddha, GSoC ‘22

[4] GSoC ‘22 and ‘23 blog posts from Prashiddha

Hi everyone, I am Gabriel, and I am currently studying Web Application Development and Mobile Application Development. This summer, I will be participating in Google Summer of Code 2024 to work on developing a Modular Assistant for OpenWrt.

Abstract

The goal of this project is to develop a modular assistant for OpenWrt that simplifies the initial configuration of OpenWrt devices. This assistant will enable users to configure their devices quickly and easily, even without deep knowledge of wireless routers. The assistant will be implemented using LuCI, a framework for creating web interfaces for configuration and monitoring.

Plan Of Action

This project will focus on creating an assistant that guides users through the initial configuration of OpenWrt devices. The assistant will have a modular design, allowing it to be adapted to different scenarios and specific needs of communities like Freifunk.

Implementation

1. Analysis of LuCI’s Client-Side API:
   • Understanding the operation of LuCI’s client-side API will be fundamental for the development of the assistant.
2. Development of rpcd Services:
   • Learn how to write rpcd services for communication between the user interface and the backend of the OpenWrt system.
3. Configuration of Permissions and Menu Entries:
   • Configure permissions and menu entries to ensure an intuitive user interface that is easy to navigate.
4. Adaptable Modular Implementation:
   • Develop the assistant in a modular manner, allowing it to be adapted to different scenarios through scripts.

Deliverables

• Creation of a stable, working modular assistant for OpenWrt.
• Detailed documentation explaining the implementation and use of the assistant.
• Ensure the assistant is adaptable for different scenarios and needs of the Freifunk community.

Concluding

I am delighted to be part of the GSoC 2024 program and contribute to the Freifunk project. I’m looking forward to collaborating with the Freifunk team to create a valuable tool for the community.

I would like to thank my mentors, Martin and Tobias, for their guidance and support. I hope to have a productive and successful summer working on this project.

Hi!

Very happy to be in this project. Piranha captive portal solves a well-known problem in community networks: the ability to manage vouchers and access to the internet and local services. As it says in it’s README:

It could be used in a community that wants to share an Internet connection and for that the user’s pay a fraction each, but needs the payment from everyone. So the vouchers allows to control the payments via the control of the access to Internet.

My name is Henrique and I’m currently working as a substitute teacher. My background is system administration and computer networks, so developing this project will be really challenging but I feel very comfortable doing so.

I’m also part of Coolab, a collaborative laboratory that fosters communities networks in Brazil.

Context

This project aims to develop the new release of Piranha, a package from Libremesh that enables communities networks to setup a captive portal and control/share internet access in a sustainable way. Currently Piranha is only supported in OpenWRT version 19.07.

The following are objectives of this project:

• Migrate from iptables to nftables;
• Include Piranha package on OpenWRT repository
• Make necessary changes to work with DSA on newer routers

The use of captive portals in communities enables the creation of vouchers and parental control, for example, it’s possible to disable access to social networks during night time. Since community networks can have multiple gateways to the internet, there’s a need to share information about current vouchers. This problem is solved by the shared-state package. Below is an illustration of a home user setup and a community network setup:

Acknowledgment

I would like to thanks Hiure and Illario from being my mentors on this project. Libremesh is an awesome project that enables non-technical people to deploy a mesh network in a matter of seconds.

Conclusion

I’ve never developed or upgraded a package before, i’m more into system administration, so it will be really challenging. 🙂

Thanks for reading and see you in the next post! Happy coding!