Hello everyone!
First things first so here is the code that I’ve written. You can find it in these repositories:
OpenWifiCore (core server application)
OpenWifiFeed (LEDE/OpenWRT feed with boot flasher and boot notifier)
OpenWifiWeb (web frontend)
OpenWifiTemplates (old templating system)
OpenWifiLocation (plugin that detects the location of a node via google location api and nearby wifi aps)

I also worked together with Arne to interact with his SDWN controller and agent. We created a small website that informs you on how to use the two tools together. (As I’m writing this it is still somewhat under construction. But I hope everything will be there soon 🙂 ) Furthermore there is also specific documentation (WIP) for OpenWifi here.

To be a little more precise here is the list of commits that have been done:

May

OpenWifiCore
OpenWifiFeed
OpenWifiWeb

June

OpenWifiCore
OpenWifiWeb
OpenWifiLocation

July

OpenWifiCore
OpenWifiWeb
OpenWifiFeed

August

OpenWifiCore
OpenWifiFeed
OpenWifiWeb
OpenWifiTemplates

Overview about what has been done

Everything that has been done can be put into 4 categories: authentication/authorization, api, database-model and infrastructure. I want to give you a brief overview of all these categories.

Infrastructure

Docker images are now build automatically via TravisCI and deployed on docker hub. You can find more information in my first evaluation blog post. How to use the docker images is described in the documentation. These images are also used for testing.

database-model

The graph based database model was made a first class citizen. It now automatically converts between the internal representation and the representation needed to sync to the AP. It is also now possible to create new configurations and create links between these. The query format is also used for authentication/authorization.

api

Providing all the new functionality via a REST style API was the focus of this google summer of code (web views are still needed for quite some things). It is used for managing users, services, nodes and changing the graph based configuration model. It is described in the documentation.

The concept of having a service is also something new that I created together with Arne in august. If an external application needs to make changes to configurations of specific nodes it can just subscribe as a service. The service takes a list of database queries, a name and a shell script with a compare string. If the output (stdout) of the shell scripts matches the compare value the node gets the name of the service as it’s capability (also something that was added during this GSoC). When a node has the capability the queries are applied to it’s config in a regular interval by the job-server.

authentication/authorization

This was the biggest part of this GSoC. There are two ways of giving access to a node – either by giving access to a path string of a configuration or by allowing a specific database query. For more details see my second evaluation blog post.

There were quite some challenges as this system allows for some really complex access configurations and there are still some things to improve here. (see below)

What else? Aka the smaller bits

There were also some smaller bits that were done that don’t really fit the other categories. The boot-notifier and boot-flasher scripts were improved, a way to abstract communication with the nodes was started (see documentation), some small UI fixes to the graph view were done. And probably a lot of other smaller fixes I forgot 😉

Future – or what needs to be done

I guess the most important thing is to get all of these new features exposed via the web views. That shouldn’t be to hard but wasn’t the highest priority during GSoC. Everything related to the graph should be made aware of all possible path strings that lead to a configuration. Currently just one string is used – this should be lists! The fine grained authorization needs some more testing and I also want to improve the pattern matching by combining the regular expressions you could use to describe a path string. This could be done with greenery. The authorization right now is also just focused on nodes and configuration – it would be nice to restrict access to some views and actions. For example restrict the access to LUCI, sshkeys or executing commands on a node. Furthermore there should be a per node option where “the truth” of a config lies (like if there is a difference between the actual node configuration and the configuration on the server which one is considered the one to go with) – than it would also be nice to disable the sync for some nodes (if manual changes on the node need to be made for example).make

Acknowledgments

A big thanks goes out to Google for organizing something cool as the Google Summer of Code, to Freifunk for letting me do this project with them and last but not least Julius for being my mentor!

Hello everyone,
this is the second milestone blog post about OpenWifi. In this blog post I want to talk a little more about the authorization in detail and give an update on what has been done in the last weeks.

Authentication

Authentication was enhanced with possibility to use a client side certificate or apikey (which can be send to the server with a HTTP-GET like ?key=MY_API_KEY). It uses groups to distinguish between the authentication methods and if you login with username/password it is also possible to mark a user as an admin.

Username/password login uses cookies and client side certificates have to validated by a proxy (like nginx for example) and added to the header. It is you can find the code for authentication and groups.

Authorization

The basic principle of authorization in pyramid is a ACL based authorization that gets principals (basically strings) from authentication. These ACLs are given by so called factories (if a view doesn’t use a custom factory the so called root factory is used).

I decided that it would make sense to have two strategies for authorization – coarse grained and fine grained. I’ll talk about the two in the next part.

Coarse grained authorization

Coarse grained authorization is based on specific views and access to a node. Access to specific views is managed by the root factory which is just a simple ACL that stores which groups have access to which views (like only admins can manage users and access objects, admins and client side certificate can add nodes).

Access to nodes is managed via access objects which are attached to a user or apikey and also attached to a list of nodes. (There is also a special setting that gives access to all nodes) To protect views that operate on a single node (like settings, status, etc.) there is the node context factory which generates an ACL that requests access to the specific node. (a user/apikey is given principals that match their accessible nodes)

To protect views that list nodes there is the get_nodes function that returns all nodes accessible by a user.

It might be a good idea to extend the node access that it can define what can be done with the node (to allow/disable luci access or ssh key access for example).

Fine grained authorization

Fine grained authorization is used in conjunction with the graph-based DB model. It is still WIP and I use a separate branch for it.This uses the data portion of the access object. Since access to the model is given as a rest API provided by cornice I’m using cornice validators give access. Since it shall be possible to share the same master configuration across different nodes it has to determine what are the maximum access for all nodes using the config for this user is.

Access can be granted in two ways – either by specifying a “path string” and an access (like rw, ro or none) or by just a allowing one specific query to the db. A path string has the following format: ConfigName (ConfigType).LinkName.Config2Name (Config2Type) etc. I want that it is possible to use regular expression for matching these names.

To make things easier I started by just comparing the levels of the “path string” to see if they refer to the same string when trying to figure out the maximum rights for a given node set.

Services

Julius, Arne and me started thinking if it is a good a idea to make it a little bit easier for external software to just update nodes with specific configuration options. So that this software doesn’t need to use queries for all the options it wants to modify. (And therefor use polling to update new nodes etc.) After thinking a bit about it I decided it would be a good idea to add something like services a service can register to. This service contains a shell-script and a compare value to determine if the node is desired to be modified by this service and also a list of queries that should be applied to this config.

Arne is using this to set a ONOS Server for the nodes via OpenWifi.

Discovery Script

The discovery script was updated to use client side certificates and advertise if the node is able to communicate via TLS.

Graph-Based DB Model

The GUI for the graph based DB model was updated a bit – now it is sorted with a different algorithm (based on the direction) and also displays information about the vertices (there for the is a small rest API for the vertices).

Also a lot of the code was refactored which will make it easier to add new features here.

Miscellaneous

It was evaluated how a github page could be used to add more documentation for OpenWifi. Some small things in the docker image were fixed as well. (for example discovery not working with nginx image or wrong file system rights)

What is going to be done in the next weeks

Fine grained access proved to be more complicated than I thought. But I want to finish it soon and start working on the communication abstraction. I also hope that there is enough time to improve the graph based db model (since the code is now refactored it should be not to hard).

Hello everyone,
this is the status report of the OpenWifi project before the first evaluation. I like to point out in this blog post what has been done and how I like to proceed the next weeks. Looking at my initial schedule I did some things that are not on that list and missed some others. So I also like to revamp the schedule. I did a Google-document for that and I’m happy to receive comments about it 🙂

I think a lot of things that have been done are the foundation for everything else to come and a lot of things that have not been done are low hanging fruits.

I think overall my initial schedule contained too many elements and I need to focus more on what is important and what are optional changes. I think I want to focus on testing/docker/https and authentication/authorization until the next evaluation and do the communication plugin-API from then until the end. Everything GUI-related might happen after GSoC.

What has been done

Testing, Docker and CI

That is the biggest part I did but I think also the most important point to get the project to a more professional level and make usage easier for new people. There is a docker hub for openwifi docker images. This way you don’t have to build the image – you just need to pull it. The image is build by TravisCI on new commits – so it is up to date.

I also started to implement tests based on docker images. There is a test that boots up a LEDE docker image and an openwifi docker image and checks if the LEDE container discovers the openwifi image and registers to it. This test is also done by TravisCI.

Just last week I also updated the docker image in the way that it can now also optionally ship with nginx and therefore use TLS! Nginx is also doing the client side certificate validation and sets a header value according to the outcome. (I guess something similar is also possible for apache and other servers)

TravisCI testing has also been added for pyuci.

Discovery Script

The discovery script has been completely revamped and is now a lot more readable. It also serves now as small client that updates the registration every 3 minutes.

pyuci

Pyuci has been refactored and extended. Diffs are now a separate class and can be applied and reverted from a config. Also the testing has been extended somewhat and is done by TravisCI.

Authentication

A basic user management has been added. That is a database entry and password hash storage using passlib and an openwifi authentication scheme. The admin user can add and modify users via a rest API.

I also thought about how to implement rights management – more on that below.

New DB Model

The new database model has not been on my initial schedule so much. But actually there is a lot to do. What has been done is proper syncing via sqlalchemy events of the database representation and the rpcd-json representation.

Revisions

OpenWifi now saves change to a config in a revisions database table.

What is going to happen next

Testing

Since there is now a proper infrastructure for docker based testing it should be used more like for testing https and configuration changes.

More tests are needed for the new database model and pyuci.

New DB Model

The new db model needs testing and methods for adding and removing of links and configurations. It should keep IDs consistent upon updating (to allow configuration sharing) -> so maybe make use of new pyuci diff features.

Also I should get rid of all old configuration access (like old templating and configuration changing). It might be nice to have a gui for the DB configuration but this has not the highest priority.

Authentication

The authentication needs to authenticate API keys and client side certificates.

Authorization

Give users/API-keys access based on node and pattern matching. Add option to restrict registration to client side authentication.

The pattern matching could be in the form the current master-config query or a path string (like network.lan.ipaddr or network.[interface].ipaddr or network.lan.*). It should be possible to set patters for read only, read write and restrict. (For example to have a eveything mathing pattern but restrict access to one specific pattern).

It should also be possible to restrict access luci2 (it doesn’t make sense to restrict the db querys if luci2 access is allowed ;)).

Discovery Script

The discovery script should be updated to setup capabilities and communication protocol (like rpcd, rpcd-TLS, NetJSON, etc.) on registration.

Nice to have features would be that it has an option to stop once a registration was successful and to accept a retry count.

I the future it would be nice to work together with OpenWISP and have and share the discovery mechanism.