GSoC 2018 – Easily Expandable WIDS – Final report

This summer I worked on an Easily Expandable Wireless Intrusion Detection System (called Eewids). In this blog post I’d like to present what I’ve done, the current state of the project, the lessons learned and what needs to be done in the future.

Projects repository on GitHub: techge/eewids

What I’ve done

Analyzing existing projects

Before actually starting this project I analyzed existing Free and Open Source Software (FOSS) connected to IEEE 802.11 standard. I achieved this in part by looking through all projects listed on the extensive wifi-arsenal of the GitHub user 0x90 and categorized most of the entries myself.

I realized that there just is no complete ready-to-go WIDS yet, at least regarding FOSS. For me a WIDS should

  • detect most of the known Wi-Fi attacks,
  • scale easily and thus be able to work within big organizations and
  • be easily expandable.

Although there is indeed software on GitHub which can be used to detect Wi-Fi attacks, they are usually specialized on some attacks and/or they are hobby projects which would not fit in setups of bigger environments. Please have a look at the defence-related Wi-Fi tools on the wifi-arsenal list.

A distributed, highly scalable system

Based on a proposal of my mentor Julius Schulz-Zander, I created a framework with a microservice approach. Instead of creating a monolithic solution, Eewids is supposed to consist of different well-known and actively developed software. The central point of Eewids is a message broker. The advantage is an independence between different system components. New detection methods can be added to the system without actually bothering about the capture process, the parsing or even the presentation of the results. Everything leads to the message broker.

RabbitMQ is an awesome choice for such a message broker. It is not only most widely deployed, but it also supports a lot of programming language. No matter if you want to add a detection method using C, python or Java, there already exists a library to easily attach to RabbitMQ, get the captured and parsed data and sent the results back (see RabbitMQs devtools).

The message broker also helps to make the whole system highly distributed and scalable and thus helps to fulfill the requirements to work for bigger organizations as well. Many organizations may already have experiences with RabbitMQ-Clusters.

System components

Eewids consists of:

  • a capture tool to get the actual 802.11 data packets
  • a parser to transform the information into simple json-formatted data
  • a message broker (RabbitMQ)
  • detection methods
  • a visualization

The project aimed to find a solution for all components.

The capturing should be done by Kismet, which turned out to be a bad idea (see section “Lessons’s learned”). Therefore, a simple capture tool was created, which will be further improved (see section about future work). For parsing the package a parser was created (“eewids-parser”) in part based on the existing project radiotap-python. As kind of proof-of-concept the visualization software Grafana was added (in tandem with InfluxDB). This turned out to be a very good choice and is now integrated as the main visualization in Eewids. Grafana connects easily to various systems, so that other data withing a company may be connected and visualized as well.

Eewids setup with RabbitMQ as central point and an own capture tool based on standard tools
Overview of Eewids’ components

Lesson’s learned

At first, I didn’t dare to create a capture tool myself. I thought that this would take a lot of programming experience. Therefore, I have chosen Kismet as a basis for capturing the 802.11 data. Kismet is well-known and very popular. Still, it does not fulfill the above requirements – it is not a full-featured WIDS, it is monolithic and the code proved to be difficult to read/change. I invested a lot of time to integrate Kismet in Eewids, to build docker container (which did not exist before), to read out pcap export from the REST interface etc. In the end I integrated a feature-rich system to only use one function – the capturing. After several crashes of the Kismet server and an analysis of the network performance, I decided to try to create a simple capture tool myself (see this blog post).

It turned out that it is not that difficult to create a simple capture with help of libpcap. Although the capture tool is still very rudimentary it already fulfills the function which Kismet served before. While I still think, that it is always a good idea to have look at existing projects first, I would trust more in my skills next time, to avoid wasting to much time in integrating a system which is not suitable for the project at stake.

What needs to be done in future

While the framework itself is ready and works fine, there is still a lot to do. The capture tool should be further improved as described in this blog post. Getting used to libpcap needed more time than anticipated, so that I was not able to include all features yet.

The parser should get extended depending on upcoming needs. For example, only some element ids are parsed for now. All main 802.11 fields are parsed though. See the projects page for an overview of all parsed data.

More dashboards for Grafana could be added to visualize as much data as possible. Furthermore, some useful notification could be added by default.

Most of all, Eewids should get more actual detection methods in the end. The RogueAP detection that was implemented during the summer is a start, but served mainly as a proof-of-concept and shall be improved. As Eewids is supposed to be easily expandable this last objective should in part get solved by different persons in future.

In the end I am eager to further improve the system even after the end of this Google Summer of Code. I am glad that I had the opportunity to work continuously on this project. Now I am more confident in my skills than before. I’ll surely have less doubts to actually start a project in the future.

GSoC 2018 – Easily Expandable WIDS – Second Update

In this blog post I’d like to present the recent progress made for Eewids. This time, our main focus was the performance of the current setup. Besides we did some minor improvements, mostly adding some more dashboards to Grafana to visualize the data captured. This blog post focuses the results regarding Kismet as a component of Eewids. Continue reading “GSoC 2018 – Easily Expandable WIDS – Second Update”

GSoC 2018 – Easily Expandable WIDS – First Update

In this blog post I’d like to present the recent changes made in Eewids, why they were done and what’s to come next. For an introduction of Eewids see here.

In general the steps done the last weeks aimed mainly at the easiness of use and testing the main concept – having an easily expandable framework at hand. Thus, a RogueAP detection was added and visualization based on InfluxData tools and Grafana were included. Both steps were much more easy to achieve because of the architecture of Eewids.

Starting Eewids most easily

For everyone potentially interested in using Eewids it would have been a big hassle to compile Kismet (git development version) by herself. As Eewids is completely based on Docker container most of the components didn’t need to get installed. And that’s quite important. No one wants to compile, start and administrate all the stuff: Kismet, Eewids’ Parser, RabbitMQ, InfluxDB, Telegraf, Grafana and finally the plugins added to Eewids (like the RogueAP detection, see below). While all these components are provided by Docker container and can get started by simply hitting ‘docker-compose up’, the Wi-Fi card had to get accessed directly so far. Therefore, it was necessary to have a recent version of Kismet’s remote capture, which is not included in any major Linux distribution yet.

Luckily Kismet’s developer found a solution to this problem and documented it. We adapted this to the needs of Eewids and now have a solution in which one can start Eewids easily on a local machine, needing nothing more than a compatible Wi-Fi card, docker and docker-compose. Please see the getting-started.md of Eewids for more information and try it yourself! 😉

Renaming fields of captured data

To make the captured data of Eewids as accessible as possible for developers many field names saved in the message broker RabbitMQ were changed to be quite similar to Wireshark’s “Display Filter Reference”. See here.

Hearing Map for RogueAP detection

A simple RogueAP detection which existed before have been expanded by a hearing map. Now a whitelist contains not only valid ESSID:BSSID pairs, but also the information which remote capture is able to see which AP. Thus, an attacker can not use a valid ESSID:BSSID pair of a AP which is located in a different building to cover an EvilTwin attack. See here for more information.

Add a visualization tool: Grafana

We develop Eewids to make it easy to add new functions to it. To test this claim and to actually extend functionality by a way to analyze and visualize what’s happening arround, we added Grafana. It connects easily to different datasets (like InfluxDB, Elastic etc.) and let you create graphs and lists etc. As a starting point we added InfluxDB to save our captured data, Telegraf to get the data out of RabbitMQ and to send them to InfluxDB and Grafana to use the data from the InfluxDB.

Which would have been a hassle to implement on a local machine was quite easy with docker and a already existing dataset provided by Eewids in RabbitMQ. Thus, it only took us some hours to find out how to use this software. Even this time was not related to Eewids itself, but just to the missing basic understanding of Telegraf, InfluxDB and Grafana. That is to say if anyone who already know these tools would have liked to add these to Eewids could have done this easily. And this is the objective of Eewids.

We consider this a successful proof of concept. We used InfluxDB for Grafana, because we expect new things to come which depends on/use InfluxDB. Likewise we can imagine the fast and forward implementation of Elastic and the related tools and software. We’d glad to see this adapted in the future as well. 🙂

What comes next?

Now that we have a visualization tool (Grafana) added, it would make sense to extend it with more information, letting alerts visualized etc. Furthermore, we’d like to improve the “backend” features for developers. That means we would like to create some templates to easily start using Eewids data and adding detection methods. Let’s see how it works out!

GSoC 2018 – Easily Expandable WIDS – Introduction

Hello,

I am Alex and I want to create a framework for making an easily expandable wireless intrusion detection system this summer. The objective is to create a working environment which can be expanded with microservices to detect attacks on Wi-Fi networks and which fits easily within rather large organizations instead of small private setups.

All the things are happening on GitHub and thus this introduction is based on the README I created having this blog post in mind.

Background

Existing WIDS Tools

Analyzing 0x90/wifi-arsenal especially in search of wireless intrusion detection systems (WIDS) I realized that there just is no complete ready-to-go solution yet, at least regarding free and open source software (FOSS). For me a WIDS should serve the following needs:

  • detection of most of the known Wi-Fi attacks
  • scalability and thus being able to work within big organizations
  • simple expandability (there are always more attacks to come ;-))

Although there is indeed software on GitHub which can be used to detect Wi-Fi attacks, they are usually specialized on some attacks and/or they are hobby projects which would not fit in setups of bigger environments. Please have a look at the defence-related Wi-Fi tools on the wifi-arsenal list.

An exception should be mentioned: Kismet. It is probably the most famous and complete FOSS Wi-Fi solution and very popular. Still, it does not seem to fulfill the above necessities completely. And it is probably not the objective of Kismet to be a full-featured WIDS either. Instead it has many features for pentesting Wi-Fi networks and other interesting stuff.

Why Not Just Expanding Existing Programs?

One solution would be to simply add needed functionality to Kismet. And this is definitely a good idea and I encourage everyone to improve the code of Kismet. Some needs mentioned above could be solved with a microservice approach more generally though. This is exactly what EEWIDS tries to achieve. By creating a containerized framework EEWDIS enables

  • scalability
  • working easily in setups of bigger organizations
  • the possibility to add functionality easily (see below)

Main Idea of EEWIDS

Simple layout sketch of EEWIDS
Simple layout sketch of EEWIDS

Basis Kismet

EEWIDS uses Kismet as a basis. Thus, it uses Kismet’s advantages and tries to add functionality by using container techniques. As Kismet is under heavy development right now, EEWIDS uses the git version of Kismet right away, which is completely different to the last release from 2016. The Kismet remote capture (which replaces the former Kismet drone) is the only piece of software, which can not be containerized. The Kismet remote capture has to run on the machine which contains a Wi-Fi card which is able to monitor the traffic. As Kismet is very popular the Kismet remote capture will already run on many different machines and platforms, e.g. on OpenWrt. Therefore, it is better to use Kismet as a basis for capturing the data instead of building an own system.

The Kismet remote capture will send the data to a Kismet server instance which is running in a container. By using the Kismet server we will be informed about every attack which Kismet did detect and thus we can reuse the work already done on this side. EEWIDS will attach to the Kismet server to:

  • pull the pcap-ng data stream which contains all data captured
  • pull all alerts raised by Kismet server itself

Message Broker RabbitMQ

Both kind of information will be parsed and submitted to a Message Broker afterwards. The Message Broker is the central point of EEWIDS. By using RabbitMQ – one of the most popular systems of its kind – it is easily possible to subscribe to a needed information. This is supposed to be the big advantage for developers. Thus, instead of capturing and parsing Wi-Fi packets itself, a detection method only needs to subscribe to the needed information and will receive it directly from the Message Broker. Furthermore, the developer can use any programming language or system which is needed for this kind of detection, without bothering C++ or other stuff, which may would be necessary for Kismet plugins.

Analyzing and Visualization

The actual analyzing is done in services dedicated to this task. E.g. instead of parsing packages, looking for Beacons and analyzing it afterwards, a service will just subscribe to all Beacon frames. All other frames are not of interest. The service does not need to parse the Beacon frames, it just needs to access the json-formatted information it got from the Message Broker, e.g. data[‘wlan.ssid’] or data[‘wlan.bssid’]. This can be done independent of the programming language, as most of them already have modules for json and are able to access RabbitMQ. This setup should indeed work for every language which already has a client listed on RabbitMQ website.

Another advantage is the freedom of choice of visualization/analyzing software. It is easily possible to include either influxdata’s TICK stack or the elastic stack, both Open Source analyzing software which also have anomaly detection methods. These stacks and other software already have interfaces to access RabbitMQ and to read json-formatted data and thus it is easy to extract the collected information as needed.

This should make it easy to extend EEWIDS in various ways. Let’s see what can happen.

Focus

The usability on a developers perspective depends on the availability of logged frame information actually stored in RabbitMQ and the existence of easily adaptable templates. Furthermore, it has to be as easy and straight-forward to deploy the system as possible. That’s why I’d like to focus on three things:

  • the parsing of Kismet’s pcap-ng files should be as complete as feasable
  • there should exist templates for some major programming language to describe the usage
  • the deployment should work straight forward

Freifunk-Broschüre “Freie Funknetze in der Praxis” von Medienanstalt Berlin-Brandenburg

Die Medienanstalt Berlin-Brandenburg (mabb) hat eine eine Broschüre “WLAN für alle – Freie Funknetze in der Praxis” veröffentlicht. Aus der Pressemitteilung: Aktuelle Publikation der mabb informiert umfassend über das Thema „Freifunk“.

Sie erläutert ausführlich, was sich hinter „Freifunk“ verbirgt und welche Chancen und Risiken mit diesem Netz verbunden sind. Mit der Publikation möchte die mabb die Freifunk-Community dabei unterstützen, die Bekanntheit von Freifunk in der Öffentlichkeit zu vergrößern. Noch kennen zu wenige Bürger und Institutionen die Möglichkeiten dieses Netzes. Und die, die es kennen, haben Bedenken und Vorbehalte es zu nutzen oder selbst anzubieten. Fragen wie „Ist das Freifunk-Netz sicher?“, „Mache ich mich strafbar wenn ich meinen Router für andere öffne?“ oder „Hafte ich für illegale Downloads anderer?“ werden immer wieder gestellt. Die Publikation der mabb greift diese Fragen auf und gibt praktische Erläuterungen und Anwendungshinweise für Nutzer und Anbieter.

Die Broschüre kann als als PDF heruntergeladen oder analog bestellt werden. Die Inhalte stehen unter einen freien Creative Commons-Lizenz.

 

Rathaus Berlin-Neukölln funkt frei!

Ab sofort haben Besucherinnen und Besucher sowie Anwohnerinnen und Anwohner die Möglichkeit, rund um das Rathaus Neukölln über das Berliner Freifunk-Netz freien, zeitlich uneingeschränkten und kostenlosen Zugang zum Internet zu erhalten.

Die im Rathausturm installierten Router sind über Richtantennen an das bestehende Berliner Freifunk-Netzwerk angebunden und stellen so nicht nur eine Verbindung ins Internet her, sondern ermöglichen den Nutzerinnenn und Nutzern auch Freifunk interne Dienste anzubieten und nutzen. [1]

Nach einem einstimmigen, postiven Echo aus dem Ausschuss für Verwaltung und Gleichstellung, stimmte am 24.04.2013 die Bezirksverordnetenversammlung Neukölln für die Installation von Freifunk-Routern auf dem Rathaus-Gebäude [2]. Der Antrag im Ausschuss für Verwaltung und Gleichstellung ging auf eine Initiative der Piratenpartei zurück.

Der Bezirk stellt den Ort, sowie den Strom für die Freifunk-Hardware zur Verfügung. Die Kosten für Baumaßnahmen und Router konnten aus Mitteln einer Förderung der Medienanstalt Berlin Brandenburg (mabb) gedeckt werden [3]. Die Installation und Wartung der Netzwerktechnik erfolgt ehrenamtlich durch Mitglieder der Berliner Freifunk-Community.

Eine der insgesamt 14 installierten Antennen funkt zum ca. 3.8 km entfernten Rathaus in Kreuzberg, auf dem bereits im Oktober 2013 Router installiert worden waren. Beide Standorte sind Teil des sogenannten BerlinBackBone (BBB), einem WLAN basierten stadtweiten Mesh-Netzwerk über den Dächern Berlins, das Dank der finanziellen Unterstützung durch die mabb derzeit von der Freifunk-Community intensiv modernisiert und ausgebaut wird. Neben den beiden Rathäusern befinden sich darunter auch bereits mehrere Kirchen und Hochschulgebäude.

Über diese Relaisstationen sind die lokalen Kieznetze (Meshwolken) der Freifunkenden untereinander und mit dem Internet verbunden. So ist es möglich, innerhalb der Stadt auch über größere Entfernungen drahtlos untereinander zu kommunizieren, ohne auf Netze der kommerziellen Provider angewiesen zu sein.

Die Zugänge zum Internet werden innerhalb des Netzes dabei sowohl von Privatpersonen geteilt [4] als auch über den Förderverein Netzwerke e.V. und andere Partner bereitgestellt. Besonders der IN-Berlin e.V. und neuerdings auch de Berlin Commercial Internet eXchange (BCIX) e.V. stehen hier tatkräftig mit Know-How und Bandbreite zur Seite.

Aufgrund der vielen positiven Erfahrungen, wie aktuell auch in den Rathäusern Kreuzberg und Neukölln, freut sich die Berliner Freifunk Community darauf, weitere Installationen auf und in anderen öffentlichen und privaten Gebäuden durchzuführen. Freifunk Berlin sucht dazu neben weiteren Standorten auch noch Unterstützerinnen und Unterstützer, die bei der Verwirklichung eines stadtweiten freien Netzes helfen wollen! Weitere Informationen finden sich auf der Website http://freifunk.berlin.

[1] http://wiki.freifunk.net/Berlin:Standorte:Bezirksamt_Kreuzberg

[2] http://www.berlin.de/ba-neukoelln/bvv-online/vo020.asp?VOLFDNR=3790&options=4

[3] http://www.mabb.de/information/wlan/public-wifi.html

[4] https://digitalegesellschaft.de/portfolio-items/storerhaftung-beseitigen/#gesetz

Freifunk Wireless Community Weekend 2014 at c-base Berlin, May 30 – June 1

Community mesh networkers from around the world are meeting in May 2014 at the c-base in Berlin.

There are Pre-Meetings May 28 and 29, the „official program“ starts 30th and last until June 1st. In fact the final version of the agenda will be created by attendents on-site. If you add your idea or wish to our wiki upfront, we can take care of your talk or workshop. Hacks sessions are taking place over the whole weekend.

This year we focus on topics like crypto, IPv6 and new services, but also like always latest firmware, new installation setups and last not least legal and political update plus public relations. Following the tradition we’ll have barbecue in the evening hours at Spree beach and we don’t charge entry fee. For more information and taking part, please click here: Wireless_Community_Weekend_2014

when

  • 30 May till 1 June 2014 at c-base berlin

where

inside locations

  • main hall
  • c-base beachpark

Freifunk-Erlebnisbericht Netze über Berlin

Der Freifunk-Erlebnisbericht “Rooftop Bombing mit Netzwerkknoten in Berlin” von Boris Niehaus im Magazon vice.com gibt einen Eindruck vom Bauen von freien Netzen über Berlins Dächern.

Freifunk Berlin

“Wir betreten das Gebäude und treffen im Foyer den Chef der IT­ Abteilung des Rathauses, mit dem wir in die 10. Etage fahren und durch eine Luke aufs Dach klettern. Mir schlägt der Wind ins Gesicht und Herr Zachler, der Rathaus-IT, schützt sich mit einer Plastiktüte gegen den Regen, mit Kabelbindern befestigt. Die FREIFUNKER springen ziemlich routiniert auf dem Dach rum, verlegen Kabel und richten insgesamt 14 Antennen aus. Diese bauen eine bis zu 5 Kilometer lange Funkverbindung zu anderen großen Knotenpunkten, wie dem Zoofenster in Charlottenburg oder den Arkaden in Neukölln, auf.”

“…ein selbstverwaltetes Netzwerk hat viele Vorteile. Es liefert eine unabhängige Infrastruktur, die auch dann noch funktioniert, wenn unser Internet durch ein Unwetter ausfällt, oder wenn das Netz wie in totalitären Regimen einfach mal abgestellt wird. Du kannst dich mit deiner Nachbarschaft vernetzen, Community­-Radios einrichten, Netzwerkspiele spielen. Ein paar wenige Internetzugänge können zudem einen ganzen Stadtteil versorgen. Nicht schlecht für sozial schwächere Gegenden, insbesondere da Harz­-IV­-Haushalte vom Amt immer noch kein Geld für einen eigenen Internetzugang erhalten.”
http://www.vice.com/de/read/rooftop-bombing-mit-netzwerkknoten-in-berlin

Freifunk verbindet! Trefft uns auf dem Wireless Community Weekend vom 10.-12. Mai 2013 in Berlin

Philipp Seefeldt hat auf Vimeo den Animationsfilm Freifunk verbindet! veröffentlicht. Die Idee dazu kam von Sven Heinze und Juergen Neumann. Beim Text half Silke Meyer, gesprochen hat Anne Helm und die Musik wurde von Thomas Deittert geschrieben. Ihr könnt das Video gerne auf Eurer Website einbinden. Es steht unter einer freien Creative Commons Lizenz (SA-By) 

Wenn ihr mehr über Freifunk und Freifunker aus Deutschland und anderen Ländern erfahren möchtet, kommt zum Wireless Community Weekend in die c-base nach Berlin vom 10.-12. Mai. Am 8. und 9. Mai finden bereits informelle Treffen statt.

Wann

  • 10.-12. Mai 2013

Wo

Mehr Infos

Mehr Infos im Wiki hier: http://wiki.freifunk.net/Wireless_Community_Weekend_2013

Freifunk WLAN in und auf dem Rathaus Kreuzberg: Freier Internetzugang trotz Störerhaftung

Besucher des Bürgeramts im Rathaus Kreuzberg haben über ein neu eingerichtetes Freifunk-WLAN ab sofort kostenlosen, freien und zeitlich uneingeschränkten Zugang zum Internet. Neben einem Access Point im Warteraum in der dritten Etage wurden auch auf dem Dach des elfstöckigen Gebäudes drei WLAN-Router installiert. Diese Router haben durch Richtantennen eine Reichweite bis zu zehn Kilometer und stellen Verbindungen zu dem bereits über den Dächern von Berlin bestehenden Freifunk-Netz und den darin angebundenen Internetzugängen her.

Die Bezirksverordnetenversammlung beschloss im Frühjahr den Start eines WLAN-Pilotprojekts, welches dann von Freifunkerinnen und Freifunkern um André Gaul zusammen mit dem Bürgerdeputierten Andreas Pittrich realisiert wurde. Unterstützt wurde das Projekt durch Dietmar Zachler von der Rathaus-IT und Stadtrat Hans Panhoff. Alle Geräte wurden von Freifunk installiert, das Bezirksamt stellt die Aufstellorte und Strom für die Geräte zur Verfügung.

Die Zugänge zum Internet werden von Bürgerinnen und Bürgern innerhalb des Freifunk-Netzes bereitgestellt. Durch das Bereitstellen eines Internetanschlusses geht man jedoch ohne weitere Maßnahmen wegen der in Deutschland noch immer geltenden Störerhaftung ein juristisches Risiko ein, da man für die Durchleitung von fremdem Internet-Verkehr haftbar gemacht werden kann. Um die Störerhaftung zu umgehen, leitet Freifunk die Daten verschlüsselt durch ein VPN über Schweden und zeigt damit auch die Unsinnigkeit der Störerhaftung auf. Die gleiche Technik benutzt auch die Freifunk Freedom Fighter Box, die seit Juni 2012 kostenlos verteilt wird, um freies WLAN an öffentlichen Plätzen bereitzustellen und gegen die Störerhaftung zu protestieren.

Freifunk unterstützt den Gesetzentwurf der Digitalen Gesellschaft, der Bürgerinnen und Bürger sowie Gewerbetreibende, die einen Internet-Zugang via WLAN anbieten, mit kommerziellen Internetprovidern haftungsrechtlich gleichstellt. Aufgrund der durchweg positiven Erfahrungen im Rathaus Kreuzberg hofft Freifunk, bald weitere Installationen auf und in anderen öffentlichen Gebäuden durchführen zu können.

[via Juergen Neumann]

Links

Dokumentation: wiki.freifunk.net/Berlin:Standorte:Bezirksamt_Kreuzberg

Beseitigung der Störerhaftung, Gesetzentwurf DigiGes: digitalegesellschaft.de/portfolio-items/storerhaftung-beseitigen/

Freifunk Freedom Fighter Box: freifunkstattangst.de/2012/06/14/aktion-gegen-storerhaftung-anonym-im-wlan-an-offentlichen-platzen-mit-freifunk/

BVV Beschluss: www.berlin.de/ba-friedrichshain-kreuzberg/bvv-online/vo020.asp?VOLFDNR=4948&options=4