Tag: berlin

This summer I worked on an Easily Expandable Wireless Intrusion Detection System (called Eewids). In this blog post I’d like to present what I’ve done, the current state of the project, the lessons learned and what needs to be done in the future.

Projects repository on GitHub: techge/eewids

What I’ve done

Analyzing existing projects

Before actually starting this project I analyzed existing Free and Open Source Software (FOSS) connected to IEEE 802.11 standard. I achieved this in part by looking through all projects listed on the extensive wifi-arsenal of the GitHub user 0x90 and categorized most of the entries myself.

I realized that there just is no complete ready-to-go WIDS yet, at least regarding FOSS. For me a WIDS should

• detect most of the known Wi-Fi attacks,
• scale easily and thus be able to work within big organizations and
• be easily expandable.

Although there is indeed software on GitHub which can be used to detect Wi-Fi attacks, they are usually specialized on some attacks and/or they are hobby projects which would not fit in setups of bigger environments. Please have a look at the defence-related Wi-Fi tools on the wifi-arsenal list.

A distributed, highly scalable system

Based on a proposal of my mentor Julius Schulz-Zander, I created a framework with a microservice approach. Instead of creating a monolithic solution, Eewids is supposed to consist of different well-known and actively developed software. The central point of Eewids is a message broker. The advantage is an independence between different system components. New detection methods can be added to the system without actually bothering about the capture process, the parsing or even the presentation of the results. Everything leads to the message broker.

RabbitMQ is an awesome choice for such a message broker. It is not only most widely deployed, but it also supports a lot of programming language. No matter if you want to add a detection method using C, python or Java, there already exists a library to easily attach to RabbitMQ, get the captured and parsed data and sent the results back (see RabbitMQs devtools).

The message broker also helps to make the whole system highly distributed and scalable and thus helps to fulfill the requirements to work for bigger organizations as well. Many organizations may already have experiences with RabbitMQ-Clusters.

System components

Eewids consists of:

• a capture tool to get the actual 802.11 data packets
• a parser to transform the information into simple json-formatted data
• a message broker (RabbitMQ)
• detection methods
• a visualization

The project aimed to find a solution for all components.

The capturing should be done by Kismet, which turned out to be a bad idea (see section “Lessons’s learned”). Therefore, a simple capture tool was created, which will be further improved (see section about future work). For parsing the package a parser was created (“eewids-parser”) in part based on the existing project radiotap-python. As kind of proof-of-concept the visualization software Grafana was added (in tandem with InfluxDB). This turned out to be a very good choice and is now integrated as the main visualization in Eewids. Grafana connects easily to various systems, so that other data withing a company may be connected and visualized as well.

Lesson’s learned

At first, I didn’t dare to create a capture tool myself. I thought that this would take a lot of programming experience. Therefore, I have chosen Kismet as a basis for capturing the 802.11 data. Kismet is well-known and very popular. Still, it does not fulfill the above requirements – it is not a full-featured WIDS, it is monolithic and the code proved to be difficult to read/change. I invested a lot of time to integrate Kismet in Eewids, to build docker container (which did not exist before), to read out pcap export from the REST interface etc. In the end I integrated a feature-rich system to only use one function – the capturing. After several crashes of the Kismet server and an analysis of the network performance, I decided to try to create a simple capture tool myself (see this blog post).

It turned out that it is not that difficult to create a simple capture with help of libpcap. Although the capture tool is still very rudimentary it already fulfills the function which Kismet served before. While I still think, that it is always a good idea to have look at existing projects first, I would trust more in my skills next time, to avoid wasting to much time in integrating a system which is not suitable for the project at stake.

What needs to be done in future

While the framework itself is ready and works fine, there is still a lot to do. The capture tool should be further improved as described in this blog post. Getting used to libpcap needed more time than anticipated, so that I was not able to include all features yet.

The parser should get extended depending on upcoming needs. For example, only some element ids are parsed for now. All main 802.11 fields are parsed though. See the projects page for an overview of all parsed data.

More dashboards for Grafana could be added to visualize as much data as possible. Furthermore, some useful notification could be added by default.

Most of all, Eewids should get more actual detection methods in the end. The RogueAP detection that was implemented during the summer is a start, but served mainly as a proof-of-concept and shall be improved. As Eewids is supposed to be easily expandable this last objective should in part get solved by different persons in future.

In the end I am eager to further improve the system even after the end of this Google Summer of Code. I am glad that I had the opportunity to work continuously on this project. Now I am more confident in my skills than before. I’ll surely have less doubts to actually start a project in the future.

In this blog post I’d like to present the recent progress made for Eewids. This time, our main focus was the performance of the current setup. Besides we did some minor improvements, mostly adding some more dashboards to Grafana to visualize the data captured. This blog post focuses the results regarding Kismet as a component of Eewids. Continue reading “GSoC 2018 – Easily Expandable WIDS – Second Update” →

In this blog post I’d like to present the recent changes made in Eewids, why they were done and what’s to come next. For an introduction of Eewids see here.

In general the steps done the last weeks aimed mainly at the easiness of use and testing the main concept – having an easily expandable framework at hand. Thus, a RogueAP detection was added and visualization based on InfluxData tools and Grafana were included. Both steps were much more easy to achieve because of the architecture of Eewids.

Starting Eewids most easily

For everyone potentially interested in using Eewids it would have been a big hassle to compile Kismet (git development version) by herself. As Eewids is completely based on Docker container most of the components didn’t need to get installed. And that’s quite important. No one wants to compile, start and administrate all the stuff: Kismet, Eewids’ Parser, RabbitMQ, InfluxDB, Telegraf, Grafana and finally the plugins added to Eewids (like the RogueAP detection, see below). While all these components are provided by Docker container and can get started by simply hitting ‘docker-compose up’, the Wi-Fi card had to get accessed directly so far. Therefore, it was necessary to have a recent version of Kismet’s remote capture, which is not included in any major Linux distribution yet.

Luckily Kismet’s developer found a solution to this problem and documented it. We adapted this to the needs of Eewids and now have a solution in which one can start Eewids easily on a local machine, needing nothing more than a compatible Wi-Fi card, docker and docker-compose. Please see the getting-started.md of Eewids for more information and try it yourself! 😉

Renaming fields of captured data

To make the captured data of Eewids as accessible as possible for developers many field names saved in the message broker RabbitMQ were changed to be quite similar to Wireshark’s “Display Filter Reference”. See here.

Hearing Map for RogueAP detection

A simple RogueAP detection which existed before have been expanded by a hearing map. Now a whitelist contains not only valid ESSID:BSSID pairs, but also the information which remote capture is able to see which AP. Thus, an attacker can not use a valid ESSID:BSSID pair of a AP which is located in a different building to cover an EvilTwin attack. See here for more information.

Add a visualization tool: Grafana

We develop Eewids to make it easy to add new functions to it. To test this claim and to actually extend functionality by a way to analyze and visualize what’s happening arround, we added Grafana. It connects easily to different datasets (like InfluxDB, Elastic etc.) and let you create graphs and lists etc. As a starting point we added InfluxDB to save our captured data, Telegraf to get the data out of RabbitMQ and to send them to InfluxDB and Grafana to use the data from the InfluxDB.

Which would have been a hassle to implement on a local machine was quite easy with docker and a already existing dataset provided by Eewids in RabbitMQ. Thus, it only took us some hours to find out how to use this software. Even this time was not related to Eewids itself, but just to the missing basic understanding of Telegraf, InfluxDB and Grafana. That is to say if anyone who already know these tools would have liked to add these to Eewids could have done this easily. And this is the objective of Eewids.

We consider this a successful proof of concept. We used InfluxDB for Grafana, because we expect new things to come which depends on/use InfluxDB. Likewise we can imagine the fast and forward implementation of Elastic and the related tools and software. We’d glad to see this adapted in the future as well. 🙂

What comes next?

Now that we have a visualization tool (Grafana) added, it would make sense to extend it with more information, letting alerts visualized etc. Furthermore, we’d like to improve the “backend” features for developers. That means we would like to create some templates to easily start using Eewids data and adding detection methods. Let’s see how it works out!

Hello,

I am Alex and I want to create a framework for making an easily expandable wireless intrusion detection system this summer. The objective is to create a working environment which can be expanded with microservices to detect attacks on Wi-Fi networks and which fits easily within rather large organizations instead of small private setups.

All the things are happening on GitHub and thus this introduction is based on the README I created having this blog post in mind.

Background

Existing WIDS Tools

Analyzing 0x90/wifi-arsenal especially in search of wireless intrusion detection systems (WIDS) I realized that there just is no complete ready-to-go solution yet, at least regarding free and open source software (FOSS). For me a WIDS should serve the following needs:

• detection of most of the known Wi-Fi attacks
• scalability and thus being able to work within big organizations
• simple expandability (there are always more attacks to come ;-))

Although there is indeed software on GitHub which can be used to detect Wi-Fi attacks, they are usually specialized on some attacks and/or they are hobby projects which would not fit in setups of bigger environments. Please have a look at the defence-related Wi-Fi tools on the wifi-arsenal list.

An exception should be mentioned: Kismet. It is probably the most famous and complete FOSS Wi-Fi solution and very popular. Still, it does not seem to fulfill the above necessities completely. And it is probably not the objective of Kismet to be a full-featured WIDS either. Instead it has many features for pentesting Wi-Fi networks and other interesting stuff.

Why Not Just Expanding Existing Programs?

One solution would be to simply add needed functionality to Kismet. And this is definitely a good idea and I encourage everyone to improve the code of Kismet. Some needs mentioned above could be solved with a microservice approach more generally though. This is exactly what EEWIDS tries to achieve. By creating a containerized framework EEWDIS enables

• scalability
• working easily in setups of bigger organizations
• the possibility to add functionality easily (see below)

Main Idea of EEWIDS

Basis Kismet

EEWIDS uses Kismet as a basis. Thus, it uses Kismet’s advantages and tries to add functionality by using container techniques. As Kismet is under heavy development right now, EEWIDS uses the git version of Kismet right away, which is completely different to the last release from 2016. The Kismet remote capture (which replaces the former Kismet drone) is the only piece of software, which can not be containerized. The Kismet remote capture has to run on the machine which contains a Wi-Fi card which is able to monitor the traffic. As Kismet is very popular the Kismet remote capture will already run on many different machines and platforms, e.g. on OpenWrt. Therefore, it is better to use Kismet as a basis for capturing the data instead of building an own system.

The Kismet remote capture will send the data to a Kismet server instance which is running in a container. By using the Kismet server we will be informed about every attack which Kismet did detect and thus we can reuse the work already done on this side. EEWIDS will attach to the Kismet server to:

• pull the pcap-ng data stream which contains all data captured
• pull all alerts raised by Kismet server itself

Message Broker RabbitMQ

Both kind of information will be parsed and submitted to a Message Broker afterwards. The Message Broker is the central point of EEWIDS. By using RabbitMQ – one of the most popular systems of its kind – it is easily possible to subscribe to a needed information. This is supposed to be the big advantage for developers. Thus, instead of capturing and parsing Wi-Fi packets itself, a detection method only needs to subscribe to the needed information and will receive it directly from the Message Broker. Furthermore, the developer can use any programming language or system which is needed for this kind of detection, without bothering C++ or other stuff, which may would be necessary for Kismet plugins.

Analyzing and Visualization

The actual analyzing is done in services dedicated to this task. E.g. instead of parsing packages, looking for Beacons and analyzing it afterwards, a service will just subscribe to all Beacon frames. All other frames are not of interest. The service does not need to parse the Beacon frames, it just needs to access the json-formatted information it got from the Message Broker, e.g. data[‘wlan.ssid’] or data[‘wlan.bssid’]. This can be done independent of the programming language, as most of them already have modules for json and are able to access RabbitMQ. This setup should indeed work for every language which already has a client listed on RabbitMQ website.

Another advantage is the freedom of choice of visualization/analyzing software. It is easily possible to include either influxdata’s TICK stack or the elastic stack, both Open Source analyzing software which also have anomaly detection methods. These stacks and other software already have interfaces to access RabbitMQ and to read json-formatted data and thus it is easy to extract the collected information as needed.

This should make it easy to extend EEWIDS in various ways. Let’s see what can happen.

Focus

The usability on a developers perspective depends on the availability of logged frame information actually stored in RabbitMQ and the existence of easily adaptable templates. Furthermore, it has to be as easy and straight-forward to deploy the system as possible. That’s why I’d like to focus on three things:

• the parsing of Kismet’s pcap-ng files should be as complete as feasable
• there should exist templates for some major programming language to describe the usage
• the deployment should work straight forward