The original plan was a full kernel-space SOCKS proxifier, but that would be a little bit complex for the goal: a faster TCP proxy. Then I found a very elegant solution for the problem: eBPF sockmap support. There is a API for redirect packets between sockets in kernel space using sockmap eBPF program. I decided to extend my shadowsocks-libevfork with the eBPF support. The disabled encryption already give some additional performance, so if anyone already using this one, there is a new option to get more performance.
Last month I introduced my test setup intended for fast kernel trials and network development. After that updated my shadowsocks-libev fork for the latest 3.2.0 version which is the latest upstream stable version. This fork dont do any encryption which is not so secure but faster – and in our new approach: we can put the data plane into the kernel (because we cant do any data modification in the userspace).
The problem emerged in a different environment recently: at the cloud/datacenter scope. In the cloud transmission between containers (like Docker) happens exactly like in our SOCKS proxy case: from user to kernel, than back to user (throught the proxy) than back to kernel, and to user. Lots of unnecessary copy. There was an attempt to solve that: kproxy .This solution is working pretty well, butthere are two drawbacks: not merged into the kernel (the main part is a module, but also modifies kernel headers) and in my testsit is slower than the regular proxy with the extra copies. Sadly I dont know the exact problem, but with my loopback tests on a patched 4.14 kernel were about ~30% slower than a regular proxy.
The kproxy is currently AFAIK not in development anymore, because featuring TCP zero-copy there is a better solution with zproxy, but its not released yet. But some part of the original kproxy code is already merged into the kernel part of the eBPF socket redirect function: https://lwn.net/Articles/730011/
This is nice because its standard, already in the vanilla 4.14 kernel, but a bit more complicated to instrument it, so I will test it later.
The backup solution if none of them works the I will try it with netfilter hook with the skb_send_sock function, but that version is very fragile and hacky.
I decided to give you a brief intorduction to the development of my testbed. In the past month most of the time I experimented with different virtual environments for kernel development. The pros of virtualization:
Fast test cycles: multiple virtual machine (VM) can use the same, freshly compiled kernel
No physical devices, you dont have to reboot your machine every time when you want to test your recent kernel changes. VMs reboots very fast (about 6-7 sec in my current setup)
Flexible network virtualization: you can connect your VMs with virtual ethernet links to virtual switches
My current worflow looks like this:
1. Make changes in the kernel code or configuration (make menuconfig or .config file)
2. Compile the modified kernel
3. Boot the virtual machines with the new kernel
4. Test if works, debug, etc.
5. Goto 1.
In the following you can find a detailed intro how to setup the kernel development and test environment with QEMU and virtual networking
Welcome! I’m Ferenc Fejes from University of Debrecen, Hungary. In 2017 I did my first GSoC project with my mentors Benjamin Henrion and Claudio Pisa. You can find all of my post from the project on this link here: https://blog.freifunk.net/tag/mptcp+lede/
In a nutshell it was an experiment to create a test-bed for aggregating the speed of multiple Wi-Fi links in transport layer with MPTCP. We succeeded, I also provided a detailed tutorial with the method for reproducing the experiment in home. I did not expected too much enquiry about the project but after the blogposts and an OpenWRT summit workshop, I received lots of emails and messages – many experimenter interested about the project which is a great honor of me. In China, where the router proxyfication is necessary because of the great firewall, many people applied the method of my GSoC 2017 project outcome. (Cont.) Continue reading “GSoC 2018 – Kernel-space SOCKS proxy for Linux”→
In the first post (beginning of the GSoC 2017 project) I made a few checkpoints to complete at the end of the summer. Now I don’t copy them here, but the good news is all of them completed successfully. The main goal was a very simple transparent multipath Wi-Fi link bandwidth aggregation. The proof in the video above. And the details below.
After lots of experiments, I figured out the way which enables the transparent MPTCP operation for both the client and the server (with the regular kernel).
I tested the configurations at my virtualized environment, check the video below. For configuration infos, please read the video description. (the post continues)
For more info whats the goal of the project, please visit my first blogpost which is here.
What done yet
I created this simple topology for trying out the technologies. I’m still in experimenting status but have some results as well. This virtualized test enviroment run on my machine. The paths between the MPTCP supported machines are shaped to 10 Mbps. With iperf3 and Wireshark I verified that the MPTCP protocol is working fine: clearly shows 20 Mbps between the two MPTCP supported machine (I use the latest 0.92 MPTCP kernel which is based on the 4.4 kernel version). Tha path between the MPTCP and Endhost machines are not shaped, so the throughput is much higher than 20 Mbps. I configured shadowsocks-libev on MPTCP1 (ss-server) and Endhost1 (ss-redir) machine. Then I tired an iperf3 download between Endhost1 and MPTCP2. This looks like:
Endhost1 <——————]TCP[——————-> MPTCP1 <———————]MPTCP[——————–> MPTCP2
I successfully get the 20 Mbps on Enhost1 which is the aggregated throughput of path1 and path2. (continue)
My name is Ferenc Fejes. I’m MSc CE student at the University of Debrecen, infocommunication networks specialization. I glad to introduce my project which is Add MPTCP support in LEDE/OpenWRT. The main idea is from my mentor, Benjamin Henrion. If we have a Wi-Fi mesh network, there are probably more than one network path between the nodes. The point is, we can gain additional throughput if we use those path simultaneously.
Currently no MPTCP support in GNU/Linux nor Windows systems. So the key is, the user using regular TCP/UDP at his system until the first hop, which is a LEDE box with MPTCP support. At this point, this router operating as an intercepting proxy: read the incoming traffic from the host, and distribute that on multiple interface, using MPTCP. At receiver side, we can do the same but in reverse order: get the data from the MPTCP connection and send is to the sink host in regular TCP (or UDP). With this operation, the whole MPTCP transmission remains hidden for the end hosts, but they will enjoy the gain from it – because MPTCP has a good capability to aggregate the bandwitdh of the different paths. Of course, if the bottleneck between the end host and the first hop, the user did not get any gain from this operation.
The key steps for reach the goal of the project:
1. Set up virtual/real machines with MPTCP support. This is not a problem since kernel patches for MPTCP support available.
2. Make a simple multipath topology: end hosts with no MPTCP support, two machine with MPTCP support, functioning as a router and has two different network path between them
3. Search and configure an efficient and transparent proxy software on them, which can suitable for intercept TCP traffic and work as a tunnel for other protocols (like UDP, SCTP, etc.)
4. Reproduce the working configuration on LEDE supported devices, with two Wi-Fi bridge links between the LEDE boxes.
5. Verify the aggregation with iperf3 measurements.