Author: Tillmann Gansky

GSoC: Retroshare social network plugin review and future

Posted on by Tillmann Gansky

During Google Summer of Code i did these things:
– learn what users expect from a social network. Figure out who should receive which messages.
– map social network functions on the existing Retroshare General Exchange System
– build a backend with basic features
– build a frontend with basic features
– started a new programming interface on top of libretroshare: a JSON over http interface

The Secushare homepage says: “RetroShare should provide more social functionality” Indeed! Every country is spying on their citizens. Storage, computing power and bandwidth became cheap. These are good conditions to build a distributed social network. The first step was already started a while ago with the General Exchange System for Retroshare v0.6. This project is the second step in this direction. One thing is missing: a release for normal users.

The backend and frontend made during GSoC can display a wall with avatar image and profile text. It is possible to create posts and reference them on walls. Posts can be commented. Read more details in the previous blog post. Still it is not suitable for daily use. There was not enough time to implement a scalable user interface architecture. The web interface can’t handle more than 100 posts, because pagination is missing. Whats more the UI is filled iterative. This causes many updates to the html tree with high cpu load. It would be better to send all information to the browser in one piece. The browser would then only have to update the HTML tree once. The design and layout should be improved to highlight the content and to get the date label out the way. (See the this screenshot.)

Future

Wt is nice because it allows to make a web interface without touching HTML JavaScript and CSS. But this is only the half truth. There where issues:
– completely destroyed layout by setting the image size
– menu bar was horizontal instead of vertical, this required to manual set a style class from C++
– WTimer stops working. I had to build my own server side timer

These issues where solvable. A real pain with Wt is the layout and UI design testing. You first have to go through the complete compile and start cycle to see changes. What if you missed a closing tag in a HTML template? You have to recompile and restart. Now i saw that with real web technologies you can see a live preview while you type. I think when doing layout and design it is important to immediately see the result. This is not possible with Wt. Conclusion: you can build a web interface with C++. But hard coding CSS class names and embedding HTML snippets in C++ is a pain that should be avoided.

On the other hand there exist advanced tools and frameworks for web development. AngularJS is a very nice JavaScript library. It offers data binding from JavaScript to HTML: you update a JavaScript object, and Angular updates the HTML. JavaScript objects are more flexible than C++ objects. You can even make a JavaScript object out of a JSON file. Angular makes handling of button clicks easy. AngularJS makes  Bootstrap offers a set of widgets with HTML example code and style sheets. Now i want to use other web frameworks instead of Wt.

I did not know about these web technologies when starting the project. This is some sort of a chicken-egg problem: Retroshare does not have web developers, because it doesn’t have a web interface. And there will be no web interface until a web developer can show how it can be done. Gladly there is now a web developer who can teach me how to make a web interface.

The new idea: make a web interface using web technologies

The next goal is to make a clean and easy to use JSON over HTTP api for Retroshare. Then web developers are free to make a nice web interface using their favorite frameworks. This api is not only useful for web interfaces, but also for scripting. You could then send a chat message from a shell script using curl:

curl -X PUT -d ‘{“msg”:”hi all<br/>(send from bash)”}’ http://localhost:9090/api/chatlobbies/<id>

What i will do now
I will do more research for a JSON over HTTP interface for Retroshare and rssocialnet. If this stays a good idea i want to implements it. Maybe i will try to start a new web interface. But i hope someone else is faster than me. I prefer to work on the rssocialnet backend and libretroshare. Whats more a web interface is a perfect place for new contributors. If i would make a clearly documented JSON over HTTP api, then a web developer would not need C++ knowledge or experience with libretroshare.

A dream is to bring Retroshare to Android. This is possible, but we have to make a new touch-friendly user interface and we have to optimize Retroshare to make it more battery friendly. I currently see two ways to build a GUI for Android: with QML/QtQuick or HTML based with the Ionic Framework. There is already a QML prototype.

To get rssocialnet ready for daily use, i need your help. Unfortunately I’m again in a research/planning phase. This means I’m not sure how the JSON over HTTP interface will look at the end. It also means if you tell me “i want to help coding”, i have to disappoint you because i don’t know what we have to code and how to code it. Anyway, here are some things you can think about:

– which features are important for a social network? read some ideas
– can you make a better gui mockup than me?
– which frontend technologie should be used? QML/QtQuick or Bootstrap and AngularJS?
– what are the requirements for a first release?
– how can you use your skills to help?

Thank you Freifunk and other mesh network communities for donating one GSoC slot to Retroshare. This was a good decision, it made it possible to start a mesh friendly social network application.

GSoC: Features of the Retroshare social network plugin

Posted on by Tillmann Gansky

Content is obviously the most important element of a social network. Currently only support for plain text is implemented. The content can have an author, but this is not required.

Future: It would be nice to have support for images. This is very easy on the backend side, but it needs a frontend which scales the image to fit on the screen. With Retroshares file transfer capabilities it would also be possible to publish large files like audio and video files. It would be nice to restrict access to content to a set of people. Retroshare is prepared for this and it only needs small changes in the social network backend. Of course this requires a user interface to sort people into circles and to select circles.

Content alone is useless without a place where it gets displayed. As explained in a previous blog post, every piece of content is stored for its own. To make content visible it has to get referenced on a wall. This happens automatically on the own wall when creating a new post. It also gets triggered by clicking the share button. A reference always has an author.

Future: maybe allow to reference content from other services. For example if Retroshare gets a Photo Share service, allow to reference a picture or photo album on the wall.

A Wall is a place where a profile text, an avatar image and references to content are stored. A user subscribes to another user to download all posts referenced on the wall. The wall owner and others can reference content on a wall.

All new posts are collected and displayed in the news feed. A news feed shows the new content, the comments and how others interact with the content. Who commented this post? Who shared this post? Currently the news feed displays posts in the order in which they where received.

Future: it is probably desired to have a more advanced logic to sort news feed entries. Imagine a user comes online after a week and gets bombed by hundred new posts. It would be possible to sort news in two dimensions: topics and rating. Example: have one tab for content from close friends, and one tab for other content. Then calculate a score to display more important content at the top. This requires a bit of backend work, but it is doable.

Users can interact with content in two different ways: they can comment it, and they can share it. Sharing creates a reference to the content on the own wall, and thus forwards the content to friends. Comments are stored with the content, so everyone who received the content will also receive the comments.

Future: one can think of other ways to interact with content. Examples are like, bookmark, vote and hide. In general these interactions are each a form of tagging. For the backend it does not make a difference if content is tagged with “GSoC14” or “like” or 3.1415. This is more a matter on the frontend side: which meaning does the tag have for the user? How does the frontend show different tags? (star, heart, thumbs up, plus sign, text, …) How can the user filter posts with specific tags?

There has to be an entry point to let the user see the people around him. If the user recognizes a known person he might want to subscribe to this person. For now there is a widget to display all identities with their name and avatar image. Of course later this list should get filtered to fit on the screen. Retroshare circles could be used to make friends lists accessible to friends. This would allow automatic circle intersection to search for people the user probably knows.

Below is screenshot of the Retroshare social network plugin.

GSoC: Retroshare social network plugin compared with Retroshare forums

Posted on by Tillmann Gansky

In the last post i wrote about the decentralized structure of Retroshare. I said: similar features to Facebook are also possible in a decentralized manner. Now let’s have a closer look at those features.

Communication tools have the purpose to take a piece of information and transmit it. The question is who should receive it, when will it be received and to which next hop should the information be send.

Very simple forms of communcation are one to one messages. We have a clearly defined receiver which can be adresse by a RSA public key. This is very simple if we are directly connected to the receiver. If the receiver is not a direct neigbor, then Retroshare uses it’s distant tunnels. Current distant chat and distant messages only work when sender and receiver are online at the same time. Retroshare v0.6 will cache messages on neighbor nodes to deliver them when the receiver is online.

Retroshare can also do many to many communication. The Chatlobbies are for real time communication. If you don’t receive the message a few minutes after it was send you will never receive it. Messages are send to all neighbors in the chatlobby. RS keeps messages for a few minutes to prevent echos. Forums and channels offer persistant messages. This means messages are kept for a year in forums and a month in channels. The messages are not targeted at a person, but they belong to a context. This context is a chatlobby or a forum/channel. Users have to subscribe to the context.

Every message is bound to the context. It appears only at one place. In Retroshare forums and channels the context is a topic like “Developer’s Discussion” or “RetroShare Windows builds by Thunder”. In the social network plugin the posts are decoupled from a context. The context only stores a reference to the post. This allows posts to appear at different places. For example: a post can appear on the walls of user A and user B. And if i see the post and like it, i can forward it to my wall. Forwarding of specific posts only, increases the quality of the received content.

With forums and channels you first have to subscribe to them. Retroshare will then download the posts in it. The social network plugin automatically downloads the posts your friends like. And if your friends like it, then the probability is high that you also like it.

While forums are about a topic, a social network wall is about the user. It is a personal place where you can leave message for your friends, and your friends can leave messages for you. Still your are not limited to own content. If you want to share existing posts with your friends, just put it onto your wall.

Scope of the GSoC project
The goal of the GSoC project is to make a basic wall service. This includes the backend for message processing and the user interface. Chat and private messages are not planned for GSoC. But this is something that is very easy to add later.

Internally the social network plugin is based on the new Retroshare General Exchange System(GXS). GXS has two basic data structures: gxs-groups and gxs-messages. A gxs-message belongs to a gxs-group. If a gxs-group is marked as subscribed, then all messages in this group are downloaded. In the gxs-forums this mechanism is clearly visible to the user: gxs-groups are downloaded automatically, but the user has to subscribe to download the contained gxs-messages. The posts in the social network plugin should be independent. This requires that every root-post has its own gxs-group. The wall services automatically subscribes to interesting gxs-groups. Gxs-groups can be interesting because:
– the posts appears on a wall of a friend
– the user follows the author of the post or the user is friend with the author
– a friend made a comment on this post

So the interesting content is not defined by a forum title, but by the way friends interact with it.

GSoC: Social network plugin for Retroshare

Posted on by Tillmann Gansky

Retroshare is a communication tool build on two principles: decentralisation and privacy (Read more about the ideals behind Retroshare). Decentralisation means there is no central server. It also leads to privacy, because Retroshare runs on your own devices. Retroshare gives you the cryptographic tools to control who exactly can access your data. Privacy is enabled by encrypting all data which goes over the wire. Read Cryptography and Security in Retroshare for details.


Figure 1: Retroshare is a decentralized network.

Commercial social networks like Facebook have opposite properties: they are centralized and all data is visible to one company. The good news is: similar features to Facebook are also possible in a decentralized manner. The plan is to build this as a plugin for Retroshare. Always having decentralisation and privacy in mind, while offering a very similar interface with a similar feature set.


Figure 2: Sketch of the user interface.

Unlike other peer-to-peer software, Retroshare connects only to a set of trusted friends. This has some practical advantages, for example you don’t have to fear malicious peers. It still scales nicely, because friends tend to gather the same type of data.
But often you want to reach content that was not created by your direct friends. Or you want to contact someone because you are not friends yet. For this Retroshare offers another layer which i call the service layer. At first a service can send messages to direct neighbors only. But if it receives a data packet it can also forward it to a friend. This is how you can reach friends of friends. This philosophy has been applied to multiple information sharing algorithms within Retroshare. Read more about them in the following articles on the Retroshare team blog: Retroshare Forums, Distributed chat(a.k.a. Chat Lobbies), RetroShare’s anonymous routing model and Distant chat and messaging, using generic tunnels.


Figure 3: Retroshare forwards messages from friend to friend

On the neighbor Layer, you have to add your friends keys to your keyring by hand. But this is not a problem in practice, because the number of direct friends is small and Retroshare offers various options to get to know neighbor nodes one hop away. On the service layer you want to have the public keys of people many hops away. In Retroshare v0.6 there is a identity exchange system, which downloads public keys at the same time it receives a post from a unknown author. The identity exchange system can also work decoupled from the neighbor layer. This allows us to have pseudonymous identities. These identities can be used for different kinds of services like chat, mail, forums and the new social network plugin.

Retroshare is a steadily grown project, so the developers gained a lot of experience. Out of this they developed new systems to distribute messages in a Retroshare network (short name is gxs for General Exchange System). So all the complex code for doing the signing, verification, encryption and decryption of data is already done and avaliable. During the Summer of Code i will wrap this complex stuff in a nice webbased user interface. The RSWebUI Plugin is a starting point. It uses Wt, which is a library to create a webinterface with C++, without writing own code in web languages. You can watch the development in the git repository.

Read the next blog post to learn what features the social network plugin will have and how it will work in details.

Download Retroshare: http://retroshare.org

About me: i study electrical engineering. I first started programming to control hardware. I came to Retroshare because i wanted to chat securely without depending on a server. Then i started to dream about what else would be possible with a secured connection. So i started reading the code. Today i have enaugh knowlegde to contribute.