GSoC 2019 – Import public datasets to Retroshare network second evalutaion

Here again!

This evaluation I spend the work creating an automatically generated wrapper for the API. This wrapper is generated analyzing the Doxygen XML files generated when Retroshare is build. 

Creating the API wrapper

First of all, I modified the python script (made by @sehraf ) that generates the C++ API files, to create a python wrapper for the API. Analyzing the script and the XML files I get my script working generating a first version of the wrapper. Then I test the wrapper, giving support to async functions also. Some features of the wrapper are:

• Document the code using DocString convention.
• Parse also ‘manualwrappers’ like attempt login.
• Requests with authentication and without.
• Support basic authorization or token auth via ‘Authentication: basic bas64Token’ header.
• Async methods support and callback implementation.

Here an example of the API wrapper generated: https://gitlab.com/snippets/1877207 . Some tests for the wrapper can be found here:

class TestMultiple(TestCase):
    def test_login(self):
        res = wrapper.RsLoginHelper.isLoggedIn()
        print(res)
        # Do login
        if not res['retval']:
            res = wrapper.RsLoginHelper.attemptLogin(ACCOUNT, PASSWORD)
            print(res)
            self.assertEqual(res['retval'], 0, "CANT LOG IN")
            return
        self.assertEqual(res['retval'], True, "is not loged in")

    def test_authorizedMethod(self):
        res = wrapper.RsGxsChannels.getChannelsSummaries()
        print(res)
        self.assertEqual(res['retval'], True, "Can't get channel summaries")

class TestAsyncMethods(TestCase):
    def cb(self,res):
        print("cb", res)
    def test_asyncMeth(self):
        wrapper.RsGxsChannels.turtleSearchRequest("XRCB", 300, wrCallback=self.cb, wrTimeout=4)

Creating Retroshare Classes wrapper

After that, the problem was that a lot of functions need to have Retroshare classes as parameters. For example, to create a Retroshare forum, are needed classes like RsGxsForumGroup that at the same time need other inner classes like RsGroupMetaData. With the first version of the wrapper all this classes was passed in JSON format that was really annoying to assemble. 
So the next step was parse also this Retroshare classes recursively from the XML files to a Retroshare classes wrapper. On this step it was difficult to parse it correctly: differentiating the different types of classes, and class attributes, translating the types to python, if they are enums, primitive types etc… Finally I created this second class wrapper so when you need to pass a RsGxsForumGroup to the API wrapper you can just instantiate it and the wrapper do all the necessary to transform it to python and call the API. Some features:

• Parse “compound” classes (structs on C++) recursively.
• Parse “enums” and get their values.
• Parse “typedef” and “using” classes and translate it to the appropriate type on Python .
• Document it using Docstring convention .

Here an example for the class wrapper: https://gitlab.com/snippets/1875153 . Some tests can be found here:

    def test_createChannel(self):
        channelMetadata = RsClass.RsGroupMetaData(mGroupName="TestChdddannelCreation2", mGroupFlags=4, mSignFlags=520)
        channel = RsClass.RsGxsChannelGroup(mMeta=channelMetadata, mDescription="Channel Test")
        res = wrapper.RsGxsChannels.createChannel(channel)
        print(res)
        self.assertEqual(res['retval'], True, "Can't create channel")

For “v2” methods I opened an issue because I can’t communicate with the API. It was resolved as my “retroshare-service” wasn’t updated at all.

Next steps

This script will be adaptable to generate the wrappers for the language needed, for example for an OpenAPI format, TypeScript… Making much easy to other developers to start developing over Retroshare network.

Also it will be very easy to update when new feature is added to the API because it can be generated each time Retroshare is built.

Now will be time to apply the wrapper to the scripts that will import the public datasets!