GSoC 2019 – Import public datasets to Retroshare network final evalutaion

Hi all!

On this final evaluation we focus the work on give to the automatic API wrapper generator support for openapi-swagger specification.

As Wikipedia said:

The OpenAPI Specification, is a specification for machine-readable interface files for describing, producing, consuming, and visualizing RESTful web services.

In addition, there are lots of tools that generate the front and backend code automatically from an OpenAPI specification. So, theoretically, an API interfacedefined with the OpenAPI specification, can easily be generated in an amount of languages using the OpenAPI Generator Project .

So with my mentor decided that this is more useful and priority for RetroShare project, because anybody could generate their own wrapper in a lots of languages in an automated way from the last commit of RetroShare master branch without write any code. 

So the workflow should be:

  1. Compile the RetroShare desired commit to generate the Doxygen documentation.
  2. Use the jsonapiwrapper-generator-openapi.py script that generates the OpenAPI YAML specification for RetroShare API.
  3. Generate the frontend code using  the OpenAPI Generator Project on to the desired language. For example, you can check the result for python on this example.

Following this steps, easily you can have an updated RetroShare API wrapper without coding anything!.

Some interesting tip about auto generated code is that the documentation are generated automatically with working examples.

Workaround

After we decided to give the support to OpenAPI specification we start to adapt the previous script https://gitlab.com/jpascualsana/retroshare-api-wrapper-generator/blob/master/jsonapiwrapper-generator.py to support the new specification. 

Trying to use the generated YAML against the generator validator shows us a lotsof warnings and crashes of our script. So we need to solve lots of bugs that we didn’t know that exists, bugs related of RsClasses generation, types etc… https://github.com/RetroShare/RetroShare/pull/1614

Also we deeply studied how to translate the RS API on to the OpenAPI specification, studying how to translate the types, types of functions, the authentication etc… 

Then, how to integrate the documentation on to the YAML to see the result on the generated code, following the specification. 

Finally we get in trouble with the asynchronous functions, that are not fully supported for the python generator and we are already looking for a solution.

So once we had the generated API wrapper we start to test it:

  • Tests for models: when the model objects are created using the OpenAPI Generator, you can pass an entire object as dictionary or instantiate it with the provided Class created by the generator. For example, on the link above we can see how to instantiate the object instead of pass it as dictionary.
groupMetada = RsGroupMetaData(m_group_name="Just another test channel2", m_group_flags=4, m_sign_flags=520)
channel = RsGxsChannelGroup(m_meta=groupMetada, m_description="Test it!")
req_rs_gxs_channels_create_channel = {"channel": channel}

Conclusion

So on this GSoC we started thinking that we  are going to code a way to import public datasets on the RetroShare network .

To do that we understood the necessity of create an automatically API wrapper, to do not code by hand each end-point of the API. 

Finally we found the necessity to provide a way to generate wrappers on any language needed on an automatic way, finding an already created solution on the OpenAPI Generator Project. 

Now is easier for developers to write applications over RetroShare network using the RetroShare JSON API, unifying the wrappers generation. 

On the back we let the project to import pulic datasets that I would like to finish on the following months, after GSoC, implementing the OpenAPI wrapper to the “import public datasets” repository. 

Repositories

Freifunk posts

  1. Arrival
  2. First evaluation
  3. Second evaluation
  4. Third evaluation

GSoC report

https://gitlab.com/jpascualsana/gsoc-2019-report

One thought on “GSoC 2019 – Import public datasets to Retroshare network final evalutaion

Leave a Reply

Your email address will not be published. Required fields are marked *