Tag: JSON

Before you ingress

Hello everyone!, The main intention of this project is to update the existing API spec schema files to the latest version of JSON schema, and also few tools which are dependent on the spec files.

The current latest version of JSON schema is version 2012-12, but unfortunately there isn’t much support to update the dependent tools(to be specific, the generator). So my mentor Andibraeu and I have chosen to work with draft7 version, as it has upper hand with implementation support compared to other recent versions.

Our initial map out

• Migrate all the spec schema file to draft 7 version.
• Pick out a framework to update the generator.
• Generate and test the forms.[1]
• List and update all the remaining dependent tools.
• Test the updated tools.
• Fix bugs, if any.

Spec files

Migrating the spec files to draft 7 version is not a difficult task. At the beginning, I have only migrated the recent version of spec file, so that we can immediately start working with the tools. And after updating all the tools, I have migrated all the spec files to draft 7 version.

Some features to point out:

• Used definitions to reuse some fields like (phone, email, city, URL, etc.).
• Added formats to some fields like (url, email, lastChange).
• Used minProperties instead of required for contacts.
• Used AdditionalProperties to block extra fields.

References:

1. Initial Migration -pull request
2. Patch -pull request
3. Patch -pull request

Generator

This is one of the significant tool depending on spec schema files. The job of the tool is very simple, it takes JSON schema as input, generates HTML forms to render in the browsers, handles validation of the form data against the input JSON schema and finally, generates a JSON out file.

Unfortunately, no framework seemed perfect at the beginning, so I have picked up several frameworks to try them out and weigh the pros and cons to finally pick one. I had mentioned all the pros and cons in a document(check the references section).

Frameworks that I have tried

• UI Schema for React
• React JSON schema forms (mozilla)
• Restspace Schema forms
• JSONForms (Eclipse Source)

By weighing all the pros and cons, we have chosen JSONForms (Eclipse Source) to proceed with the further development.

JSONFORMS (Eclipse Source)

And thereafter, I developed UI schema for custom layout for the form field in the webpage. Also, at this point, I have to develop a custom renderer to render a map for picking the latitude and longitude of the communities locations. So upon looking at the documentation I developed the custom renderer (renderer, control and tester) thereafter to adopt this renderer to schema we need to have single object which only embeds longitude and latitude fields. So I quickly discussed with my mentor and added a new spec file to the organization. In conclusion, we have our generator up and running a demo in GitHub Pages.

References:

1. Evaluation of frameworks -document
2. Implemented frameworks -repo
3. Generator tool -repo
4. live demo -GHPages

Dependent tools

The API viewer and the Travis job CI are completely dependent on the spec schema files to validate the communities API files.

API Viewer

This tool generates a static build of pages, which show the validation result of the communities API file data.

Improvements:

• The prior tool existed in python 2, I have updated to python 3.
• Update to Validate data against draft 7 schema and show validation.
• Added datatables to list the communities.

References:

1. Python 3 migration with Draft 7 data validation -pull request
2. Patch -pull request

Travis Job

All the API files are collected in the directory repository. And this Travis job validates the data of the API file data, when they are updated or added to the directory.json

A build of the travis job can be found here.

Improvements:

• Also, this tool (test) existed in python 2, I have updated it to python 3.
• Updated to validate data against draft 7.

References:

1. Python 3 migration with draft 7 validation -pull request

Common API(collector script)

If you recall as I have added new spec file by embedding latitude, longitude into an object to adopt with the jsonforms custom renderer for map picker using react-leaflet. This would affect a lot of other tools like Community finden, Kontakt, etc which are truly based on the lon, lat fields of the API files. But luckily all these tools use a summarized API file, And the collector script is used to collect all the communities files.

Improvements:

• Deserialized geoCode object and appended the fields to the respective locations. So that the fields are set to their old locations.

References:

1. Altering Location fields -pull request

References

Here are the previous blogs of the project at different stages:

1. Initial Stage(Before coding period)
2. Phase I evaluation

Wind-Up

I have started the project with minimal understanding of react, typescript and jsonschema. But it was very fun to understand and work on. I really liked this way to learn new things rather than reading or doing a course. Every issue that I have encountered had leaded me to understand the things briefly. I’m really thankful to freifunk for the opportunity. And a big shout out to my mentor Andreas Bräu for absolutely wonderful guidance and support.

~ Shiva Shankar aka sh15h4nk

Prologue

Welcome back!, This blog is for the first evaluation of the project.

Recap

In the prev section, we saw a viable way-out for the inconsistency of the generator tool for the latest version of the schema(2020-12) by mentioning some frameworks and also some targets.

Frameworks Review

I have tested the frameworks with the draft 7 version of the schema and I evaluated the framework by noting down the pros and cons of the results by each framework.

• UI schema for react: This framework supports very advanced features of the schema, but has less UI integration. Also the framework supports the 2019-09 version of the schema but sadly UI schema cannot be segregated from our original JSON schema.
• React JSON schema forms: This framework supports basic features of the schema and also has good UI customization, but has less integration of validation UI.
• Restspace schema forms: This framework supports basic features of schema, but misses proper documentation.
• JSONForms (eclipse source): This framework supports basic features of the schema and also has good UI customization with limited options.

A full document of the pros and cons of the frameworks can be found here.

Summary

By considering all the pros and cons of each framework, my mentor Andi and I have decided to work on with JSONForms (eclipse source).

Ongoing Status

I rewrote the schema for the API generator tool to the draft 7 version of JSON schema also I have added formats to the schema. And generated the forms from the schema. I have been using React library (JSONForms) to generate the forms. Also, I developed the UI schema which is required to generate the form.

I have rendered the form, and also the bounded data of the form, which is the current data of the in the form and will be updated on change of the form data. Also, I have rendered the validations errors to show all the validation errors at one place before and after submission of the form.

Validation and Submission.

The JSONForms only emits errors through an event. So I have added a state to track the errors emitted by the event and on the event emit I recorded the errors into the state.

And for the submission of form we have to consider there are no recorded errors and the form data should not be empty because I have recorded errors into state only if the forms data is not empty. Errors are emitted by the event even before starting to fill the form. By validating all this checks, I have generated the output JSON file.

Loading data

For the testing purposes, I have added a button to load the weimarnetz API file data into the forms. I have fetched the ffSummarizedDir.json file from api.feifunk.net which consisted of all the communities API file data and rendered all the communities into a select field. So then I can add a on change event to load the data into the form.

References

• A full document of the pros and cons of the frameworks can be found here.
• A live development site can be found here.
• The GitHub repository link is here.

Accomplished Targets

• Developing the Schema to the latest version (taking draft 7 as base version).
• Choosing a Frameworks (Chosen JSONForms).
• Integrating the updated schema with the framework to generate the forms.
• Examining the forms (validating input data and functionality of the forms).

Final goals

• Try creating a custom renderer for location (lon,lat) picker.
• Upgrading all the dependent tools.(API viewer, Travis job validator)
• Testing all the tools and fixing bugs if any found.

~ Shiva Shankar Genji aka sh15h4nk