Google Summer of Code ’18, at Oppia

About the project

Oppia is an organisation that relies on the community not only for developing the infrastructure but also to get good quality content on the site. The explorations that are present on the website are all created by the community for the use of the community. The key explorations in Oppia are currently created by a smaller set of people mostly a part of the development team. We would like to allow people from the community to also help out to create and contribute to key explorations.

But just allowing more number of contributors may not be the best idea. Too many cooks spoil the broth. To allow valuable contributions by other members of the community while maintaining quality of content produced, we can have a system where the contributions of the community are put forth to the creators, and they review the changes. If they are happy with the changes, they should be able to accept them into the exploration. This allows us to effectively crowdsource Oppia’s content while maintaining quality.

I feel that adding this system is really important for the organisation. It allows us to increase our contributor count while also maintaining quality. This will enable us to produce many more better explorations, at a much faster speed.

Milestone 1

In the first part of the project, we focussed on completing the backend models, services and controllers for the suggestions framework. The emphasis was on building a general system that could be applicable to a large variety of suggestion types. Also, we already had existing suggestions for exploration state content changes, which needed to be migrated to the new framework. This involved writing a migration job to populate our new models with existing data, and then making changes to the frontend to ensure proper functioning of the suggestion framework. Another task I had taken up was to clean up some of the existing code for the feedback framework, which had ambiguously used “thread_id”.

List of PRs for this milestone:

• Backend models for suggestions (https://github.com/oppia/oppia/pull/4943)
• Domain objects for suggestions (https://github.com/oppia/oppia/pull/4966)
• Removing ambiguous definitions of thread_id (https://github.com/oppia/oppia/pull/4971)
• Backend services for suggestions (https://github.com/oppia/oppia/pull/4982)
• Refactor suggestion domain to use sub-classes for various types of suggestions (https://github.com/oppia/oppia/pull/4989)
• Rename the suggestion model, due to model name clashes (https://github.com/oppia/oppia/pull/5016)
• Migration job for populating the new suggestion model with data from the old model (https://github.com/oppia/oppia/pull/5043)
• Suggestion controllers and wiring up with the frontend (https://github.com/oppia/oppia/pull/5093)

Milestone 2

In the second milestone, the focus was to introduce a scoring system to help allow a lot of people to contribute to suggestion reviews while we also maintain standard of reviews. Each person would earn a certain score on successfully getting an accepted suggestion (for a particular category), and on accumulating a certain threshold, the user can then accept suggestions in that particular category. Then there was a feedback thread generalisation which needed to happen. We needed to unlink feedback threads (and related models) from explorations and make them usable for any general entity. This was essential to ensure that the suggestion framework would be applicable to all lessons. We also wrote a cron job that will automatically accept all stale suggestions. A suggestion is said to be stale, if it had no activity (no messages on the thread or no one accepted the suggestion) for at least 7 days.

We generalised the query functions so that we would not need to create new functions for each of the types of queries we encounter. Then we generalised the decorators, and added permissions for users with scores above the threshold to be able to review suggestions, and for admins to review any suggestion. We fixed a major frontend issue where the suggestion subject was not displayed correctly. In the process, we also created frontend domain objects for suggestions, and feedback threads.

List of PRs in this milestone:

• User contribution scoring models (https://github.com/oppia/oppia/pull/5128)
• Cron job to accept stale suggestions (https://github.com/oppia/oppia/pull/5155)
• Generalized function to query suggestions (https://github.com/oppia/oppia/pull/5164)
• Change permissions of users who can review suggestions (https://github.com/oppia/oppia/pull/5170)
• Validation job for the suggestion migration job (https://github.com/oppia/oppia/pull/5174)
• Feedback thread generalization (https://github.com/oppia/oppia/pull/5235)
• Email templates for the suggestion framework (https://github.com/oppia/oppia/pull/5245)
• Frontend domain objects for suggestions and feedback threads (https://github.com/oppia/oppia/pull/5273)

Milestone 3

In this milestone, we attempt to create a new type of suggestion using our existing model. The new structures team and the review system team collaborated to build a suggestion based system to add questions (which is a new construct in Oppia, which was introduced in another GSoC project this year!). We built the backend domain objects and controllers for the suggestion type and then worked on the interface to create, view and review question suggestions. Another thing that we worked on was a common suggestions view in the creator dashboard itself. This view would be useful for reviewers as it serves as a common place to see all suggestions needing review. We implemented a rotation for reviewers in a particular category, and wrote a cron job that will email the reviewer in cycle daily (provided, there exist suggestions needing reviews).

List of PRs for this milestone:

• Suggestions for questions backend (https://github.com/oppia/oppia/pull/5361)
• Common suggestions interface UI (https://github.com/oppia/oppia/pull/5419)
• Question suggestions UI (https://github.com/oppia/oppia/pull/5486)
• Cron job and reviewer rotation (https://github.com/oppia/oppia/pull/5387)

Miscellaneous tasks and bug fixes

From time to time, certain bugs were introduced, which needed to be fixed immediately, and also some miscellaneous tasks were taken up to clean up some of the existing codebase.

List of PRs for bug fixes/miscellaneous tasks:

• Add check for response status in get_json (https://github.com/oppia/oppia/pull/5001)
• Removed refresh option from issues card — release bug fix (https://github.com/oppia/oppia/pull/5100)
• Add waits in suggestions e2e test (https://github.com/oppia/oppia/pull/5109)
• Fix backend test failure (https://github.com/oppia/oppia/pull/5120)
• Remove assigned_reviewer field (https://github.com/oppia/oppia/pull/5203)
• Fix for backend test after flipping flag (https://github.com/oppia/oppia/pull/5206)
• Fix travis build after merging #5155 (https://github.com/oppia/oppia/pull/5269)
• Fix ordering of suggestions in the feedback tab (https://github.com/oppia/oppia/pull/5404)
• Fix issue in the status selection dropdown, where it displayed too many options (https://github.com/oppia/oppia/pull/5460)
• Fix feedback thread migration job to preserve created_on and last_updated fields (https://github.com/oppia/oppia/pull/5495)

My full list of pull requests can be found here

Future ideas

This is just the beginning for the review system project. During GSoC I have managed to implement a framework which is reusable to implement various different kinds of suggestions. We had some ideas in mind like suggesting translations to content in a lesson, suggesting changes to hints, adding worked examples for a concept, adding an exploration to a story, etc. The interesting thing is that all of these ideas can be represented using the same backend model. The UI for each of these tasks will need to be built separately to suit the application.

Conclusion

I would like to take this opportunity to thank my mentor Anmol Shukla for all the guidance, feedback and reviews during the GSoC period. I would also like to thank Sean Lip for all the help, clarifications and also guiding me when something went wrong. I would also like to thank Google for having such an initiative for students to learn about good coding practices, how production code runs, etc while collaborating with open source companies. I would also like to thank the mentors and org admins at Oppia for considering my proposal, and accepting me into the program.

I have learnt a lot during these 3 months, and have also significantly improved my technical design skills! This was an intense project that needed careful considerations to make sure that it is applicable in a wide range of scenarios, and also in some situations, some small tweaks to the framework would make it applicable to some scenarios where (trivially) it wouldn’t apply. The joy after having a successful migration is not explainable. I also worked on features like emails, cron jobs, and decorators where I had to investigate a lot of the existing code, and do a lot of searching for docs/articles online to get a sense of how it works. This was really interesting and enriching. I would definitely continue my work with Oppia and would also gladly help out other contributors trying to use the suggestions framework for different applications.