Google Summer of Code ’19 at Oppia Foundation
Introduction
Oppia is a platform that enables anyone to learn and teach anything they want in an effective and enjoyable way.
Learning in a slow environment would not be enjoyable, hence the need to optimize the speed of the platform.
About the project:
My project aimed at serving Oppia’s pages statically, which means getting rid of Jinja.
Jinja is a python templating engine and is used with the AppEngine webapp2 framework.
Since the jinja templates will have to be converted every time before rendering the page, a cached version of each page cannot be kept in the browser and this means that for every time a user visits a page, a freshly baked version of the page is sent to the user with all its dependencies.
On the other hand, serving the pages without Jinja will mean losing all the template inheritance and other features which Jinja offers, hence the new static solution should be able to provide similar functionalities to what was gotten using Jinja.
Technologies used:
The technologies I used to achieve this include:
- AngularJs:
AngularJS is the frontend library which is currently used at oppia.org. I used AngularJS to maintain similar template inheritance which was previously done using Jinja templates. This was possible through the use of transclusion components in the pages which provide slots for different sections of the page to be inserted. - Webpack:
Webpack is the module bunder used at Oppia. It got introduced into the codebase shortly before the commencement of the GSoC program.
I primarily used webpack for bundling scripts and static HTML templates which would then be injected into the page during the build process. This helped as this is what jinja was used for previously.
Milestone 1
My target for the first milestone was to remove jinja from the fairly static pages. These pages did not fetch much data from the backend and mostly used Jinja templates for template inheritance and for maintaining a consistent structure.
Pull Requests Made:
- Serving Get started page statically:
The get started page was used to launch the use of webpack in the page and the use of angular transclusion components for template inheritance.
Here is the link to the pull request: https://github.com/oppia/oppia/pull/6818. - Serving other static pages using transclusion components:
Following the success of the work on get started page, the transclusion components were used in other mainly static pages.
The complete work on this can be found here: https://github.com/oppia/oppia/pull/6874. - Serve remaining static pages using transclusion components:
This pull request which was the final pull request made for the first milestone serves the remaining static pages using transclusion components. After the work on the second PR, the transclusion components were modified a bit and then used to serve the remaining static pages.
Work on this can be found here: https://github.com/oppia/oppia/pull/6934. - Miscellaneous:
The pull requests listed here include bug fixes, optimizations and other minor tasks that were related to the completion of the first milestone. https://github.com/oppia/oppia/pull/6835
https://github.com/oppia/oppia/pull/6845
https://github.com/oppia/oppia/pull/6880
Milestone 2
The work for the second milestone was to extend the work on transclusion components to the more dynamic pages. This was somewhat tricky because the dynamic pages load lots of data from the backend into the page.
Pull Requests Made
- Serve static pages via app.yaml:
Following the work on the first milestone, we discovered that there wasn’t any need to hit the python handler for some of the static pages but instead can be served directly from app.yaml. This pull request makes that possible: https://github.com/oppia/oppia/pull/6974. - Serve first set of dynamic pages using transclusion components:
This pull request uses the transclusion components in some dynamic pages: https://github.com/oppia/oppia/pull/7068. - Serve other dynamic pages using transclusion components:
These pull requests use the transclusion components in other dynamic pages:
https://github.com/oppia/oppia/pull/7104, https://github.com/oppia/oppia/pull/7187 - Send data needed by pages via AJAX and other means:
Not all data needed by the pages should be sent via an ajax request as some are fairly static. These set of PRs uses some means including but not limited to AJAX requests to get some data needed by the pages:
https://github.com/oppia/oppia/pull/7131 https://github.com/oppia/oppia/pull/7140 https://github.com/oppia/oppia/pull/7149 https://github.com/oppia/oppia/pull/7186 - Miscellaneous:
The pull requests listed here include bug fixes, optimizations and other minor tasks that were related to the completion of this milestone. https://github.com/oppia/oppia/pull/7123 https://github.com/oppia/oppia/pull/7191 https://github.com/oppia/oppia/pull/7199
Milestone 3
The final milestone was meant to remove jinja completely from the codebase and fix all the issues arising from the Jinja removal
Pull Requests Made
- Jinja removal from exploration editor page:
The exploration editor page happens to be the heaviest page in the Oppia codebase and as expected, work here was the most complex.
Jinja was removed from this page in 2 PRs. https://github.com/oppia/oppia/pull/7134 https://github.com/oppia/oppia/pull/7194. - Send data needed by pages via ajax and other means:
Some data needed by other pages which were not covered in the second milestone were covered in this milestone. https://github.com/oppia/oppia/pull/7203 https://github.com/oppia/oppia/pull/7374 https://github.com/oppia/oppia/pull/7385 - Serve more pages via app.yaml:
App.yaml was used to serve other static pages which were not covered in the second milestone.
https://github.com/oppia/oppia/pull/7268 - Webpack Integrations:
Jinja was used to load some additional scripts and styles used in different pages. These have been moved to webpack and the work done in 2 PRs: https://github.com/oppia/oppia/pull/7286 https://github.com/oppia/oppia/pull/7350
Some script imports were also moved to webpack to be integrated as a part of the webpack bundles for the page: https://github.com/oppia/oppia/pull/7447 https://github.com/oppia/oppia/pull/7409
Some pages contained imports that were not used and these increased the size of the page bundle. These imports were removed in one PR: https://github.com/oppia/oppia/pull/7422. - Remove jinja from other locations that were missed in previous PRs:
https://github.com/oppia/oppia/pull/7396.
Serve error pages using the transclusion components: https://github.com/oppia/oppia/pull/7449. - Miscellaneous:
Bug fixes that came with one of my changes: https://github.com/oppia/oppia/pull/7269
Speed Improvements
Speed improvements can easily be noticed in the pages as they are now lighter.
The time to first byte tests as carried out using chrome dev tools are shown below:
app.yaml — 42ms
Jinja render — 147ms
Django template — 119ms
Future Plans
Speed improvement is a continuous process. During GSoC, I have been able to implement static serving for Oppia pages which greatly reduces the initial page load time. Future work will be put into other areas to improve the page speed and ensure very fast loading pages.
Conclusion
I want to especially thank Vojtěch Jelínek for all the guidance, feedback, reviews and mentoring given to me throughout this period. I also want to thank Sean Lip for his guidance and for always ensuring that the code I wrote was of great quality. I would also love to thank my fellow GSoC students for their guidance when things went wrong.
I want to especially thank Google for the organizing the Google Summer of Code program which has helped launched me into the open-source world.
I also want to thank the Oppia Org Admins for accepting my proposal, guiding and mentoring me these 3 months.
These 3 months have been a thorough period of learning and coding. Working on a live project with such a great community, having a large number of people reviewing my code and an audience of users immediately waiting to use features which have been built. The joy of having thousands of people use what you’ve built is unexplainable.
I will continue contributing to Oppia and I’d love to help new contributors joining us.
