Google Summer of Code 2023 with Liquid Galaxy Organization

Mahinour Alaa Elsarky
17 min readAug 25, 2023

--

Contributor: Mahinour Elsarky

Mentor: Claudia Diosan, Karine Pistili

Organization Admin: Andreu Ibanez

Project: HAPIS — Humanitarian Aid Panoramic Interactive System

Size: Easy 175h

Organization: Liquid Galaxy project

Hello everyone, I am Mahinour Elsarky, from Cairo, Egypt. I am currently studying computer engineering in Cairo University. My expected graduation is 2024. I am an AI enthusiast and I love to work on AI and data science applications. I am also into mobile development, especially flutter.

This was my first-time joining Google Summer of Code, and it was an extraordinary experience where I learnt so much, and I am thrilled to have been a part of this amazing journey.

I’ll try to summarize the work that I did in my GSoC project. Let’s get started!

  1. Acknowledgement
  2. Liquid Galaxy Organization
  3. Project Description & Project Goals
  4. Important Project Links
  5. Work Done (Week wise)
    Week 1 (June 12 — June 18)
    Week 2 (June 19 — June 25)
    Week 3 (June 26 — July 02)
    Week 4 (July 03 — July 09)
    Week 5 (July 10 — July 16)
    Week 6 (July 17 — July 23)
    Week 7 (July 24 — July 30)
    Week 8 (July 31 — August 06)
    Week 9 (August 07 — August 13)
    Week 10 (August 14 — August 20)
    Closing and Finalization (August 21- August 28)
  6. Current State
  7. Future Work & What’s left to do
  8. Conclusion

Acknowledgement

I am so grateful for working at Liquid Galaxy organization this summer. I really enjoyed every second with this motivating, encouraging and amazing community. With Liquid Galaxy, I was able to create something extraordinary by making visualizations on Google Earth using KMLs, and making a helpful flutter application that can benefit people all over the world and really make a change.

I want to thank my organization Liquid Galaxy and Google summer of Code program for granting me this amazing opportunity to contribute this summer. I was able to learn more things than I would have in a year from simply doing university courses, and not only in coding, but in plenty of other aspects. As for the technical side, my flutter skills have enhanced greatly, and I can now say that I am able to write clean code that follows the best practices. I also learnt how to understand someone else’s code base and extract useful info from it that can help me in my code. I learnt a lot about KMLs too and how to use it effectively to show nice looking visuals on Google Earth. Since setting up Liquid Galaxy rig on my machine requires some knowledge about virtualization and networking, I was able to gain more experience in that too. I was also able to learn about the process of uploading a flutter app to the play store, and how this could be really challenging. For the non-technical side, I was able to enhance my communication skills greatly, and I made lots of new friends which I am really grateful for.

My project HAPIS, was originally My mentor Claudia’s idea, but there was a lag in the launching of the app due to Covid-19 and some of the regulation policies and that is why the idea of HAPIS-Refurbishment came along, and my role was to bring HAPIS back to life and be able to make it easy and simple for all users.

I want to thank my main mentor Claudia Diosan for giving me continuous motivation and encouragement throughout the entire program. She always believed in me, and that we would make a great project together, and I am so grateful for having her as my mentor. I am also thanking her for thinking of the idea of HAPIS in the first place in 2019 to help people all over the world and connect them together, and I am delighted I got this opportunity to contribute to something that will be helpful to humankind.

I also want to thank my secondary mentors Karine Pistili, Emilie, Deniz Yüksel for their support and help throughout the entire program and giving some nice feedback to help make the project succeed and help make it look nicer and better. And I want to thank all Liquid Galaxy mentors who also helped along the way, they were all so supportive and helpful.

The app wouldn’t have succeeded too if it wasn’t for Liquid Galaxy Lab Testers, Mohamed Zazou, Navdeep Singh, Imad Laichi, who helped in testing the app on real liquid galaxy rigs and helped me find any troubles that might have appeared during the testing process.

Last but not least, I want to thank mentor Alejandro Illan Marcos for the time and effort made to help put our apps on the play store, no matter how challenging this process was.

Lastly, a special thanks to my Organization Admin, Andreu Ibanez, for being so helpful and active throughout my internship. He was always there helping us and guiding us along the way, and I appreciate everything he has done for us to help us implement a successful application which could be uploaded on the play store.

I thank all of the Liquid Galaxy community starting from my org. Admin to my mentors, to all of the contributors, as the app couldn’t have been better without their help and support.

Liquid Galaxy Organization

The Liquid Galaxy is an open-source project founded by Google and created by Google Employee Jason Holt in 2008. Liquid Galaxy started out as a panoramic multi-display Google Earth Viewer but has evolved to become a general data visualization tool for operations, marketing and research. It gives a unique experience to allow users to fly around Google Earth, view photos, screen layouts, balloons, placemarks and develop interactive tours.

Liquid Galaxy consists of 3 or more computers for multiple displays, and each computer manages a display. It depends on a master/slave architecture where the master is responsible for receiving all information and replicating it to the other nodes using SSH message passing, allowing the screens to behave in a synchronized manner. You can find more about Liquid Galaxy from here.

Liquid Galaxy

The Liquid Galaxy Project, while making use of Google Earth software, does not develop the Google Earth codebase itself. Google Earth is not open-source software, although it is free (as in beer). Instead, the Liquid Galaxy Project works on extending the Liquid Galaxy system with open-source software both improving its administration and enabling open-source applications, so that content of various types can be displayed in the immersive panoramic environment.

Every year there are new android apps developed using the Liquid Galaxy technology to help in either social or educational aspects of life. Evey app will astonish you in its own unique way. You can take a look for yourself on some of the apps that were uploaded on the play store or simply taking a look at the source code of some of the apps.

Project Description & Project Goals

Project Description:

The main purpose of HAPIS is to be able to connect those who need something with those willing to offer it back through a simple android application, and to be able to visualize all this data in real-time on Google Earth on the Liquid Galaxy machines and view them in a nice panoramic display. We generate KML files for displaying balloons to show the global statistics, local statistics for each city and the user's info and their current transactions. We also use KML files to fly to a certain POI “point of interest” and make tours and orbits around a place. This app should be available for anyone with an android mobile or tablet with version 11 or above, and that is why it is called its Humanitarian Aid Panoramic Interactive System.

Originally, HAPIS, developed by mentor Claudia in 2019, stood for Homeless Aid Panoramic Interactive System, and it had the same goal, but it was only for helping out homeless people with necessary life items such as food or clothes. Due to covid-19 and some regulations and policy rules, the app had a lag in the launching and that is why we decided to refurbish the app. The new HAPIS is now simpler without complex policy regulations and helps not only homeless but everyone in need for anything. Things offered can be multiple things and divided into different categories, so aside from basic human needs, maybe people can share useful educational materials, unwanted electronics or many other things.

The app was originally developed in Java, however in the refurbishment I used flutter for developing the app, as flutter has very good performance that enables smooth animations and faster app loading times. This ensures a snappy and responsive experience for our users. Flutter also has a vibrant community that provides countless packages and plugins, making it easy to add new functionalities to the app. We found answers to our questions and valuable insights, supporting our journey throughout the development process. Additionally, Flutter’s hot reload feature significantly sped up our development. Seeing changes instantly without recompiling the entire app was a game-changer, allowing us to tweak the UI and fix bugs on the fly. Another main thing is that since our app should work on tablet and mobile devices, flutter’s responsive package made it so much easier to implement layouts that can be for any device with any dimensions without having to implement different layouts.

Main functionalities & Project Goals of HAPIS:

Liquid Galaxy view:

  • Fly to functionality that uses the latitude, longitude & altitude specified in the KML file to fly to certain location.
  • Orbit functionality that cycles around a specific location of a user with different speeds based on given latitude, longitude & altitude in the KML file.
  • Display global statistics functionality that shows a box filled with global statistics such as: current number of givers current number of seekers, total donations, top city with most donations, top 3 donated things.
  • Connecting with the Liquid Galaxy functionality by configuring the LG from the settings from the menu icon
  • Settings functionalities such as clearing the KML, relaunching the LG, shutting down the LG, or Rebooting the LG
  • Listing the cities available by retrieving the data from the database. Each city will have attributes such as: latitude longitude, altitude, name, country that will be sent to the KML file for functionalities needed such as the orbit or fly to functionalities in the LG.
  • Local statistics functionality that will gets executed when the user presses on one of the cities and the fly to function is executed where it will highlight or pin on the map the locations of the users whether they are donors or seekers in this specific city & show a box containing total number of seekers, total number of givers, total donations, top 3 donated categories on the LG.
  • Listing all the users of specific city (when user presses on a city, list of users appear) by retrieving the data from the database.
  • Users Info & Transactions functionality where we show the users info in a balloon sent to the LG and all the transactions made.

Main App View:

  • Submit a form as giver or seeker.
  • View all seekers and givers from the database.
  • View all requests, matchings and In-progress donations from the database.
  • Accept or reject requests and matchings.
  • Request contact to a user
  • Finish a process or cancel it.
  • Match when a seeker and a giver are in the same city with the same item displayed and send notification accordingly.
  • Sign in/out of their accounts using Google Sign-in
  • Creating SQL database inside a docker container, so it can be hosted by anyone on any server either on the cloud or remotely using few commands to build the image with the persistent database.

Secondary functionalities of HAPIS:

  • Implementing the Tour functionality in the Liquid Galaxy view

Work Done (Week wise)

Week 1 (June 12 — June 18)

  • Project Code structure
  • Implemented UI for the Liquid Galaxy view (home page, settings, about page)
  • Implemented Splash Screen
  • Connecting with LG using ssh2 dart package
  • Understanding common liquid galaxy functions such as orbit, fly to..etc. as well as the LG commands for shutting down, rebooting, relaunching the LG and clearing the KML.
  • Added Screen overlay for the logos to appear in the LG.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week1

Connection screen & Settings Screen
Splash Screen & Home page

Week 2 (June 19 — June 25)

  • Learnt about SQLite and started setting up the database using SQLite initially.
  • Implemented CRUD operations, implemented necessary models for the database.
  • Implemented UI for the cities.
  • Understood how KML balloons work and placemarks.
  • Implemented them in my code, and after some challenges, I was able to show the balloons.
  • Added all country flags and mapped them for easier reference later on
  • Implemented fly-to certain cities — but wasn’t getting accurate coordinates regarding latitude, longitude and altitude.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week2

Cities
Local statistics

Week 3 (June 26 — July 02)

  • Changed ssh2 package to dartssh2 package as ssh2 didn’t work in release mode. Connection worked after that.
  • Implemented the global statistics.
  • Filled in the tables of the database with fake data and imported the fake data tables into the SQLite database.
  • Fixed the wrong coordinates problem in fly-to function by understanding more about the longitude, latitude and altitude.
  • Worked on the orbit around the globe — but had problems with it.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week3

Global statistics balloon

Week 4 (July 03 — July 09)

  • Implemented necessary SQL statements for retrieving global, local statistics and users' data and tested them.
  • Implemented Search bar for cities and users.
  • Implemented Users UI
  • Recorded the midterm demo video.
  • Fixed the “lost connection” issue I was facing in the LG.
  • Fixed the orbit.
  • Had a problem that changes weren’t showing on the LGs instantly.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week4

users page

Week 5 (July 10 — July 16)

  • Searched for ways to have a centralized database for all devices to share the same database. One of the ways was to use docker, where I gathered some resources on.
  • By the help of my mentor, I was able to solve the problem I had where changes weren’t showing instantly.
  • Showed user images on KML balloons.
  • Implemented pins placemarks and both seekers and donors pins showed.
  • Wrote down an initial documentation for the Wiki and Readme on GitHub
  • Refactored the code to make it responsive for all devices using the Responsive Framework package in flutter.
  • Gathered fake user images for the database.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week4

user pins
user balloon

Week 6 (July 17 — July 23)

  • Implemented mobile app structure.
  • Fixed some UI in donors and seekers tab.
  • Implemented SQL statements that will be needed for the main app view.
  • Implemented UI of the main application view (all pages) and modified it on tablet and mobile devices.
  • Searched for ways to do the login/signup authentication.
  • Implemented Google Oath in code where I used to google-sign in-Api and in console.cloud.google where I made new credentials and Google Oath consent screen.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week6

Home page of main app view

Week 7 (July 24 — July 30)

  • Implemented the rest of SQL statements for getting requests, matchings and donations.
  • Tested the login and signup functionalities.
  • Learnt about shared preferences in flutter to save the login session of the user and whether he is logged in or not.
  • Implemented geolocation for user to get his current location.
  • Filter and search functionalities for seekers and donors.
  • Recorded the flutter video for the liquid galaxy streaming.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week7

Sign up Page

Week 8 (July 31 — August 06)

  • Implemented accept and decline requests, accept and decline matchings.
  • Implemented insert new form.
  • Implemented Check matching between 2 users function, and create a match function.
  • Implemented finish donation process function.
  • Implemented requesting someone function.
  • Implemented user personal forms and adding edit/delete forms function.
  • Implemented the settings page for the main app view (sign in — sign out — edit profile — see notification — delete account)
  • Tested all scenarios that could happen and added all database checks and error messages.
  • Fixed some bugs and resolved some of the feedback I got from my mentors and from the testers in liquid galaxy LAB.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week8

Main app view Pages

Week 9 (August 07 — August 13)

  • Worked on resolving some feedback I got.
  • Started implementing the tour functionality in the Liquid Galaxy view.
  • Adjusted some UI.
  • Implemented notifications functions.
  • Tested everything on mobile and tablet.
  • Added badges for bottom navigation bar.
  • Recorded the final demo video.
  • Gathered the required Play store data.
  • Installed docker, MySQL image, and MySQL workbench. Made a new container with volume to persist the database. Refactored the database from SQLite to MySQL, and had some issues with connecting to it from my flutter app.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week8

Notification Page from Settings

Week 10 (August 14 — August 20)

  • Connected the database from my flutter app.
  • Fixed any conflicts from changing to MySQL instead of SQLite.
  • Worked on the tour functionality.
  • Created a VM server on Azure and created a new image with docker file to run the script to get the data from database.
  • Wrote down documentation for how to set up HAPIS database on any server, either in cloud or remotely, using some simple commands to build their own image that has MySQL image and the persistent database.
  • Was trying to find a way to add user images in the database but figured that the only way to do this was to have a backend. So, for now, I just saved the fake users images in the flutter project itself, till we have a backend for this.
  • Added necessary .env files.
  • Implemented “How it works” page in main app view.
  • Worked on resolving any issues found.
  • Full documentation on GitHub (Readme & Wiki)
  • Attended the final demo day in liquid galaxy streaming.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/week8

Users Image on Balloon
How it works Page

Closing and Finalization (August 21- August 28)

  • Testing the application.
  • Added loading indicators.
  • Solving play store issues regarding the google sign in, and the app signature.
  • Making sure everything is finished correctly.

Code source:

https://github.com/Mahy02/HAPIS-Refurbishment--Humanitarian-Aid-Panoramic-Interactive-System-/tree/development

Current State

For this point today, as for 23/8/2023, I have finished implementing all main functionalities I wrote in my proposal, and HAPIS is in good state, with all issues found fixed.

As for the database, I left out good documentation on how anyone can host it on any server using docker, to be able to make HAPIS work with real data and real people, not only with fake users.

For the secondary functionalities, the tour function is now working, but without the balloons.

Future Work & What’s left to do

HAPIS can be enhanced even more by adding more features and functionalities to it to make it more structured and ready for all people to use.

For example:

  • A backend, instead of interacting directly with a database, would be a very good idea if we want HAPIS to really be an app used by people all over the world due to many reasons like security, performance, abstraction, validation, business logic separator, scalability, error handling and optimized queries. Also, a backend would solve the image problem of saving users images.
  • Integrating Twitter APIs or YouTube APIs within the app could also be very beneficial.
  • Hosting the database on the server is a necessary step for the app to work globally with real people and real data.
  • If we found a way to show balloons in the tour functionality, it would be even better.

I also have some thoughts about new functionalities and features we can add to HAPIS such as:

  • Integrating an ai bot that users can ask on anything related to the statistics, and the bot can generate answers in 2 forms. The first is in form of normal text, and the second is by generating a KML and sending it to the LG. The generated KML could be an integration of fly-tos, balloons, orbits and also generating an ai voiceover in the KML file.

Conclusion

I am really grateful for this experience; I have learnt a lot and was able to create an app that could benefit people someday. I will always be ready for updating the code and fixing any issues that might appear along the way to make HAPIS shine even more.

Participating in GSoC and joining Liquid Galaxy had a profound impact on me. Throughout this journey, I encountered numerous challenges and unfamiliar problems that pushed my boundaries. Yet, I successfully employed innovative thinking and leveraged the latest technologies to overcome these obstacles. This experience significantly bolstered my confidence in my problem-solving abilities. Presently, I am proud to affirm that solutions exist for every problem, and I firmly believe that with determination, any challenge can be resolved.

I absolutely loved the experience, and if there was an opportunity for me to join again, I wouldn’t hesitate. I am looking forward to contributing to Liquid Galaxy even after GsoC ends!!

If you are interested in contributing to Liquid Galaxy projects, you can reach out by sending email to Liquid Galaxy Lab

Email: liquidgalaxylab@gmail.com

Feel free to reach out to me!

LinkedIn: https://www.linkedin.com/in/mahinour-alaa-122958216

GitHub: https://github.com/Mahy02

Email: mahinouralaa2002@gmail.com

Thanks for reading!! ♥️🌏

--

--