Cross-Platform Application Design with Electron, React, and gRPC
QueryPie Development #6: QueryPie Architecture Configuration
📑한국어(Korean Version)- https://medium.com/p/184601de5057/
Creating a single visible product requires a lot of invisible touches. When it comes to software development, the product is important but in many ways how the product is implemented is more important.
As mentioned in the two previous posts (Searching for my own Database IDE, QueryPie: a fresh design for SQL IDE), QueryPie is a database IDE that operates on three operating systems: Mac, Windows and Linux. In addition to supporting database security solutions with QueryPie Protocol, we are planning an integrated data platform that will enable us to analyze and visualize data like Business Intelligence (B.I) on the web in the future.
Because we planned to support all these functions as a single solution, the configuration and design of the software was important to establish prior to the development.
For applications installed and used on a PC/Laptop, programming languages that support multi-operating system compilation and run-time were essential. UI Framework was also a very important factor to provide convenient and excellent UI/UX.
So all of the development team members, including myself, have taken into account the technical capabilities and experiences that the company has and our mid-to-long term product road map. This article covers the story of what Technology QueryPie is being developed on.
1. QueryPie UI (Front-end) : Electron, TypeScript, React
Electron is a framework that enables desktop applications to be developed on Mac, Windows, and Linux. It is open source, developing business logic as JavaScript (or Type Script) and UI/UX areas are HTML/CSS with high extensibility and excellent visual implementation. In addition, since NodeJS and NPM ecosystems can be used as they are, it is possible to secure development scalability through a variety of libraries.
Products with a high amount of established users such as VSCode, Skype, Slack, and Atom, guarantee the completeness and stability of Electron.
2. QueryPie Engine (Back-end) : OpenJDK & Java & JDBC
Because we decided to use Electron for our front-end, we considered NodeJS for the back-end development. But we changed our plan because of these factors:
- The Database Connectivity Library in NodeJS is not as mature as JDBC
- The Java Eco System is appropriate for stable implement of key functions such as simultaneous SQL Execution, Data Export/Import, and more
So we decided to use OpenJDK-based Java and JDBC libraries for our back-end development.
3. Communication between Processors : gRPC
Front-end and back-end are run as separate processors, so stable communication between the two processors is required. While general web applications communicate via HTTP/Web Locket, QueryPie has chosen gRPC (a TCP/IP-based RPC framework) because it operates on a user’s local machine and application characteristics require fast response and reliable communication.
gRPC is an open source RPC Framework that is available from Google and delivers data through Google Protocol Buffer-based Binary Protocol. Protocol Buffer is one of the ways to serialize structured data, and by defining the Interface Description Language (IDL) Proto, it automatically generates serialized models in various languages. This way users can easily understand what data is sent and received without any separate documentation.
4. Overall Architecture
Using Electron, Java, JDBC and gRPC, we’ve configured QueryPie’s overall architecture as follows:
5. Engine Exception Handler: Error Tracking
No matter how high test coverage and meticulous Q/A were done in the product development phase, it is difficult to fully test all environments. This includes hardware, OS, and OS patches for customers using the product. So it is important to quickly detect and reproduce the errors that occur.
One of the most important aspects of engine development is the accurate and detailed recording of errors throughout the engine so that if an error occurs in the product, it can reproduced 100% or quickly within the company’s internal environment.
We used gRPC Interceptor, SLF4J MDC, and Logback Custom Appender to track in detail what user behavior instigated the error and what the situation was like at the time.
As previously described in the architecture, QueryPie Middleware requests the engine for the required data via gRPC to handle user behavior or requests. In order to track which gRPC Endpoint was called by Middleware, the gRPC Interceptor always has API Request Information on MDC, which is to be retrieved from the Logback Appender when an error occurs.
To track all error logs, we added a QueryPieLogAppender to the ROOT Appender so we could record all log.error (which is taken via SLFJ4). This error log information is sent to the server with the user’s consent, and the server quickly detects and reproduces errors based on the log data to resolve the problem.
6. Application Performance Monitoring: Skywalking
Finally, the performance measurement of the application remains. This should be a preconceived idea because it’s much easier to start measuring the performance of an application from the development process than to track it at once after the development is complete.
Consideration is required for APM behavior from the beginning of the development process because some of the features in QueryPie have some aspects that might be slow or take much longer than expected once tested, which might not be foreseen during development.
If all developers flew an APM-linked engine locally, they wouldn’t be able to identify the exact metric. So we decided to activate the Engine on the Remote Server and configured it to run with the APM.
If Git Push occurs, Jenkins automatically builds the engine and distributes it to the remote server and restarts it. So front-end and middleware developers can access the Remote Engine without having to run an engine locally. All testing and execution records during development are recorded in the APM database. Among the many APMs out there, we chose the Apache Incubation Project called Sky Walking, which has simple installation and clean UI, topology maps, as well as gRPC support and run-time tracking.
We are constantly finding effective ways to quickly find and improve bottlenecks by analyzing them at the development stage with APM application. And we hope that these processes will enable us to operate with high quality and superior performance when the product is delivered to the users.
In Conclusion:
QueryPie is gaining momentum every day. We are constantly working with our best engineers to build a cornerstone with the right technology, to improve usability, and solve problems that traditional users find uncomfortable. Quite a few key features have been established and we’ve successfully been able to run the product in-house. We can’t wait to see how our users will use and enjoy these features!
To become the world’s leading database tool, QueryPie is being actively developed even as you read this development log. Please just wait a bit more for this great experience!
