A Situation where Process is better than a Library
I recently had an argument with one of my colleagues about expanding the capabilities of a crucial application(consists of highly important functionality) by introducing a 3rd party library as a dependency for that application.
Following are the two basic ways of giving that capability to the application.
- Adding capability as a 3rd party library - In this method, the new capability is added as an internal compilation dependency for your program.
- Using capability as another process - In this methods, the new capability will not be an internal compilation dependency. It gets compiled separately and will be run as another process. New capability can be used as a service exposed by that particular process.
I would have agreed with my colleague’s suggestion if we are going to add this capability as the main function of the application or as an extension to the main functionality. However, our context is as follows.
- The mainly expected functionality of the application should be well tested minimal component.
- Capabilities which are newly added change frequently. Hence frequent deployments are needed.
- Environments cannot be assumed as homogeneous.
One of the arguments of my colleague is, introducing another process will be an overhead to the system as we need to allocate resources to it even though we don’t use that process frequently.
However, when using a library, it will be an internal compilation dependency for our program which adds another complexity by introducing newer code lines.
Errors which damage the Process Address Space
One may think that 3rd party library cannot crash the application as long as we properly handle the places where that library is being used. However, if the 3rd party library does something which damages the Process Address Space, nothing can be done to prevent it. Memory Overflows, Stack Overflows, and various kill commands are such situations.
Thread Crash vs Process Crash
A crashing process won’t bring down other processes, whereas a crashing thread will probably wreak havoc with other threads.
Not all thread exceptions can be captured with try-catch. Threads can do much more than raising an exception. A rogue thread can, via buggy native or ctypes code, trash memory structures anywhere in the process, including the python runtime itself, thus corrupting the entire process.
The problem of having unmet dependencies
When we introduce 3rd party library we need to make sure all dependencies which are expected by that library as prerequisites should be satisfied. This is not a huge problem if we can confirm the homogeneity of target environments. If environments are heterogeneous the compilation of the codebase will be a failure.
Lack of autonomy for the application
Whenever you need to do major transformations(i.e switch the entire language or the version which it is written in) to the application, you need to make sure the 3rd party library is also migrated properly.
Cannot change(i.e introduce new version) unless you recompile/redeploy your program
Every time when you introduce new versions of the 3rd party library you need to deploy the whole program again to take that new version. This is not a good condition for an application whose fundamental functionality is another thing. Recompiling the program for another purpose other than for amending above fundamental functionality can be an additional risk for the application.
Special Situations like Global Interpreter Lock in Python
In Python, because of GIL (Global Interpreter Lock), a single python process cannot run threads in parallel (utilize multiple cores). It can, however, run them concurrently (context switch during I/O bound operations)
