Optimization can be done in many parts → such as portfolio optimization or device optimization and more. (and data fitting → which is machine learning and more).
This seems to be more or less → civil engineering problem → we want to optimize many different functions → within some of the constraints. (optimize the world). (machine learning and statistics and more!).
The analytical solution does not mean that the problem is 100% solvable.
How do we test the problem the least square? → it is very easy to note? → easy to find. (very effective tricks).
A lot of problems can be formed as linear programming → might not have an analytical solution.
Convex problem → hard to recognize but we can solve them → and we can solve this.
Local optimization for non-linear → might not be the solution → the filed of ML. (we can give up on speed).