Member-only story

Notes from Industry

Impact of Computational Complexity on your Product Value

Pick the most valuable algorithm for your product

Venkatesh Pappakrishnan, Ph.D.
Towards Data Science
7 min readJan 3, 2022

--

The importance of computational complexity including both time- and space- complexities are underestimated especially for the algorithms that becomes a part of a product. For general algorithm development for competitions such as offered by Kaggle, data analysis using ML, model development and optimization for proof-of-concept (POC) the computational complexity is of low importance. However, for the algorithms that go into production systems as a product, computational complexity should be taken seriously due to limited resource availability, time constraints for prediction and model building, and most importantly cost incurred from the cloud resources.

The computational complexity plays a major role in model training where tuning and optimization are required compared to prediction where the complexity is usually O(n*p) where n is the number of predictions and p is the number of features in the data set.

Model Building Paradigms:

  1. Streaming: Whenever you get a new datum we re-build models using all (or limited) the previous data. Possibly most accurate but expensive, time consuming, and delay in prediction. Ideal for certain…

--

--

Towards Data Science
Towards Data Science

Published in Towards Data Science

Your home for data science and AI. The world’s leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.

Venkatesh Pappakrishnan, Ph.D.
Venkatesh Pappakrishnan, Ph.D.

Written by Venkatesh Pappakrishnan, Ph.D.

Data Scientist | Physicist | Entrepreneur | Book Author | Content Writer

No responses yet