We all know that, Apache Spark is a fast and a general engine for large-scale data processing. It can process data up to 100x faster than Hadoop MapReduce in memory, or 10x faster on disk.
For all these years, we have have been galvanized by the advancements in the field of computing. Most of it is aligned with the introduction of better and powerful CPUs. First there was a single CPU whose speed was the main focus. Once, that part hit the wall, there were more cores on the CPUs. Today…