Sigma rules are used to detect anomalies in cyber security logs. We use Spark structured streaming to evaluate Sigma rules at scale. — The Rise of Data Sketching Data sketch is an umbrella term for data structures and algorithms that use theoretical mathematics, statistics and computer science to solve set cardinality, quantiles, frequency estimation, with mathematically proven error bounds. Data sketches are orders-of magnitude faster than traditional approaches, they require less compute resources and sometimes are the only…
