Jean-Claude CoteinTowards Data ScienceOptimizing Sigma Rules in Spark with the Aho-Corasick AlgorithmExtending Spark for improved performance in handling multiple search termsJun 20Jun 20
Jean-Claude CoteinTowards Data SciencePerformance Insights from Sigma Rule Detections in Spark StreamingUtilizing Sigma rules for anomaly detection in cybersecurity logs: A study on performance optimizationJun 1Jun 1
Jean-Claude CoteinTowards Data SciencePerformant IPv4 Range Spark JoinsA Practical guide to optimizing non-equi joins in SparkJan 251Jan 251
Jean-Claude CoteinTowards Data ScienceUnleashing the Power of SQL Analytical Window Functions: A Deep Dive into Fusing IPv4 BlocksHow to summarize a geolocation table by merging contiguous network IPv4 blocksJan 10Jan 10
Jean-Claude CoteinTowards Data ScienceAnomaly Detection Using Sigma Rules: Build Your Own Spark Streaming DetectionsEasily deploy Sigma rules in Spark streaming pipelines: a future-proof solution supporting the upcoming Sigma 2 specificationJun 12, 2023Jun 12, 2023
Jean-Claude CoteinTowards Data ScienceAnomaly Detection using Sigma Rules (Part 5) Flux Capacitor OptimizationTo boost performance, we implement a forgetful bloom filter and a custom Spark state store providerMar 17, 2023Mar 17, 2023
Jean-Claude CoteinTowards Data ScienceAnomaly Detection using Sigma Rules (Part 4): Flux Capacitor DesignWe implement a Spark structured streaming stateful mapping function to handle temporal proximity correlations in cyber security logsMar 2, 2023Mar 2, 2023
Jean-Claude CoteinTowards Data ScienceAnomaly Detection using Sigma Rules (Part 3) Temporal Correlation Using Bloom FiltersCan a custom tailor made stateful mapping function based on bloom filters outperform the generic Spark stream-stream join?Feb 14, 2023Feb 14, 2023
Jean-Claude CoteinTowards Data ScienceAnomaly Detection using Sigma Rules (Part 2) Spark Stream-Stream JoinA class of Sigma rules detect temporal correlations. We evaluate the scalability of Spark’s stateful symmetric stream-stream join to…Feb 2, 2023Feb 2, 2023
Jean-Claude CoteinTowards Data ScienceAnomaly Detection using Sigma Rules (Part 1): Leveraging Spark SQL StreamingSigma rules are used to detect anomalies in cyber security logs. We use Spark structured streaming to evaluate Sigma rules at scale.Jan 24, 2023Jan 24, 2023