Necati Demir

#machinelearning #datascience , Chief AI Officer @ http://datagran.io , ML Consultant

Highlighted by Necati Demir

From Forecasting Releases Isn’t About Deadlines; It’s About Probability by Cycligent

…total amount of estimated hours which gets reduced each day is the ‘speed’ for that day. Ie, you had 10 hours of actions, and after updating you have 8 hours or work left, you get a speed of 2. (If the…

From Local testing with Google Cloud Composer — Apache Airflow by John Bencina

At this point you can simply move your DAG(s) to your Cloud Composer environment using the Google Cloud SDK

From A Simple and Scalable Analytics Pipeline by Ben Weber

The core piece of technology I’m using to implement this data pipeline is Google’s DataFlow, which is now integrated with the Apache Beam library. DataFlow tasks define a graph of operations to perform on a collection of events, which can be streaming data sources. This post presents a DataFlow task implemented in Java that streams tracking events from a PubSub topic to a data lake and to BigQuery. An introduction to DataFlow and it’s concepts is available in Google’s documentation. While DataFlow tasks are portable, since they are now based on Apache Beam, this post focuses on how to use DataFlow in conjunction with additional managed services on GCP to build a simple, serverless, and scalable data pipeline.