Create your own command line argument with argparse and parametrize a run of prophet forecast model

Image for post
Image for post
https://unsplash.com/photos/qjnAnF0jIGk

What’s Argparse ?

Parameterize your code : productivity, flexibility and more quality.

import argparse
parser = argparse.ArgumentParser()


Forecast several time series at once with prophet and pandas UDF without looping.

Image for post
Image for post
  • spark ≥ 2.4
  • pyarrow ≤ 0.14.1 (above this version there’s some issue)
sudo pip3 install pyarrow=0.14.1
from pyspark.sql import SparkSession
from pyspark.sql.types import *
from pyspark.sql.functions import pandas_udf, PandasUDFType, sum, max, col, concat, lit
import sys
import os
# setup to work around with pandas udf
# see answers here https://stackoverflow.com/questions/58458415/pandas-scalar-udf-failing-illegalargumentexception
os.environ["ARROW_PRE_0_15_IPC_FORMAT"] = "1"
from fbprophet import Prophet
import pandas as pd
import numpy as np


Image for post
Image for post
  • a popularity based recommender
  • a content based recommender (Through KNN, TFIDF, Transfert Learning)
  • a user based recommender
  • a hybrid recommender
  • a deep learning recommender

Introduction


A full guide from preprocessing to modeling

Image for post
Image for post
+
  1. Import data
  2. Filter data
  3. Features engineering (features creation)
  4. Imputing data
  5. Features engineering (features transformation)
  6. Applying a gradient boosted tree regressor
  7. Optimise the model with Kfold and GridSearch Method
  8. Oneshot

I) Import data

from pyspark.sql.types import *schema = StructType([
StructField("DATE", DateType()),
StructField("STORE", IntegerType()),
StructField("NUMBERS_OF_TICKETS", IntegerType()),
StructField("QTY", IntegerType()),
StructField("CA", DoubleType()),
StructField("FORMAT", StringType())])
df = spark.read.csv("gs://my_bucket/my_table_in_csv_format"…


Build a CNN classifier and turn it to a recommendation engine.

Image for post
Image for post


Calcul, aggregate, transform any data

Image for post
Image for post


A brief guide to import data with Spark

Image for post
Image for post

Import a CSV

csv_2_df = spark.read.csv("gs://my_buckets/poland_ks")#print it
csv_2_df.show()


Image for post
Image for post

Qu’est-ce que Scrapy ?

Pourquoi choisir Scrapy ?


A journey through the world of pre-trained models applied to image recommendation

Image result for transfer learning


A guide to integrating variable creation into a spark pipeline

Image for post
Image for post
df.show()+----------+-----+
| date|sales|
+----------+-----+
|2018-12-22| 17|
|2017-01-08| 22|
|2015-08-25| 48|
|2015-03-12| 150|
+----------+-----+

Alexandre Wrg

Data scientist at Auchan Retail Data

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store