Create your own command line argument with argparse and parametrize a run of prophet forecast model

Image for post
Image for post

What’s Argparse ?

Parameterize your code : productivity, flexibility and more quality.

import argparse
parser = argparse.ArgumentParser()

Forecast several time series at once with prophet and pandas UDF without looping.

Image for post
Image for post
  • spark ≥ 2.4
  • pyarrow ≤ 0.14.1 (above this version there’s some issue)
sudo pip3 install pyarrow=0.14.1
from pyspark.sql import SparkSession
from pyspark.sql.types import *
from pyspark.sql.functions import pandas_udf, PandasUDFType, sum, max, col, concat, lit
import sys
import os
# setup to work around with pandas udf
# see answers here
os.environ["ARROW_PRE_0_15_IPC_FORMAT"] = "1"
from fbprophet import Prophet
import pandas as pd
import numpy as np

Image for post
Image for post
  • a popularity based recommender
  • a content based recommender (Through KNN, TFIDF, Transfert Learning)
  • a user based recommender
  • a hybrid recommender
  • a deep learning recommender


A full guide from preprocessing to modeling

Image for post
Image for post
  1. Import data
  2. Filter data
  3. Features engineering (features creation)
  4. Imputing data
  5. Features engineering (features transformation)
  6. Applying a gradient boosted tree regressor
  7. Optimise the model with Kfold and GridSearch Method
  8. Oneshot

I) Import data

from pyspark.sql.types import *schema = StructType([
StructField("DATE", DateType()),
StructField("STORE", IntegerType()),
StructField("NUMBERS_OF_TICKETS", IntegerType()),
StructField("QTY", IntegerType()),
StructField("CA", DoubleType()),
StructField("FORMAT", StringType())])
df ="gs://my_bucket/my_table_in_csv_format"…

Build a CNN classifier and turn it to a recommendation engine.

Image for post
Image for post

Calcul, aggregate, transform any data

Image for post
Image for post

A brief guide to import data with Spark

Image for post
Image for post

Import a CSV

csv_2_df ="gs://my_buckets/poland_ks")#print it

Image for post
Image for post

Qu’est-ce que Scrapy ?

Pourquoi choisir Scrapy ?

A journey through the world of pre-trained models applied to image recommendation

Image result for transfer learning

A guide to integrating variable creation into a spark pipeline

Image for post
Image for post
| date|sales|
|2018-12-22| 17|
|2017-01-08| 22|
|2015-08-25| 48|
|2015-03-12| 150|

Alexandre Wrg

Data scientist at Auchan Retail Data

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store