DPR — A Tool for Data Prep and Scripting

Rajat Gupta
Jan 25, 2018 · 2 min read
{"type": "compare", "sourceDataset": "CustDBDump", "compareToDataset": "NewCustInput", "sourceMatchFields": "emailaddr", "source2MatchFields": "email_address", "sourceCompareFields": "firstname,lastname,company,phone", "source2CompareFields": "first_name,last_name,account_name,primary_phone"}
{"type": "readExcel", "file": "custfile.xlsx", "worksheet": "Sheet1", "startCell": "a1", "endCell": "d1"}
"settings": { "dbConnections": [
{"name": "CustDB", "dbtype": "MySQL", "parameters": [
{"name": "server", "value": "localhost"},
{"name": "port", "value": "3306"},
{"name": "databaseid", "value": "CustomerDB"},
{"name": "username", "value": "rajat"},
{"name": "password", "value": "mysecret"}
]}
]}
{"type": "dbQuery", "dataSource": "CustDB", "sql": "SELECT * FROM V_USER_LIST"}
{ "settings": { "dbConnections": ...},
"project": { "storageMethod": "file",
"processes": [
{"name": "CustDBDump", "steps": [
{"type": "dbQuery"....}
]},
{"name": "NewCustInput", "steps": [
{"type": "readExcel"....}
]},
{"name": "CustDiff", "steps": [
{"type": "compare"....}
]},
],
"taskLists": [
{"name": "RunDiff", "tasks": [
{"type": "executeProcess", "process": "CustDBDump"},
{"type": "executeProcess", "process": "NewCustInput"},
{"type": "executeProcess", "process": "CustDiff"}
]}
]
}
cd c:\users\rajat\CustCampaignAnalysis
dpr
dpr> run RunDiff
Rajat Gupta

Written by

Qvikly Lists is the simplest tool to gather and share information, with tasks, activity streams, and history. Now available at http://qvikly.com.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade