Dec 10, 2021
2 min read
Dynamic Column in Azure Data Factory Flow
Based on the requirement, we need load source data into Landing Table, using Azure Data Factory.
primary key is deviceId, day 1’s value need mapping to column Mon. day 2’s value need mapping to column Tue, and so on.
my data flow is:
link source to the txt file.
Add one parameter: ColumnName, calculate the column name in pipeline (by filename or other logic) and pass it to data flow.
in the Select1, add a rule-based mapping column, “locate(‘data’, lower(name)) != 0” then set the column as “$ColumnName”
then add DerivedColumn1, to setup all calculate columns.
finally, we got the result like this:
we can sink the data into the database.
