link: https://platform.openai.com/chat/edit?optimize=true

Prompt optmizer uses the guidelines proposed by OpenAI itself for the gpt 5 model.

link: https://cookbook.openai.com/examples/gpt-5/gpt-5_prompting_guide

💡 How does it work?
1. Choose the model for which you want to optimize the prompt

2. Paste your original prompt

3. Click “Optimize”

4. The tool rewrites it, applying the official prompting guidelines for the various models (those published by OpenAI itself)

5. Use the new optimized prompt

🎯 Result:
- Highlights the steps to follow
- Eliminates ambiguity
- Clearly defines inputs and outputs
- Ability to save “optimized” versions and reuse them

Conclusion:

This tool is really very useful to make prompts complete and effective

Definition:
Data products are foundational components in a modern data ecosystem, particularly within a data mesh architecture. They are designed to serve analytical data effectively, applying the principles and practices of software product development to the data domain. By doing so, data products ensure data is accessible, understandable, trustworthy, and valuable to its users.

Key Characteristics of Data Products

The following attributes define an effective data product:

1. Discoverable
Users should easily locate and understand the data products available to them. This involves robust metadata, intuitive search, and clear documentation.

2. Addressable
Each data product must have a unique identifier (e.g., a URI or URL), ensuring consistent and reliable access programmatically or manually.

3. Understandable (Self-Describable)
Comprehensive documentation is critical. This includes the data product’s purpose, structure, field-level descriptions, usage instructions, and sample data for easy evaluation.

4. Trustworthy
Trust is built through transparency about the product’s quality, performance, and reliability. This includes publishing Service Level Objectives (SLOs) and adhering to measurable Service Level Indicators (SLIs).

5. Natively Accessible
Data products should be tailored for various consumer needs:

• Managers might need prebuilt reports.
• Data scientists could require SQL-based access.
• Engineers might prefer API-based programmatic access.

6. Interoperable (Composable)
To enable seamless integration with other data products, interoperability is key. This includes supporting standard business keys and common access patterns for joining, filtering, and aggregating data.

7. Valuable on Its Own
A data product must deliver independent value, representing a cohesive concept within its domain without requiring additional context or joins to be useful.

8. Secure
Strong access controls, encryption, and compliance with domain-specific regulations ensure the data product is protected against unauthorized use.

Note: to learn more about the topic see Zhamak Dehghani’s book “Data Mesh: Delivering Data-Driven Value at Scale”

Functional Perspective

A data product is not merely a dataset; it is a packaged, self-contained entity that:

• Provides structured, semi-structured, or unstructured analytical data.
• Caters to specific user groups and their data consumption patterns.
• Embeds necessary automation and infrastructure to efficiently achieve its purpose.

Business Value of Data Products

By aligning with the principles of modern software development, data products:

• Decompose large, monolithic data systems into manageable, reusable components.
• Encourage scalability, discoverability, and maintainability.
• Support consistent delivery of meaningful business insights and value.

Data products are the smallest valuable unit of analytical data within a data mesh, facilitating data-driven decision-making while prioritizing usability and reliability for diverse user personas.

What Data Products Are Not

A robust definition of data products requires not only specifying what they are but also clarifying what they aren’t. By doing so, organizations can avoid common pitfalls, ensure focus, and streamline the creation and management of effective data products.

Why Clarity Matters

• A narrower, well-defined scope prevents confusion across teams.
• It enables the development of reusable blueprints and “paved paths” that simplify building and managing data products.
• It maintains focus on solving problems specific to analytical data rather than trying to address all data-related challenges simultaneously.

Common Misrepresentations and Missing Characteristics

Key Differentiators

• Data Products vs. Data-Driven Applications:
• Data Products are designed for programmatic access and composability, making them building blocks in a data mesh.
• Data-Driven Applications are intended for human interaction and lack inherent composability.
• Focus on Analytical Data:
  Data products apply software engineering principles to analytical data to address ownership and quality issues, but they are not the catch-all solution for broader enterprise data challenges.

Why Misunderstandings Happen

• Conflation of Concepts: Teams sometimes mix up data products with other data artifacts or tools due to the overlap in terminology or functionality.
• Diluted Language and Focus: Attempting to solve all ownership and quality issues under the banner of data mesh risks diluting the transformative impact of focused data product design.

Conclusion

By clearly distinguishing what data products are and aren’t, teams can maintain focus, improve clarity, and build scalable, reusable solutions that align with the principles of a data mesh. This ensures data products deliver on their promise of creating composable, high-quality analytical data units that drive business value.

Credits: https://martinfowler.com/articles/designing-data-products.html

In this short DAX pill we see how to find the first and the last value of a given column within a selected time range.

the context is that of the analysis of the trend of a cryptocurrency (e.g. BITCOIN) in a predefined time frame (e.g. quarter)

Let’s create the following calculated measurs:

1. “FirstAdjClose”
2. “LastAdjClose”

where:

CryptoHistorical is the table where we have the historical data of a cryptocurrency

FIRSTDATE is the function that returns the first date in the current context for the specified date column.

LASTDATE is the function that returns the last date in the current context for the specified date column.

see the link:

https://learn.microsoft.com/it-it/dax/firstdate-function-dax

https://learn.microsoft.com/it-it/dax/lastdate-function-dax

the DAX sintax is the following:

FirstAdjClose =

VAR FirstDateValue = FIRSTDATE(CryptoHistorical[historical.date])

VAR result = CALCULATE(MAX(CryptoHistorical[historical.adjClose]), FILTER(CryptoHistorical, CryptoHistorical[historical.date]=FirstDateValue))

RETURN result

**********************************************************************

LastAdjClose =

VAR LastDateValue = LASTDATE(CryptoHistorical[historical.date])

VAR result = CALCULATE(MAX(CryptoHistorical[historical.adjClose]), FILTER(CryptoHistorical, CryptoHistorical[historical.date]=LastDateValue))

RETURN result

**********************************************************************

Note that the date (first or last) is stored in a variable and then passed as a filter to the CALCULATE function

Reference to my previous post about Api economy:

https://blog.devgenius.io/the-apis-economy-c3ecd72061cd

I will talk about APIs available online and usable for real purposes and applications, I will talk about tools for API development, I will also talk about products (e.g. API marketplace) dedicated to the world of APIs.

In this first pill we will see a very useful list of APIs and resources (work in progress) relating to the world of weather (current, forecast), air-quality and other data.

AccuWeather:

Weather and forecast data.

Access takes place via an ApiKey in HTTP mode and there are various free and paid plans for using the APIs

API Reference

Weatherstack:

Real-Time & Historical World Weather Data API.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs

Weatherstack - Real-Time World Weather REST API

OpenWeatherMap:

Weather forecasts, nowcasts and history data.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs

Weather API

WeatherApi:

JSON and XML Weather API and Geolocation API

Access takes place via an ApiKey in HTTP or HTTPS mode and there are various free and paid plans for using the APIs. Support for CORS is present.

Weather and Geolocation API - Weather and Geolocation API JSON and XML - WeatherAPI.com

US Weather API:

Data from US National Weather Service.

All of the information presented via the API is intended to be open data, free to use for any purpose. Access takes place in HTTPS mode.

API Web Service

WeatherBit:

Current, forecasts and alert weather data. Air quality data.

Access takes place via an ApiKey in HTTP or HTTPS mode and there are various free and paid plans for using the APIs.

API Documentation

Tomorrow.io:

Enterprise-grade Weather API (current, real-time, forecast)

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

Tomorrow.io APIs

Stormglass.io:

Weather forecasts & historical data from the world’s most trusted meteorological institutions. Include marine weather (wave and wind).

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

stormglass.io Weather API

M3O Weather:

Real time weather forecast.

Access takes place via an ApiKey in HTTPS mode and there are various free and paid plans for using the APIs.

Weather API - Real time weather forecast

OpenSenseMap:

OpenSenseMap is an open sensor data platform that anyone can participate in. Data comes from Personal Weather Stations called “SenseBoxes”. Air-quality, temperature and humidity data are available.

Access requires token authentication only for some APIs. Access in HTTP mode.

openSenseMap.org

INGV Earthquake events:

Italian open data on earthquakes.

All of the information presented via the API is intended to be open data, access is free and does not require ApiKey. Access in HTTP mode.

INGV Earthquake event

API economy : pills n.1 — A very useful collection of weather API was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is an API:

An API (Application Programming Interface) is a tool that allows companies to make the data and functionality of their applications available to external developers, business partners and internal departments of the company itself. This allows different services and products to communicate with each other and leverage each other’s information and functionality via a well-documented interface. Developers don’t need to know the technical details of how the API was created; they simply use the interface to interact with other products and services. The use of APIs has grown significantly over the last decade, so much so that many of the most popular web applications today could not exist without the use of APIs.

How the API works:

An API (Application Programming Interface) is a set of clearly defined rules that dictate how computers or applications should communicate with each other. APIs serve as an intermediary layer situated between an application and a web server, facilitating the processing of data exchange between different systems.

Here’s how an API works:

1. A client application initiates an API call, often referred to as a request, to obtain specific information. This request is transmitted from the client application to the web server using the API’s Uniform Resource Identifier (URI). It typically includes a request verb, headers, and sometimes a request body.

2. Once a valid request is received, the API acts as an intermediary and makes a call to the external program or web server that holds the requested data or functionality.

3. The server processes the request and sends a response back to the API, containing the requested information.

4. The API then passes this data back to the initial requesting client application.

This request and response process takes place entirely through the API. While user interfaces are designed for human interaction, APIs are specifically designed for computer or application-to-application communication.

APIs are designed with security in mind. Their intermediate position helps simplify the abstraction of functionality between two systems, with the API endpoint acting as a separation layer between the requesting application and the underlying infrastructure providing the service. Here are some key aspects of API security:

1. Authorization Credentials: API calls typically include authorization credentials, such as API keys or tokens. These credentials help verify the identity of the requesting application and reduce the risk of unauthorized access to the server.

2. API Gateway: An API gateway can be used to manage and secure API access. It serves as a central entry point for API requests and can enforce security policies, throttle requests, and provide logging and monitoring to minimize security threats.

3. HTTP Headers, Cookies, and Query Parameters: During the data exchange between the client and server via the API, HTTP headers, cookies, and query string parameters can be used to add additional layers of security. For example, secure cookies can help maintain session state, and headers can be used to specify content types and enforce security policies.

4. Rate Limiting: API security often includes rate limiting to prevent abuse. By restricting the number of requests a client can make within a specific time frame, it helps protect the server from potential denial-of-service attacks.

5. Encryption: To secure data in transit, APIs commonly use encryption protocols such as HTTPS (HTTP Secure) to encrypt the communication between the client and server, ensuring that data remains confidential and cannot be easily intercepted.

6. Authentication: APIs can implement various authentication methods, such as OAuth, API keys, or token-based authentication, to verify the identity of clients and grant access only to authorized users or applications.

7. Access Control: Access control mechanisms within APIs ensure that users or applications can only access the data and functionality they are authorized to use. This helps prevent unauthorized access to sensitive information.

In summary, APIs incorporate multiple layers of security measures, including authentication, authorization, encryption, and access control, to protect both the API provider’s data and the users of the API. These security features are essential for ensuring the integrity and confidentiality of data exchanged through APIs.

For instance, let’s take the case of an API provided by a payment processing service. In an e-commerce store application, customers input their card details into the frontend. Instead of needing access to the user’s bank account directly, the payment processor’s API generates a unique token for that specific transaction and includes it in the API request sent to the server. This approach enhances security by reducing the risk of potential hacking threats because the actual card details are not exposed during the transaction.

Why we need APIs:

Using APIs offers numerous advantages for both managing existing tools and designing new ones. Some of the key benefits of APIs include:

1. Improving Collaboration: Many companies use a wide range of cloud applications, often not integrated with each other. APIs enable integration, allowing these platforms to communicate seamlessly. This integration automates workflows and improves workplace collaboration. Without APIs, companies would be forced to use siled data, compromising productivity and performance.

2. Simple Innovation: APIs offer flexibility, allowing companies to establish connections with new business partners, offer new services to their existing market and enter new markets. For example, Stripe started as an API with a few lines of code and has grown into a billion-dollar company through partnerships with large companies and diversification of services.

3. Data Monetization: Many companies start by offering APIs for free to build a developer base and establish relationships with potential partners. However, if the API offers access to valuable data, you can monetize it by selling access. AccuWeather, for example, has attracted thousands of developers and created a growing community by selling API keys.

4. Added Security: APIs add an extra layer of security between the data and the server. Developers can further enhance API security using tokens, digital signatures, and Transport Layer Security (TLS) encryption. Using API gateways to manage and authenticate traffic helps further protect resources. Additionally, effective API management is critical to ensuring secure operations.

In summary, APIs are essential tools that facilitate collaboration, foster innovation, enable data monetization, and strengthen the security of business applications and services. They are a key element in digital transformation and the creation of new business opportunities.

Types of API protocols:

As the use of Web APIs has increased, various protocols have been developed to provide users with a set of well-defined rules that specify the commands and data types accepted. These API protocols simplify the standardized exchange of information. Here are some of them:

1. SOAP (Simple Object Access Protocol): This is an XML-based API protocol that allows users to send and receive data via protocols such as SMTP and HTTP. SOAP APIs make it easy to exchange information between applications or software components running in different environments or written in different programming languages. It is known for its robustness and support for complex operations.

2. XML-RPC: This protocol uses the XML format for data transfer, but unlike SOAP, it uses a simpler XML format. XML-RPC is older than SOAP but is known for its lightweight and minimal bandwidth usage.

3. JSON-RPC: Similar to XML-RPC, but uses JSON (JavaScript Object Notation) format for data transfer. Both protocols are known for their simplicity and support remote procedure calls (RPCs) with parameters and results.

4. REST (Representational State Transfer): This is a principle-based Web API architecture rather than a specific protocol. REST APIs (known as RESTful APIs) must respect certain architectural constraints, such as the use of URIs (Uniform Resource Identifier) ​​to identify resources and the use of HTTP methods (GET, POST, PUT, DELETE) to interact with them . There is no official standard for RESTful APIs, but they follow a set of key API design principles.

It’s important to note that while it’s possible to build RESTful APIs using protocols like SOAP, the two approaches are often thought of as competing specifications and follow different design philosophies. The choice of protocol often depends on the specific needs of the project and the preferences of the development team.

Common API examples:

APIs have become a valuable asset in the modern business world as they allow companies to offer access to their resources while maintaining security and control. Below are some popular examples of APIs you may encounter:

1. Universal Logins: Many platforms allow users to log in to websites using the login details of their accounts on services such as Facebook, Twitter or Google. This is made possible through APIs, which allow websites to use the authentication credentials of these services to simplify the login process. This saves users the hassle of having to create new accounts for every web service they use.

2. Third Party Payment Processing: Services like “Pay with PayPal” on e-commerce websites use APIs to allow users to make payments securely without having to share sensitive financial information with the website. Payment APIs enable secure and convenient transactions.

3. Travel Booking Comparisons: Travel booking websites aggregate information on flights, hotels and other travel options from multiple sources. This is possible thanks to APIs that provide application users access to a wide range of data updated in real time. Users can easily compare travel options without having to manually consult different sources.

4. Google Maps: Google Maps offers a set of APIs that allow developers to integrate interactive maps and geolocation services into their applications. These APIs allow users to view maps, get directions, find places of interest, and much more. Google Maps APIs are widely used in web and mobile applications.

5. Twitter: Twitter offers an API that allows developers to access tweet data and interact with the platform. This allows developers to create applications and services that use public tweets and replies. For example, social media management applications use the Twitter API to publish tweets and analyze social media activity.

In summary, APIs are critical to building modern services and applications that rely on data and functionality provided by third parties. They enable integration, security and simplified access to a wide range of digital assets, improving the user experience and expanding business opportunities.

The APIs economy was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

DAX FILTER Function

The FILTER function in DAX allows you to specify a filter criterion on the data.

The FILTER function return a filtered table.

The FILTER function has 2 parameters:

• Table: Iterator, is the table to be filtered
• FilterExpression: RowContext, is a boolean (True/False) expression that is to be evaluated for each row of the table

Syntax:

FILTER( <Table> , <FilterExpression> )

What are DAX Filter Function Used For?

You can use filter DAX Power BI functions in a variety of scenarios, like when you want to:

• Reduce the number of rows in a table
• Hide irrelevant columns or rows
• Perform arithmetic operations on predefined fields
• Calculate new parameters or new measures
• Create dynamic calculations
• Manipulate data context

DAX FILTER Function in action

Now let’s see some practical examples of using the FILTER function. We will use as an example the dataset “Employee hiring and history sample” which you can find at the following link

powerbi-desktop-samples/powerbi-service-samples at main · microsoft/powerbi-desktop-samples

in this dataset:

Employee = fact table of employee data

TermDate = date of termination of employment

if we use the function:

FILTER(Employee, ISBLANK(Employee[TermDate]))

we get all the employees (of the “Employee” table) whose “TermDate” is empty and therefore are active.

Note that the ISBLANK function only returns records that have the “TermDate” field equal to “blank”

if we create the measure:

Actives =
CALCULATE ( [EmpCount], FILTER ( Employee, ISBLANK ( Employee[TermDate] ) ) )

we get the count of Actives employee

Now we want to create a measure that filters the data in the “Employee” table based on a related dimension. For example the region “North” in the BU dimension

EmpRegionNorth =
CALCULATE (
[EmpCount],
FILTER ( Employee, RELATED ( ‘BU’[Region] ) = “North” )
)

in this case we get the count of Employees that belong to the North region

Note that the RELATED function allows us to use a filter criterion that is on the “BU” dimension of our data model.

Other FILTER functions

The filter and value functions in DAX are some of the most complex and powerful, and differ greatly from Excel functions. The lookup functions work by using tables and relationships, like a database. The filtering functions let you manipulate data context to create dynamic calculations.

At the following link the Microsoft page that the list of filter functions.

https://learn.microsoft.com/en-us/dax/filter-functions-dax

in these sectors, as in others, the data sources can be heterogeneous both in terms of format and the technology with which the data is made available. Furthermore, it is often necessary to interface with third-party software (such as SAP, Oracle or Microsoft) to have certain information available.

The project that I will talk about in this post concerns a DATA ANALYTICS solution for the construction sector.

The project is based on the Microsoft technology stack (Azure Data Lake, Azure Synapse Pipelines, Azure Synapse Analytics, PowerBI) and can be schematized in the figure below to get a general picture of the information and the components and technologies used.

DATA SOURCES

Data sources can be heterogeneous

Relational databases (on-premises): to get data from ERP software (ex. SAP) , cost estimation software (ex. TeamSystem Construction) or other operational software

API: to get data from cloud apps or services (ex. IoT sensor or equipment management platforms)

Files: to get data from user input or other application that dont have services or API

ORCHESTRATION

Azure Data Lake (Gen2): to store the data

Through a procedure implemented by my development team (in c# or python), or with the help of Azure Synapse Pipelines (or Azure Data Factory), we are going to save the information coming from the data sources in a datalake on Azure.

The data is saved in “parquet” type files inside the datalake.

Azure Synapse Analytics: to manage and monitor the data in the data lake

ANALYSIS & VISUALIZATION

Power BI: to model and analyze the data

the data model on which to build the analyzes and visualizations to be made available to users

Built with a star schema to optimize performance and ease analysis definition

Power BI: to visualize the data

above the data model we built visualizations to extract the maximum value for the end user.

The visualizations are designed according to the typologies and expectations of the end users

REPORTS, ANALYTICS & APPLICATIONS

Reports: To guide the user in data navigation

Analytics: To offer the user specific “insights” about the data

Applications: to publish the data

through the publication of data on the PowerBI service, users can share information (for those who have permission)

by creating a PowerBI app we can give access to it from other applications (embeddings) cloud or desktop

Conclusions:

There are many tools available and sometimes identifying the most useful ones for the result may not be simple, however the correct definition/collection of the end user’s needs and expectations guides us in designing the most effective solution.

In representations involving the time dimension, there is often the need to display data, such as amounts, in progressive rather than absolute terms.

the running total is a value that increases (or decreases) over time always taking into account the values ​​previously achieved.

Dataset and data model

Let’s imagine to have a data model like the following and we focus on the SalesAmount (Fact) and Date (Measure) fields

Let’s imagine we have amounts (e.g. sales) distributed over time like those shown below

DAX formula

Let’s imagine we want to obtain a running total over time rather than a simple subdivision over time.

To get this result, we create the following measure in DAX

RunningTotalSalesAmount =

CALCULATE (

SUM ( ‘Sales’[SalesAmount] ),

FILTER (

ALL(‘Sales’),

‘Sales’[OrderDate] <= MAX ( ‘Sales’[OrderDate] )

)

)

to this measure we use the following logic:

• we use the CALCULATE function with the SUM of SalesAmount as expression
• we use the FILTER function in the CALCULATE filter expression
• in the FILTER functione we use ALL to return all the rows in the table, ignoring any filters that might have been applied.
• we apply a condition to the OrderDate

Result

As a result we get the data and their following representation:

the comparison with the first representation is the following:

Buy me a coffee

If you find my work interesting, consider buying me a coffee here:

https://www.buymeacoffee.com/Entango

For a data governance and business intelligence project that I’m following, I found myself having to populate a SQL SERVER database starting from heterogeneously structured EXCEL files

Required: Import data into SQL SERVER from EXCEL

What tools to use?!

The operational choice was to use procedures written in Python in order to have high flexibility and be able to introduce more or less articulated logics for the treatment of data present in EXCEL.

Tools used:
1. Excel
2. Visual studio Code with Python environment
3. SQL Server

Code structure:

1.Import Python libraries:

For my procedure I used Pandas (reading from Excel), Numpy (data processing), PyOdbc (connection to SQL SERVER), plus other service libraries

# import python libraries

import pandas as pd

import numpy as np

import pyodbc

import uuid

from datetime import datetime

2.Create connection to SQLSERVER

Using PyOdbc I create and open a connection to the SQL SERVER database

# setup the connection string

server = ‘NB001566’

database = ‘EntangoData’

username = ‘**’

password = ‘*******’

conn = pyodbc.connect(‘DRIVER={SQL Server};SERVER=’+server+’;DATABASE=’+database+’;UID=’+username+’;PWD=’+ password)

cursor = conn.cursor()

3.Read data from Excel

With the read_excel function of the Pandas library I read the data from Excel by verifying its content

# read the Excel sheet

sheet = pd.read_excel(‘data/Prezzari/EmiliaRomagna/DataToImport.xlsx’, sheet_name=’Completo 2022')

print (sheet)

print (sheet.info())

Note: in this particular case the data to import are a set of information about construction materials

4.Processing of data read from Excel

Iteration on the data with the insertion of specific logics with respect to the objectives of my analysis.

# iteration on Excel sheet data

for index, row in sheet.iterrows():

Id = str(uuid.uuid4())

Name = sheet.iloc[index][0]

Description = sheet.iloc[index][1]

Rate = sheet.iloc[index][3]

ImagePath = ‘ ‘

BrandId = ‘a8b51d8d-78e9–4824–4f22–08db2eebed3c’

TenantId = ‘root’

CreatedBy = ‘ ‘

CreatedOn = ‘ ‘

LastModifiedBy = ‘ ‘

LastModifiedOn = ‘ ‘

DeletedOn = ‘ ‘

DeletedBy = ‘ ‘

CatalogDescription = ‘Emilia Romagna 2022’

CatalogId = ‘ER.2022’

Category = ‘Edilizia’

EANCode = ‘ ‘

EquipmentPerc = 0.0

ExtendedDescription = ‘ ‘

Geography = ‘ ‘

LabourPerc = 0.0

MaterialPerc = 0.0

SafetyPerc = 0.0

Synonims = ‘ ‘

UM = sheet.iloc[index][2]

print(“articolo: “, Id, “ — “, Name, “ — “, Rate, “ — “, UM)

Note: Python allows me to introduce even complex logics for data processing and for their eventual enrichment with information from other sources

5.Insert in SQL SERVER

In the final step I proceed to the insertion in SQL SERVER going to fix the numerical conversions and the null data check. Finally I trace the successful import in a log.

try:

DescriptionStr = Description

if pd.isnull(Description):

DescriptionStr = ‘’

RateDec = Rate

if np.isnan(Rate):

RateDec = 0.0

UMStr = UM

if pd.isnull(UM):

UMStr = ‘’

cursor.execute(“INSERT INTO [Catalog].[Products] (Id, Name, Description, Rate, BrandId, TenantId, CatalogDescription, CatalogId, Category, UM) VALUES(?,?,?,?,?,?,?,?,?,?)”, Id, Name, DescriptionStr, RateDec, BrandId, TenantId, CatalogDescription, CatalogId, Category, UMStr)

except Exception as e:

print(e.args[1])

# INSERT Logs in a separate table

ID_LOG = “P”

DESCRIPTION_LOG = “import prezzario Emilia Romagna 2022”

DATA_LOG = datetime.now()

cursor.execute(“INSERT INTO Logs (ID, Description, ExecutionDate) values(?,?,?)”, ID_LOG, DESCRIPTION_LOG, DATA_LOG)

print(“Register operation”)

conn.commit()

print(“Data Successfully Inserted”)

cursor.close()

print(“Cursor closed”)

CONCLUSION:

The use of Python becomes effective and decisive when there is the need to insert logic, even articulated, on the data being treated or if one wants to have flexibility regarding the result to be obtained.

Data ingestion with Python was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.

There are many ways to filter data in a PowerBI report (in report/page/visual filters, in slicers, …), so if we want to better understand our data one of the most important thing we can do is to show the filters applied to data.

“Selected filters” measure definition:

To show the filters applied to data, first of all we have to create the following measure:

Selected filters =

VAR separator = UNICHAR(10) /* linefeed and carriage return character */

VAR Cust = IF(ISFILTERED(‘Customer’[LastName]) ,

“Customer = “ & CONCATENATEX(ALLSELECTED(Customer[LastName]) , Customer[LastName] , “ ; “) & separator , “Customer = ALL” & separator )

VAR Year = IF(ISFILTERED(‘Calendar’[CalendarYear]) ,

“Year = “ & CONCATENATEX(ALLSELECTED(‘Calendar’[CalendarYear]) , ‘Calendar’[CalendarYear] , “ ; “) & separator , “Year = ALL” & separator )

VAR Prod = IF(ISFILTERED(‘Product’[EnglishProductName]) ,

“Product = “ & CONCATENATEX(ALLSELECTED(‘Product’[EnglishProductName]) , ‘Product’[EnglishProductName] , “ ; “) , “Product = ALL”)

RETURN Cust & Year & Prod

This measure is built on 3 specific dimensions of the model that we use in the report.

This measure use variables (Cust, Year, Prod) to temporary store the value of the single dimension filter retrieved.

This measure use 2 important DAX function to extract filters applied to data:

ISFILTERED( … ): Returns TRUE when the specified table or column is being filtered directly

ALLSELECTED( … ): The ALLSELECTED function gets the context that represents all rows and columns in the query, while keeping explicit filters and contexts other than row and column filters. This function can be used to obtain visual totals in queries.

“Selected filters” measure display:

When i use the measure in specific visual (card), the result is the following:

It is also possible to insert the measurement created in a tooltip to make the visualization more effective

Buy me a coffee

If you find my work interesting, consider buying me a coffee here:

https://www.buymeacoffee.com/Entango