5 killer Python scripts for automation — Part 2

Credit: DALL-E

Welcome back, fellow Python enthusiasts! In our previous article, we delved into the world of Python scripts and uncovered the first five types lurking in the depths of code. But fear not, for we have not yet uncovered all the mysteries of Python scripting. In this installment, we will bravely venture forth and discover the remaining five types of scripts that will have you coding like a pro in no time. So grab your favorite beverage and let’s get ready to script like a boss!

As we continue our journey into the realm of Python scripting, we will now uncover the remaining five types of scripts that will take your coding skills to the next level. So, join me as we dive deeper into the world of Python scripts and discover the secrets of the last five types.

1). Automating Data Analysis:

Python’s pandas library is a powerful tool for data analysis and manipulation. The following script demonstrates how to use it to automate the process of cleaning, transforming and analyzing a dataset.

import pandas as pd

# Reading a CSV file
df = pd.read_csv("data.csv")

# Cleaning data
df.dropna(inplace=True) # Dropping missing values
df = df[df["column_name"] != "some_value"] # Removing specific rows

# Transforming data
df["column_name"] = df["column_name"].str.lower() # Changing string to lowercase
df["column_name"] = df["column_name"].astype(int) # Changing column datatype

# Analyzing data
print(df["column_name"].value_counts()) # Prints the frequency of unique values in the column

# Saving the cleaned and transformed data to a new CSV file
df.to_csv("cleaned_data.csv", index=False)

The comments in above script are pretty much straight forward for a person with a basic knowledge of python. The script is a simple example to demonstrate the power of pandas library and how it can be used to automate data cleaning, transformation, and analysis tasks. However, the script is limited and in real-world scenarios, the dataset may be much larger and the cleaning, transformation, and analysis operations will likely be more complex.

2). Automating computer vision tasks:

Automating computer vision tasks refers to using Python and its libraries to perform various image processing and computer vision operations automatically. One of the most popular libraries for computer vision tasks in Python is opencv.

OpenCV is a library of programming functions mainly aimed at real-time computer vision. It provides a wide range of functionality, including image and video I/O, image processing, video analysis, object detection and recognition, and much more. For example:

import cv2

# Load the cascade classifier for face detection
face_cascade = cv2.CascadeClassifier("haarcascade_frontalface_default.xml")

# Load the image
img = cv2.imread("image.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Detect faces
faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5)

# Draw rectangles around the faces
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)

# Show the image

cv2.imshow("Faces", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Above script detects the faces in an image. It starts by loading a cascade classifier for face detection, this classifier is a pre-trained model that can recognize faces in an image.

Then it loads the image and converts it to grayscale using the cv2.cvtColor() method. The image is then passed to the detectMultiScale() method of the classifier which detects faces in the image. The method returns a list of coordinates of the detected faces.

The script then loops through the list of coordinates and draws rectangles around the detected faces using the cv2.rectangle() method. Finally, the image is displayed on the screen using the cv2.imshow() method.

This is just a basic example of what can be achieved with OpenCV and there are many more functionalities that can be automated like object detection, image processing, and video analysis. OpenCV is a very powerful library that can be used to automate a wide range of computer vision tasks, such as facial recognition, object tracking, and image stabilization.

3). Automating data encryption:

Automating data encryption refers to using Python and its libraries to encrypt and decrypt data and files automatically. One of the most popular libraries for data encryption in Python is cryptography. ‘cryptography’ is a library that provides cryptographic recipes and primitives. It includes both high-level recipes and low-level interfaces to common cryptographic algorithms such as symmetric ciphers, message digests, and key derivation functions. Following example demonstrates how can we encrypt a file using cryptography library:

import os
from cryptography.fernet import Fernet
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC

password = b"super_secret_password"
salt = os.urandom(16)
kdf = PBKDF2HMAC(
    algorithm=hashes.SHA256,
    iterations=100000,
    length=32,
    salt=salt,
    backend=default_backend()
)
key = base64.urlsafe_b64encode(kdf.derive(password))
cipher = Fernet(key)

# Encrypt the file
with open("file.txt", "rb") as f:
    data = f.read()

cipher_text = cipher.encrypt(data)

with open("file.txt", "wb") as f:
    f.write(cipher_text)

It starts by generating a key using the PBKDF2HMAC key derivation function, which is a password-based key derivation function that uses a secure hash algorithm (SHA-256) and a salt value. The salt value is generated using the os.urandom() function, which generates cryptographically secure random bytes.

Then it creates a Fernet object which is an implementation of symmetric (also known as “secret key”) authenticated cryptography.
It then reads the plaintext file and encrypts it using the encrypt() method of the Fernet object. Finally, it writes the encrypted data to the file.

It’s important to note that the key used to encrypt the file must be kept secret and securely stored. If the key is lost or compromised, the encrypted data will be unreadable.

4). Automating testing and debugging:

Automating testing and debugging refers to using Python and its libraries to run tests and debug code automatically. There are several popular libraries for automating testing and debugging in Python, such as unittest, pytest, nose, and doctest.

Here’s an example of using the unittest library to automate testing for a Python function that finds the longest palindromic substring in a given string:

def longest_palindrome(s):
    n = len(s)
    ans = ""
    for i in range(n):
        for j in range(i+1, n+1):
            substring = s[i:j]
            if substring == substring[::-1] and len(substring) > len(ans):
                ans = substring
    return ans

class TestLongestPalindrome(unittest.TestCase):
    def test_longest_palindrome(self):
        self.assertEqual(longest_palindrome("babad"), "bab")
        self.assertEqual(longest_palindrome("cbbd"), "bb")
        self.assertEqual(longest_palindrome("a"), "a")
        self.assertEqual(longest_palindrome(""), "")

if __name__ == '__main__':
    unittest.main()

This script uses the unittest library to automate testing for a Python function that finds the longest palindromic substring in a given string. The ‘longest_palindrome’ function takes a string as input and returns the longest palindromic substring by iterating through all the possible substrings and checking if it is a palindrome and it’s length is greater than the previous one.

The script also defined a ‘TestLongestPalindrome’ class that inherits from unittest.TestCase and contains several test methods. Each test method uses the assertEqual() method to check if the output of the longest_palindrome() function is equal to the expected output.

When the script is run, the unittest.main() function is called, which runs all the test methods in the TestLongestPalindrome class. If any of the tests fail (i.e., the output of the longest_palindrome() function is not equal to the expected output), an error message is printed indicating which test failed and what the expected and actual outputs were.

This script is an example of how the unittest library can be used to automate testing for a Python function. It allows you to easily test your code and catch any errors or bugs before deploying it to production.

5). Automating time-series forecasting:

Automating time-series forecasting refers to using Python and its libraries to predict future values of a time-series data automatically. There are several popular libraries for automating time-series forecasting in Python, such as statsmodels and prophet.

‘prophet’ is an open-source library developed by Facebook that provides a simple and fast way to perform time-series forecasting. It is based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data.

Here's an example of using the prophet library to perform time-series forecasting for daily sales data:

import pandas as pd
from fbprophet import Prophet

# Read in data
df = pd.read_csv("sales_data.csv")

# Create prophet model
model = Prophet()

# Fit model to data
model.fit(df)

# Create future dataframe
future_data = model.make_future_dataframe(periods=365)

# Make predictions
forecast = model.predict(future_data)

# Print forecast dataframe
print(forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']])

As said by X:

A picture is worth a thousand words

Photo by Isaac Smith on Unsplash

You can also include visuals for the predicted sales by adding the following lines of code in above:

# Import visualization library
import matplotlib.pyplot as plt

# Plot predicted values
model.plot(forecast)
plt.show()

# Plot predicted values with uncertainty intervals
model.plot(forecast)
plt.fill_between(forecast['ds'], forecast['yhat_lower'], forecast['yhat_upper'], color='pink')
plt.show()

# Plot component of the forecast
model.plot_components(forecast)
plt.show()

The first visualization, model.plot(forecast), shows the predicted values and the historical data, it gives you a general idea of how well the model is fitting the data.

The second visualization, plt.fill_between(forecast['ds'], forecast['yhat_lower'], forecast['yhat_upper'], color='pink'), shows the predicted values with uncertainty intervals, this allows you to see how much uncertainty there is in the predictions.

The third visualization, model.plot_components(forecast), shows the components of the forecast, such as the trend, seasonality, and holidays.

Conclusion

Photo by Rui Silva sj on Unsplash

In conclusion, Python’s automation capabilities are vast and constantly evolving, with new libraries and frameworks being developed every day. It can help you in a lot of different ways to improve your daily routines and to achieve your goals. The use of libraries such as pandas, opencv, cryptography, unittest, prophet and more, makes it even more easy to perform these tasks in an efficient and automated way.

Looking forward to hearing a positive feedback about this part of python scripts. I hope these help you in your scripting realm.

Happy Coding! 🧑‍💻🤓