An amazingly simple way to store and retrieve Numpy arrays in MySQL database via PNG conversion in Python

Sometime ago I was facing a challenge which involved processing of huge quantities of Numpy arrays and matrices. The data sets were grouped in clusters and nested sub categories. And because the data sets were required to be accessible via Internet I was looking for an easiest way to store them in MySQL database.

Photo by Kevin Ku on Unsplash

Let’s start with the road map

Import libraries for data handling

import mysql.connector
import os
import numpy as np
from imgarray import save_array_img, load_array_img
from os import fsync

Write your first function, which makes sure that data is written to disk, so that buffering doesn’t influence the timings.

def sync(fh):
   fh.flush()
   fsync(fh.fileno())
   return True

Write the next function which converts any given Numpy data set into PNG image. As arguments we will be using numpy array name and save path where the new image will be stored

def save_array_to_PNG(numpy_array, save_path):
    with open(save_path, 'wb+') as fh:
        save_array_img(numpy_array, save_path, img_format='png')
        sync(fh)

    return save_path