Using Optical Character Recognition to verify failed server screenshots.
During the past few months, I have wanted to extend my knowledge of individual programming languages so that I can use a wider range of systems, and while learning these I have come across different packages that can save tons of time for certain daily tasks. This article will go into more detail on how I have used Optical Character Recognition to analyse failed server screenshots to filter and report the reason to the end user.
To start, this project is running using Python 3.8.0 and using the below packages:
This project is far from complete but the basic functionality is a great base to make reporting more complex. I will show some demo results below and share the code snippets at the end.
First, let's analyze the failed screenshots that are reported by a backup manager:
Image 1. Shows an error for a missing or corrupt file and this prevents the screenshot from booting. This is a valid error and needs to be investigated.
Image 2. Shows the windows lock screen. This is not a valid error as the screenshot has booted successfully.
My goal was to iterate through a folder of images and classify them as ‘Errors’ or ‘False Positives’. Below is the output using the above images:
Once I have made the reporting more advanced I will make this public on my Github where you can fork this for yourself.
Project structure:
images/-error (1).png-false (3).pngmain.pyOCR.py
A Basic Python Implementation
main.py
from OCR import OCR, Classifier, Loggerimport osimage_folder = 'images'if __name__ == "__main__":# Set folder for conversionsocr = OCR(image_folder)# Create classifierclassifier = Classifier('errors')# Create Loggerlog = Logger()# Convert all images inside of the folder#ocr.convert_all()# Loop through directory and classify imagesfor file_name in os.listdir(image_folder):image = ocr.convert(file_name)# get image groupgroup = classifier.get_group(image)# Errorif group=='error':error = classifier.get_error(image)log.error(file_name, 'Error: ' + error)# False positiveif group=='false_positive':log.success(file_name, 'False Positive')
OCR.py:
[OCR.py]
import cv2import pytesseractimport ospytesseract.pytesseract.tesseract_cmd = 'C:/Program Files/Tesseract-OCR/tesseract.exe'
class OCR:def __init__(self, folder):self.folder = folderdef convert(self, file_name):img = cv2.imread(self.folder + '/' + file_name)img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)output=pytesseract.image_to_string(img)return outputdef convert_all(self):# Loop through files in specified folderfor file_name in os.listdir(self.folder):# Convert file and save as outputoutput = self.convert(file_name)# Create new file and save outputoutput_file = open(self.folder + '/' + file_name + '.txt', 'w')n = output_file.write(output)output_file.close()class Classifier:def __init__(self, type):self.type = typedef get_group(self, image_text):if "Press Ctrl" in image_text:return 'false_positive'else:return 'error'def get_error(self, image_text):for line in image_text.splitlines():if "File:" in line:return 'Missing or Corrupt EXE/DLL'if "Safe Mode with Networking" in line:return 'Advanced Boot Options'if "Technical Information:" in line:return 'BSOD'if "Stop code:" in line:return linereturn 'Failed to get error'class Logger:def __init__(self):passdef success(self, file, message):print('[' + file + '] ' + message)def error(self, file, message):print('[' + file + '] ' + message)
