Handling Exceptions in Python OOP

Python for AI, data science and machine learning Day 9

Python for AI, data science and machine learning series

Basics of Handling Exceptions in Python OOP

Exception handling is a critical aspect of writing robust Python applications, especially in object-oriented programming (OOP). It allows your program to respond to various errors and exceptional conditions gracefully, without crashing or producing incorrect results. In this section, we will cover the basics of handling exceptions in Python OOP, focusing on the try-except-finally syntax.

Understanding Exceptions in Python

An exception is an error that occurs during the execution of a program. Python, being a dynamic language, is prone to a wide range of exceptions — from syntax errors that prevent your program from running to runtime errors that occur while your program is executing.

In OOP, exceptions can arise from various sources, such as method calls on objects, constructor failures, or resource management issues. Properly handling these exceptions is crucial to building resilient and user-friendly applications.

The try-except-finally Block

The primary mechanism for exception handling in Python is the try-except-finally block. This structure allows you to catch exceptions, handle them gracefully, and execute cleanup code regardless of whether an exception occurred.

The try Block

The try block allows you to test a block of code for errors. This is where you place the code that you think might raise an exception during execution.

try:
    # Code that might raise an exception
    result = 10 / 0
except Exception as e:
    # Code to handle the exception
    print(f"An error occurred: {e}")
finally:
    # Code that runs after the try and except blocks
    # This is optional and used for cleanup actions
    print("Execution completed.")

The except Block

The except block lets you handle the exception. You can specify the type of exception you want to catch, allowing for more granular control over error handling. If an exception occurs in the try block, the code inside the except block is executed.

try:
    # Code that might raise an exception
    result = 10 / 0
except Exception as e:
    # Handling exception
    print(f"Division by zero is not allowed: {e}")

The finally Block

The finally block is optional and executes regardless of whether an exception was caught or not. It's ideal for performing cleanup actions, such as closing files or releasing resources.

try:
    # Code that might raise an exception
    file = open("example.txt", "r")
    data = file.read()
except Exception as e:
    # Handling exception related to I/O operations
    print(f"Failed to read file: {e}")
finally:
    # Cleanup action
    file.close()
    print("File closed.")

Types of Exceptions

Python categorizes exceptions into several types, each corresponding to different error conditions:

1. Syntax Errors: Errors detected by the interpreter when translating the source code into bytecode. These are not derived from the Exception class because they are not exceptions in the traditional sense. They are detected before the program actually runs.
2. Built-in Exceptions: Python provides numerous built-in exceptions that handle common error conditions, such as:

• ValueError: Raised when a function receives an argument of the correct type but with an inappropriate value.
• TypeError: Occurs when an operation or function is applied to an object of inappropriate type.
• IndexError: Raised when a sequence subscript is out of range.
• KeyError: Occurs when a dictionary key is not found.
• IOError: Raised when an I/O operation fails for an I/O-related reason, e.g., “file not found” or “disk full” (in Python 3.x, it is known as OSError).

User-defined Exceptions: Users can define custom exception classes to handle specific error conditions in their programs. These exceptions should be derived from the Exception class or one of its subclasses.

Exceptions Are Classes in python

In Python, exceptions are implemented as classes. This object-oriented approach to exceptions allows Python to integrate error handling seamlessly with its other object-oriented features. Understanding that exceptions are classes is crucial for effective exception handling, especially when defining custom exceptions or when you need to manage a wide range of error conditions in more complex programs.

Understanding Exception Classes

At the heart of Python’s exception handling mechanism is the Exception class. It is the base class for all other exception classes, which means that all exceptions inherit from this class either directly or indirectly. This hierarchy allows exceptions to be grouped and handled according to their characteristics.

Hierarchy of Exception Classes

The Exception class is part of a larger hierarchy of exception classes designed to categorize different types of errors:

• BaseException: The root of the exception hierarchy. All exception classes derive from this class. It includes exceptions like SystemExit, KeyboardInterrupt, and GeneratorExit, which are not meant to be caught by most programs.
• Exception: All built-in, non-system-exiting exceptions are derived from this class. All user-defined exceptions should also be derived from this class.

Let’s delve into the difference between BaseException and Exception in Python:

BaseException:

• BaseException is the base class for all exceptions in Python.
• These exceptions are associated with special situations, and catching them is almost always the wrong thing to do.
• Incorrectly handling them can prevent a graceful shutdown of your program, thread, or generator/coroutine.
• Examples of exceptions inheriting directly fromBaseException:
  -KeyboardInterrupt: Raised when the user interrupts the program (e.g., by pressing Ctrl+C)
  - SystemExit: Raised when the program exits normally
  - GeneratorExit: Raised when a generator or coroutine is closed.

Exception:

• Exception is the base class for most user-defined and built-in exceptions unrelated to the system or the interpreter.
• All user-defined exceptions should inherit from Exception.
• It is recommended by PEP 8 to derive exceptions from Exception rather than BaseException.
• Exceptions deriving from Exception are intended to be handled by regular code.
• Examples of exceptions inheriting from Exception:
  - ValueError: Raised when an operation receives an inappropriate value.
  - TypeError: Raised when an operation is performed on an object of an inappropriate type.
  - Custom exceptions the developer create in his/her code.

In summary, use Exception for most of your custom exceptions, and reserve direct inheritance from BaseException for exceptional cases where catching them is almost always the wrong approach.

Other types of base exception in ptyhon are:

exception ArithmeticError

ArithmeticError is a base class for exceptions that are raised for different arithmetic errors in Python. It serves as a parent class for several other exceptions that deal specifically with arithmetic operations, including:

• OverflowError: This occurs when an arithmetic operation produces a result that is too large to be represented.
• ZeroDivisionError: This happens when a division or modulo operation is attempted with zero as the denominator.
• FloatingPointError: This error is raised when a floating point operation fails, though it is not commonly encountered as Python tends to handle floating point errors more gracefully by default.

Being a base class, ArithmeticError itself can be used in exception handling to catch any of the above errors without specifying them individually.

exception BufferError

BufferError is raised when an operation related to a buffer (an object that directly points to a block of memory containing raw data) cannot be performed. This error might occur in scenarios involving low-level manipulation of buffer interfaces, where Python cannot perform an operation due to memory or buffer constraints. Common instances where BufferError might be raised include problems with memory allocation during buffer operations or attempting to modify a read-only buffer.

exception LookupError

LookupError acts as a base class for exceptions that occur when a specified key or index is not valid for a given data structure (such as lists, tuples, dictionaries, etc.). This class encompasses:

• IndexError: This is raised when trying to access an index that is outside the bounds of a list or tuple.
• KeyError: Occurs when a lookup in a dictionary (or similar mapping type) fails because the key is not found in the dictionary’s set of keys.

LookupError itself can be used in exception handling when you want to handle both IndexError and KeyError without distinguishing between them. This makes LookupError useful for writing cleaner and more comprehensive error handling code that deals with accessing elements from various container types.

These base exceptions help in writing more flexible and generic error handling routines, allowing a programmer to catch a range of errors with a single except clause.

Built-in specific exceptions in python

In Python, exceptions are derived from the base exceptions so that these exceptions are raised directly by the runtime to indicate specific error conditions. These are instances of exceptions that directly inherit from the base classes, like Exception, but are designed to handle very specific scenarios. Unlike their abstract base classes (e.g., ArithmeticError, LookupError), which group related exceptions, concrete exceptions provide detailed context about what exactly went wrong in your program.

Here’s a summary and explanation of several key exceptions in Python:

ZeroDivisionError

• Explanation: This is raised when a division or modulo operation has a denominator of zero. It’s a direct instance of the ArithmeticError.
• Example: result = 1 / 0 triggers ZeroDivisionError.

FileNotFoundError

• Explanation: Raised when a file or directory is requested but doesn’t exist. It’s particularly useful in file handling operations.
• Example: open('nonexistentfile.txt') results in FileNotFoundError.

ValueError

• Explanation: This occurs when an operation or function receives an argument of the right type but with an inappropriate value.
• Example: int('xyz') attempts to convert a non-numeric string to an integer, raising a ValueError.

TypeError

• Explanation: Raised when an operation or function is applied to an object of inappropriate type. The associated value is a string giving details about the type mismatch.
• Example: 3 + '3' would raise a TypeError because it tries to add an integer and a string.

IndexError

• Explanation: Triggered when trying to access an index that is out of the range of a sequence (like a list or a tuple).
• Example: Accessing the fifth element of a three-item list, as in lst[4], will raise an IndexError.

KeyError

• Explanation: Raised when a dictionary key is not found. It’s a specific case under LookupError.
• Example: dict = {'a': 1}; dict['b'] would raise a KeyError because 'b' is not a key in the dictionary.

AttributeError

• Explanation: Occurs when an attribute reference or assignment fails.
• Example: x = 10; x.append(5) raises an AttributeError because 'int' objects do not have an 'append' method.

NotImplementedError

• Explanation: This exception is raised by abstract methods or functions that require subclasses to provide an implementation. If they don’t, and the method is called, NotImplementedError is raised.
• Example: Calling a method in a class that is meant to be implemented by any subclass, but hasn’t been, will raise this error.

MemoryError

• Explanation: Raised when an operation runs out of memory but the situation may still be salvageable (unlike other fatal errors that cause the interpreter to exit).
• Example: This might occur in large data processing tasks that exceed the available memory.

OverflowError

• Explanation: Triggered when an arithmetic operation is too large to be represented.
• Example: Extremely rare in Python with integers since Python integers are of arbitrary precision, but can occur with floating point operations like exponential calculations.

These exceptions are crucial for robust error handling in Python as they allow developers to manage errors predictably and gracefully. Understanding these helps in debugging and improves the reliability of software by catching and handling errors effectively.

This is the complete built-in exception hierarchy in python:

BaseException
 ├── BaseExceptionGroup
 ├── GeneratorExit
 ├── KeyboardInterrupt
 ├── SystemExit
 └── Exception
      ├── ArithmeticError
      │    ├── FloatingPointError
      │    ├── OverflowError
      │    └── ZeroDivisionError
      ├── AssertionError
      ├── AttributeError
      ├── BufferError
      ├── EOFError
      ├── ExceptionGroup [BaseExceptionGroup]
      ├── ImportError
      │    └── ModuleNotFoundError
      ├── LookupError
      │    ├── IndexError
      │    └── KeyError
      ├── MemoryError
      ├── NameError
      │    └── UnboundLocalError
      ├── OSError
      │    ├── BlockingIOError
      │    ├── ChildProcessError
      │    ├── ConnectionError
      │    │    ├── BrokenPipeError
      │    │    ├── ConnectionAbortedError
      │    │    ├── ConnectionRefusedError
      │    │    └── ConnectionResetError
      │    ├── FileExistsError
      │    ├── FileNotFoundError
      │    ├── InterruptedError
      │    ├── IsADirectoryError
      │    ├── NotADirectoryError
      │    ├── PermissionError
      │    ├── ProcessLookupError
      │    └── TimeoutError
      ├── ReferenceError
      ├── RuntimeError
      │    ├── NotImplementedError
      │    └── RecursionError
      ├── StopAsyncIteration
      ├── StopIteration
      ├── SyntaxError
      │    └── IndentationError
      │         └── TabError
      ├── SystemError
      ├── TypeError
      ├── ValueError
      │    └── UnicodeError
      │         ├── UnicodeDecodeError
      │         ├── UnicodeEncodeError
      │         └── UnicodeTranslateError
      └── Warning
           ├── BytesWarning
           ├── DeprecationWarning
           ├── EncodingWarning
           ├── FutureWarning
           ├── ImportWarning
           ├── PendingDeprecationWarning
           ├── ResourceWarning
           ├── RuntimeWarning
           ├── SyntaxWarning
           ├── UnicodeWarning
           └── UserWarning

Custom Exceptions in Python

Python’s flexibility allows you to define custom exceptions tailored to the specific needs of your application. These custom exceptions can provide clearer, more meaningful error handling, making your code more readable and maintainable. By inheriting from the built-in Exception class or one of its subclasses, you can create your own exception hierarchies, encapsulating various error conditions specific to your application's domain.

Why Define Custom Exceptions?

Custom exceptions are useful for several reasons:

• Specificity: They can represent specific error conditions more precisely than Python’s built-in exceptions.
• Clarity: Custom exceptions can make your code more readable and self-documenting. When someone else (or future you) reads your code, the exceptions will clearly indicate what kind of error conditions you expected.
• Control: They give you more control over your program’s flow by allowing you to catch and handle very specific exceptions differently.

Defining Custom Exceptions

To define a custom exception, you start by creating a new class that inherits from the Exception class or one of its subclasses. The new class can be as simple or as complex as needed, and you can add methods and properties to provide additional functionality or information about the error.

There are two fundamental aspects of designing effective, Pythonic exception handling in your applications:

1. Inheritance from the Exception Class

At the core of Python’s exception handling mechanism is a hierarchy of exception classes, all of which inherit from the base Exception class. This design allows for a structured and extensible way to handle errors and exceptional conditions in Python programs.

• Inheritance Requirement: For your custom exceptions to integrate seamlessly with Python’s built-in error handling mechanisms, they must be part of this exception class hierarchy. This means any custom exception you define should inherit from the Exception class or one of its more specific subclasses. This inheritance is crucial because it enables your custom exceptions to be caught by except blocks that are looking to catch exceptions of the type Exception or more specific types.

class MyCustomError(Exception):
    """Base class for other custom exceptions"""
    pass

This simple custom exception MyCustomError now behaves like any other exception in Python, capable of being raised, caught, and handled. From here you can start to tailor and customize your error handling.

2. Leveraging Python’s Exception Handling Capabilities

By adhering to Python’s exception hierarchy when defining custom exceptions, you unlock the full suite of Python’s exception handling features for your custom errors. This includes being able to:

• Catch specific exceptions precisely, making your error handling code more granular and informative.
• Propagate exceptions up the call stack until they are caught by an appropriate handler, allowing for centralized error handling in higher-level functions or methods.
• Attach additional information to exceptions through custom attributes or methods, enhancing the error reporting and handling capabilities.
• Implement complex error handling logic that differentiates between various types of errors, each with its own response strategy.

Using Custom Exceptions

You can also add initialization parameters to your custom exceptions to pass additional information about the error:

class ValidationError(Exception):
    """Exception raised for errors in the input validation.
    
    Attributes:
        message -- explanation of the error
        value -- input value which caused the error
    """
    def __init__(self, message, value):
        self.message = message
        self.value = value
        super().__init__(message)

Once defined, you can raise your custom exceptions in your code using the raise statement, just like with built-in exceptions:

def validate_age(age):
    if age < 0:
        raise ValidationError("Age cannot be negative", age)
    elif age < 18:
        raise ValidationError("You must be at least 18 years old", age)
    else:
        print("Age is valid")

# Example usage
try:
    validate_age(-1)
except ValidationError as e:
    print(f"Error: {e.message} - Invalid Value: {e.value}")

The raise statement in Python is used to trigger an exception during the execution of a program. When a raise statement is encountered, Python stops executing the normal flow of the program and shifts to the nearest enclosing try block to handle the raised exception. If no such try block exists, the program terminates, and Python prints a traceback to the console, providing details about the unhandled exception.

In the context of the validate_age function raise is used to throw a ValidationError when certain conditions about the age parameter are not met.

Purpose of raise in Exception Handling

• Custom Error Reporting: By using raise with custom exceptions like ValidationError, you can provide clear, specific feedback about what went wrong in your program. This is more informative than allowing Python to raise built-in exceptions like ValueError or TypeError, which might not fully convey the context of the error.
• Control Flow: It allows you to control the flow of your program in the face of incorrect or unexpected inputs. Instead of proceeding with invalid data, you can immediately stop the execution and signal that an error has occurred.
• Encapsulation: Encapsulating error conditions in exceptions makes your code cleaner and separates normal logic from error handling logic. It also allows errors to be passed along until they reach a part of the program prepared to handle them.

In summary, the raise statement is a powerful tool in Python for managing program flow, signaling errors, and ensuring that your program behaves predictably even in exceptional circumstances.

Benefits of custom exceptions

• Clarity and Maintainability: Using a well-defined exception hierarchy makes your code clearer and easier to maintain. Other developers (or you in the future) can quickly understand the kinds of errors your code can raise and how they are related.
• Robustness: It enables more robust error handling by allowing you to catch and handle specific errors differently, providing more precise reactions to different failure conditions.
• Best Practices: This approach aligns with Python’s philosophy and best practices for exception handling, ensuring that your code is idiomatic and Pythonic.

Custom exceptions are a powerful feature in Python that can greatly enhance the readability, maintainability, and robustness of your application’s error handling. By defining your own exceptions, you can create a more intuitive and fine-grained error handling mechanism that is tailored to the specific requirements of your application. Remember to inherit from the Exception class or one of its subclasses and leverage the full potential of Python's exception handling capabilities.

Scenario: Data Validation for a Machine Learning Model

Imagine you’re preparing data for a machine learning model that predicts house prices. The dataset includes features like the number of bedrooms, square footage, and the year the house was built. Before training the model, you need to validate that the data meets certain criteria, such as:

• The number of bedrooms must be a positive integer.
• The square footage must be within a reasonable range.
• The year the house was built must not be in the future.

To handle violations of these criteria, you’ll define custom exceptions.

Step 1: Define Custom Exceptions

class DataValidationError(Exception):
    """Base class for exceptions in data validation."""
    pass

class BedroomCountError(DataValidationError):
    """Exception raised for errors in the number of bedrooms."""
    pass

class SquareFootageError(DataValidationError):
    """Exception raised for errors in the square footage."""
    pass

class YearBuiltError(DataValidationError):
    """Exception raised for errors in the year the house was built."""
    pass

Step 2: Implement Data Validation Function

Next, let’s implement a function that validates a single data entry against these criteria, raising the appropriate custom exception if any criterion is violated.

def validate_house_data(house):
    if not isinstance(house['bedrooms'], int) or house['bedrooms'] <= 0:
        raise BedroomCountError(f"Invalid number of bedrooms: {house['bedrooms']}")
    
    if house['square_footage'] < 100 or house['square_footage'] > 10000:
        raise SquareFootageError(f"Square footage out of range: {house['square_footage']}")
    
    from datetime import datetime
    if house['year_built'] > datetime.now().year:
        raise YearBuiltError(f"Year built is in the future: {house['year_built']}")
    
    print("House data is valid.")

Step 3: Validate Data

Finally, we can use this function to validate our data, catching and handling any custom exceptions that are raised.

house_data = {
    'bedrooms': 3,
    'square_footage': 2500,
    'year_built': 2025  # This will cause a validation error
}

try:
    validate_house_data(house_data)
except DataValidationError as e:
    print(f"Data validation error: {e}")

In this example, custom exceptions (BedroomCountError, SquareFootageError, YearBuiltError) allow for clear and specific error handling during the data validation process. By defining these custom exceptions, we can catch and handle data validation errors in a way that is tailored to the needs of our data science project. This approach enhances the readability and maintainability of our code, making it easier to manage the complex error scenarios that can arise in data science workflows.

Handling Exception Hierarchies in Python

Python’s exception system is built around a hierarchy of exception classes, enabling developers to handle errors at different levels of specificity. This hierarchy also introduces complexities, such as the relationship between parent and child exceptions. Understanding how to navigate these relationships is crucial for writing clear and effective exception handling code.

Understanding Exception Hierarchies

As we saw earlier, in Python, all exceptions inherit from the base class BaseException. The Exception class, which is a direct subclass of BaseException, serves as the base class for almost all other exceptions. This hierarchical structure allows exceptions to be grouped into categories based on their functionality or origin. For example, IOError and ValueError are both subclasses of the Exception class but cater to different error scenarios.

The Significance of Exception Order in try-except Blocks

When handling exceptions, Python allows you to specify multiple except blocks within a single try-except statement, each catching a different type of exception. The order of these except blocks is significant due to the nature of the exception hierarchy. A block meant to catch a parent exception will also catch all of its child exceptions, due to the principle of polymorphism.

Catching Child Exceptions First

To effectively differentiate between exceptions in a hierarchy,

it’s essential to place except blocks for child exceptions before those for parent exceptions.

This arrangement ensures that exceptions are caught and handled by the most specific handler possible. Consider the following example:

try:
    # Code that might raise an exception
    pass
except ValueError:
    # Handle ValueError
    pass
except Exception:
    # Handle any exception
    pass

In this structure, a ValueError (a child exception of Exception) is caught by the first except block. If the order were reversed, the ValueError would be caught by the except Exception block, preventing the more specific ValueError handler from executing.

Why This Matters

The practice of catching more specific exceptions first is not just about adhering to best practices. It has practical implications for your application:

• Clarity: Handling exceptions at the most specific level possible makes your error handling logic clearer and more predictable. It’s easier for other developers to understand which block handles which error condition.
• Custom Responses: Different exceptions often require different handling strategies. By catching child exceptions first, you can tailor your response to the specific error, improving the robustness and user-friendliness of your application.
• Prevent Masking Errors: Broad exception handlers can inadvertently catch and mask errors you didn’t intend to handle at that level, potentially leading to bugs that are hard to diagnose.

Example Scenario: Data Loading and Preprocessing

In the realm of data science, handling exception hierarchies efficiently can be particularly crucial due to the diverse range of operations involved — from data loading and preprocessing to model training and evaluation. These operations can raise various exceptions, and handling them appropriately ensures the robustness of your data science pipelines.

Let’s consider a practical example involving data loading and preprocessing in a data science project. In this scenario, we’ll demonstrate how to handle different exceptions that might arise, such as file-related errors and data processing errors.

Imagine you are writing a Python script for loading a dataset from a file and then preprocessing this data before it’s used for training a machine learning model. During this process, several things can go wrong:

1. The file might not exist or could be inaccessible (raising an FileNotFoundError or PermissionError).
2. The data in the file might not be in the expected format, leading to a ValueError.
3. Generic errors could occur during processing, which we’ll catch as Exception.

Handling Exceptions with a Hierarchical Approach

To handle these exceptions, we’ll use a try-except block, catching the most specific exceptions first and more general exceptions later.

import pandas as pd

def load_and_preprocess_data(filepath):
    try:
        # Attempt to load the data file
        data = pd.read_csv(filepath)
        
        # Perform some preprocessing on the data
        # This might include operations that could raise a ValueError
        # For example, converting a column to numeric, where the operation might fail
        data['price'] = pd.to_numeric(data['price'], errors='raise')
        
        # Return the preprocessed data
        return data

    except FileNotFoundError:
        print(f"Error: The file {filepath} was not found.")
    except PermissionError:
        print(f"Error: Permission denied when trying to read {filepath}.")
    except ValueError as e:
        print(f"Error processing file {filepath}: {e}")
    except Exception as e:
        print(f"An unexpected error occurred: {e}")

• FileNotFoundError and PermissionError: These exceptions are specific to file operations. Catching them first ensures we handle file access issues before considering more general errors. This differentiation is important for debugging and for providing clear feedback to the user or logs.
• ValueError: This exception is raised if there’s an issue with the data format, such as when converting a column to a numeric type fails because the data contains non-numeric values. Handling this separately allows us to provide a specific message about data processing issues.
• Exception: This is a catch-all for any other exceptions not previously caught. It’s placed last to ensure that specific exceptions are not masked by this more general handler.

This example illustrates how to manage exception hierarchies in a data science context, ensuring that each step of the data loading and preprocessing phase is robust against potential errors. By catching more specific exceptions first, we can provide targeted responses to different error conditions, improving the reliability and user-friendliness of our data science pipelines.

Complete series:

10 Days of Python OOP for AI, Data Science, and Machine Learning

Official python documentation on Built-in Exceptions:

Built-in Exceptions