Integrating Pony ORM with Django

Photo by eberhard grossgasteiger: https://www.pexels.com/photo/selective-focus-photography-of-black-pony-on-grass-1612551/

In a recent project that used data analysis and AI, I integrated Django with Pony ORM. Despite using Pony ORM and Django separately, I tackled the challenge of integrating both into a single proje1ct for the first time. I had more challenges than I anticipated, and I wanted to give some guidance to future developers.

Background:

My project architecture consisted of three distinct components:

1. Django: Serving as the user interface and data repository.
2. Web Scraping and Data Insertion: A separate application responsible for easing server load by handling web scraping and data insertion tasks. Pony ORM was employed for data management.
3. Machine Learning: Another separate application focused on performing sentiment analysis, relevance assessment, and summarizing of news alerts.

Django Setup:

In Django’s models.py, database interactions are defined through classes inheriting from models.Model. Here’s an example:

class NewsAlert(models.Model):
    title = models.CharField(max_length=255, null=True)
    content = models.TextField(null=True)
    updated_at = models.DateTimeField(auto_now=True)

When you actually migrate your tables, you’ll notice that the table in the database is yourapp_newsalert in the schema public.

Pony Integration:

Pony ORM, my choice for non-web applications, offers flexibility in file organization. I have to create a database object and bind it using a password,host, and database.

db.bind(
    provider='postgres',
    user=db_user,
    password=db_password,
    host=db_host,
    database=db_name
)

While before I just had the table name, in pony I had to change the entity name to the name of the table. I did try to use __table__ in the app, however I wasn’t able to get that to work and I assume the fault is my own.

class MyAPP_INDUSTRY(db.Entity):
    __table__ = "myapp_industry"
    id = PrimaryKey(int, auto=True)
    title = Required(str, max_len=255)
    content = Optional(str)
    url = Optional(str)
    summary = Optional(str)
    published_date = Optional(datetime)
    metadata = Optional(Json)  # Assuming JSON data is stored as string
    sentiment_score = Optional(float)
    description = Optional(str)
    created_at = Required(datetime, default=datetime.now)
    updated_at = Required(datetime, default=datetime.now)
    news_site_id = Required(int)

At the end of the file you have to generate your files. Because I already created the tables and Django is managing the table creation, I have create_tables = False

db.generate_mapping(create_tables=False)

A few minor other implementations is that foreign keys in Pony do not work the same as Django. So for each foreign key, I had to set table_name_id as an integer. I tried using set and that did not work. If I used a manytomanyfield in Django I have create that table separately.

# Define your NewsAlertAuthor entity
class MYApp_NewsAlert_Author(db.Entity):
    id = PrimaryKey(int, auto=True)
    newsalert_id = Required(int)
    author_id = Required(int)

Unlike Django, Pony ORM expects database connection details to be set directly in the entities file, I suggest using environment variables.

Other Implementations

But in Django you can create a table name directly. This would help make the integration with pony easier in that if you were looking through either the Django Models or the Pony Entities, the entity names might be more readable.

class NewsAlert(models.Model):
    ...
class Meta:
  db_table = 'schema_name\".\"news_alert”

In my settings.py file I did have to add a search to option to db_settings

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ.get('DB_NAME', 'default_db_name'),
        'USER': os.environ.get('DB_USER', 'default_db_user'),
        'PASSWORD': os.environ.get('DB_PASSWORD', 'default_db_password'),
        'HOST': os.environ.get('DB_HOST', 'localhost'),
        'PORT': os.environ.get('DB_PORT', '5432'),
         'OPTIONS': {
            'options': '-c search_path=public'
        }
    }
}

Alternatively, you could instead have Pony manage the database and you could use django sync_db instead of migrating. I have not tried that approach.

Lastly, you could have pony manage your database creation instead of Django.

Integration Challenges TLDR;

1. Ensuring consistency in table and column names between Django and Pony ORM.
2. Navigating differences in primary key handling between the two frameworks.
3. Managing ManyToManyField relationships, which Pony ORM treats differently.

Conclusion:

I had some headaches in integrating Pony and Django in separate frameworks. I think these are both powerful tools that you can use for micro-serving your applications. Allowing for multiple tools in your toolkit allows you to approach problems from a different way.