5 Upcoming Python Open Source Libraries Q4 2019 Edition
Python as a programming language has a strong presence on the Software Engineering world in 2019 with new libraries coming out everyday. In this story I will go through some new and upcoming open source libraries that can be used for your new or existing projects. The focus here is specifically on libraries that are not fully mature yet and to encourage readers to contribute to them. I highly encourage readers to support the authors in anyway they can whether it is donating, extending functionality or simply using them. Let’s get started.
1. mashumaro (マシュマロ)
Named after the Japanese version of Marshmallow, this library is perfect for serialising and deserialising JSON, YAML, msgpack and plain Dict formats into Python instances. Use cases can include consumers subscribing to queues, microservices talking to each other or just direct object orientated programming. Native field types are supported so expect timestamps to be available in datetime formats rather than strings. This library makes it quite easy to switch between other programming languages like Java or Go where usually serialising into objects is the standard way. Sample code below:
from enum import Enum
from typing import Set
from dataclasses import dataclass
from mashumaro import DataClassJSONMixin
class PetType(Enum):
CAT = 'CAT'
MOUSE = 'MOUSE'
@dataclass(unsafe_hash=True)
class Pet(DataClassJSONMixin):
name: str
age: int
pet_type: PetType
@dataclass
class Person(DataClassJSONMixin):
first_name: str
second_name: str
age: int
pets: Set[Pet]
tom = Pet(name='Tom', age=5, pet_type=PetType.CAT)
jerry = Pet(name='Jerry', age=3, pet_type=PetType.MOUSE)
john = Person(first_name='John', second_name='Smith', age=18, pets={tom, jerry})
dump = john.to_json()
person = Person.from_json(dump)
# person == john
Pet.from_json('{"name": "Tom", "age": 5, "pet_type": "CAT"}')
# Pet(name='Tom', age=5, pet_type=<PetType.CAT: 'CAT'>)Check it out: https://github.com/Fatal1ty/mashumaro
2. loguru
Logging used to be simple in Python but now thanks to loguru it is stupidly simple. Ever find yourself putting lots of print statements everywhere in code? Sometimes pdb can’t be added or logging can only be done to files? Then this is the easiest way to add logging to your application. Below is just a minor example of what it can do (from asynchronous logging to lazy evaluations).
from loguru import logger
logger.debug("Loguru for logging!")# Log to a file
logger.add("file_{time}.log")# Catch exceptions in functions
@logger.catch
def my_function(x, y, z):
# An error? It's caught anyway!
return 1 / (x + y + z)
Check it out: https://github.com/Delgan/loguru
3. dramatiq
Task scheduling can have many use cases, from background jobs to delayed tasks dramatiq is an upgrade over Celery. For large applications and consumer based workloads dramatiq supports Redis and RabbitMQ out of the box and exposes a powerful retry API with exponential back-off. I find myself using dramatiq very closely with mashumaro to work with objects end-to-end.
import dramatiq
import requests
@dramatiq.actor
def count_words(url):
response = requests.get(url)
count = len(response.text.split(" "))
print(f"There are {count} words at {url!r}.")>>> count_words.send("http://example.com")
Message(
queue_name='default',
actor_name='count_words',
args=('http://example.com',), kwargs={}, options={},
message_id='8cdcae57-af36-40ba-9616-849a336a4316',
message_timestamp=1498557015410)>>> urls = [
... "https://news.ycombinator.com",
... "https://xkcd.com",
... "https://rabbitmq.com",
... ]
>>> [count_words.send(url) for url in urls]# Retrying
@dramatiq.actor(max_retries=3)
def count_words(url):
...# Max age
@dramatiq.actor(max_age=3600000)
def count_words(url):
...# Set RabbitMQ as a broker
from dramatiq.brokers.rabbitmq import RabbitmqBroker
rabbitmq_broker = RabbitmqBroker(host="rabbitmq")
dramatiq.set_broker(rabbitmq_broker)
Check it out here: https://dramatiq.io
4. manven
Virtual Environments (or venv) are the recommended way of setting up development environments or even production ones (before docker). I usually find myself creating multiple ones for each project and the odd Python 2 venv for the library that hasn’t made its way to Python 3 yet. Instead of fancy aliases I find using manven for creating, switching or managing virtual environments.
# Create ~/.manven.conf with content[manven]
ENVS_PATH=path/to/your/dir# Create a new venv
smanven create venv# Activate virtual env
smanven activate venv# Remove virtual env
smanven remove venv# Setup a temporary venv
smanven temp
Check it out here: https://github.com/AckslD/manven
5. tranquilizer
By far the easiest way to create a JSON HTTP API in Python. A general use case can be to set up prototypes quickly without worrying about importing a bunch of libraries. I’ve used it mainly to mock for testing and to deploy proof of concepts.
from tranquilizer import tranquilize
@tranquilize()
def order(cheese):
'''I'd like to buy some cheese!'''
return {'response':"I'm afraid we're fresh out of {}, Sir.".format(cheese)}# Client side
In [1]: import requests
In [2]: response = requests.get('http://localhost:8086/order', params={'cheese':'Red Leicester'})
In [3]: response.json()
Out[3]: {'response': "I'm afraid we're fresh out of Red Leicester, Sir."}
Check it out here: https://github.com/AlbertDeFusco/tranquilizer
Wrapping up
Thank you for reading through the list. I will be planning on posting at least 5 upcoming libraries every quarter. If you are working on a new library for Python (or any other programming language) please reach out and I will add it to the next edition or feel free to reach out and have your story added to the Upcoming Open Source Publication.
