Less Known Python Features
Exploring some of the unfamiliar yet cool Python features
Python has lots of cool features such as first class functions, lambdas, generators, comprehensions etc. There are plenty of articles on the subject. However, there are some of the features in Python, which are generally not so commonly used.
In this article, we’ve tried to summarize some of the these features, which will provide us a better knowledge on the subject and can come in handy at times for developers.
So let’s dive in!
1. Limiting CPU and Memory Usage
Resources like CPU, memory utilised by our Python program can be controlled using the resource library.
To get the processor time (in seconds) that a process can use, we can use the resource.getrlimit() method. It returns the current soft and hard limit of the resource.
import resourcec_soft, c_hard = resource.getrlimit(resource.RLIMIT_CPU)
print('soft limit %d secs, hard limit %d secs' % (c_soft, c_hard))>>>
soft limit 922304775807 secs, hard limit 922304775807 secs
Each resource is controlled by a pair of limits: a soft limit and a hard limit. The soft limit is the current limit, and may be lowered or raised by a process over time. The soft limit can never exceed the hard limit. The hard limit can be lowered to any value greater than the soft limit, but not raised.
The following code snippet limits the CPU usage of our the program by setting the soft limit to 10 seconds. Hence the process won’t be getting more than 10 seconds of processor time:
import resource
import signal
import timedef time_expired(n, stack):
print('EXPIRED :', time.ctime())
raise SystemExit('(time ran out)')def set_cpu_runtime():
# Install the signal handler and set a resource limit
soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print('Soft limit starts as :', soft)
resource.setrlimit(resource.RLIMIT_CPU, (10, hard))
soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print('Soft limit changed to :', soft)
signal.signal(signal.SIGXCPU, time_expired)set_cpu_runtime()# Performing a CPU intensive task
print('Starting:', time.ctime())
for i in range(300000):
for j in range(220000):
_ = i * j# Will not be able to execute this!
print('Exiting :', time.ctime())
Here, for CPU limit we first get soft and hard limit for that specific resource (RLIMIT_CPU) and then set it to 10 seconds while keeping the hard limit same as before. Finally, we register signal that causes system exit if CPU time is exceeded.
Executing the above snippet will generate the following output:
Soft limit starts as : 922304775807
Soft limit changed to : 10
Starting: Sun Apr 5 20:27:09 2020
EXPIRED : Sun Apr 5 20:27:19 2020
(time ran out)As for the memory, we again can retrieve soft and hard limit using resource RLIMIT_AS and set it using setrlimit with desired parameters.
# To limit memory usage
def set_max_memory(size): # size (in bytes)
soft, hard = resource.getrlimit(resource.RLIMIT_AS)
resource.setrlimit(resource.RLIMIT_AS, (size, hard))2. Controlling what can be Imported and what Not
In Python, using from <module> import * will import all the public objects of that module. For example:
# person.pyname = "Sarah"
age = 26def foo():
return name# __all__ = ["name", "age"] <-- commented out
When we’ll import the person module using the from person import * statement, behind the scenes it will import all the symbols present in the __all__ variable of the module.
__all__ variable is a list of strings defining what symbols in a module will be exported. By default, it will include all the public objects as strings, hence we can access any member of person module as below:
>>> from person import *
>>> print(name, age)
Sarah, 26>>> foo()
'Sarah'
However we can limit what can be imported by overriding the value for __all__. For the above example, if we uncomment the __all__ = ["name", "age"] statement and try to access the foo() method, it will throw us an error:
>>> from person import *
>>> print(name, age)
Sarah, 26>>> foo()
NameError: name 'foo' is not defined
Therefore, __all__ lets us know the "public" features of a module. It should be noted however that the import without * are not affected at all.
>>> from person import foo
>>> foo()
'Sarah'3. Functions with only Keyword Arguments
We can intentionally create functions that only takes keyword arguments to enforce more clarity whilst using such the function:
def person(*, name, age):
print(name, age) >>> person("Sarah", 26)
TypeError: person() takes 0 positional arguments>>> person(name="Sarah", age=26)
Sarah 26
As we can see this can be easily be done by placing single * argument before keyword arguments. There can obviously be positional arguments if we place them before the * argument.
4. Saving Memory with __slots__
In Python every class can have instance attributes. If we ever write a program that is creating really big number of instances of some class, we might notice that our program will be needing a lot of memory.
This is because, by default Python uses a dictionary to store an object’s instance attributes, which makes it fast but not very memory efficient, (usually not a problem). However, if it becomes a problem for our program we might try using __slots__.
slots provide a special mechanism to reduce the size of objects.
class Person:
__slots__ = ["name", "age", "email"] def __init__(self, name, age, email):
self.name = name
self.age = age
self.email = email
In the above snippet, we define __slots__ attribute which indicates Python to use small fixed-size array for the attributes instead of dictionary, which greatly reduces memory needed for each instance.
There are also some downsides to using __slots__. We can't declare any new attributes dynamically, therefore restricted to using the ones on __slots__.
>>> p = Person('Sarah', 26, 'akp345@gmail.com')
>>> p.new = 'Not Possible'
AttributeError: 'Person' object has no attribute 'new'Conclusion:
Some of the features are not essential and even not commonly used in our day-to-day Python programming, but some of them might come in handy from time to time and they also might simplify our task. Being aware of these concepts will make us a better Pythonistas 😄
