Understanding event loop with Python

Illia Pekelny
4 min readFeb 3, 2018

--

Dining Philosophers logical problem scheme

Everybody builds event loop to understand and describe how does it work. That is great. I want to understand it too. So I decided to build one… A fake one… Like everybody does… But what I want to point out to — that it is fake!

What is event loop?

Event loop is a freely treated programming construct and a clue tool for co-op programming and concurrency itself. But it grows true sense when it used for building asynchronous programs. For that reason I consider any forms of event loops that couldn’t be used to build asynchrony — fake.

Event loop waits for and dispatches events or messages in a program. It works by making a request to some internal or external “event provider” (that generally blocks the request until an event has arrived), and then it calls the relevant event handler (“dispatches the event”).

Event loop is a solution to a classic logical problem “Dining Philosophers”:

Five silent philosophers sit at a round table with bowls of spaghetti. Forks are placed between each pair of adjacent philosophers.

Each philosopher must alternately think and eat. However, a philosopher can only eat spaghetti when they have both left and right forks. Each fork can be held by only one philosopher and so a philosopher can use the fork only if it is not being used by another philosopher. After an individual philosopher finishes eating, they need to put down both forks so that the forks become available to others.

Event loop concurs partial order acquiring shared resources and dispatching events. In the problem forks are shared resources, philosophers are acquiring processes, satiety and desire to think are events used to concur and balance resources usage.

Exact loop

Below is an event loop. It could help to understand how asynchrony possible but in current state it couldn’t implement it.

tasks = {"-=* BOOM *=-": 2, "~=* shaka *=~": 1}

Tasks. In real event loop tasks are created from events/messages coming from outer process. In the fake implementation the tasks are constant and presented forever inside the loop.

if not in_action:
in_action = [
coro(sound, delay) for sound, delay in tasks.items()
]

Actual tasks to be handled in event loop. The tasks are almost real from that point of view. A task will be handled accordingly to events which it produces. Loop itself doesn’t care about computation in tasks or relation of task with outer world it cares only about events. A difference is in task implementation.

task = in_action.pop(0)

Get the task to handle. In this particular loop I take next task blindly. In real program we can have additional information about previous events and for example heapify tasks using this data. There is no principal difference.

dl = task.send(None)

Awake task. This line combines several purposes. At first this is task handler call which leads to obtaining final result of task computation. Also it gets partial computation result at each iteration. And finally it is task event handling. We have two events ‘ready’ and ‘in progress’. When a task is in progress only intermediate result (meta data) will be obtained from awaitable object. If the task is ‘ready’ StopIteration error will be raised and result value will be available in it. In our fake implementation no computation exists behind the task. Intermediate computation results used only to keep infinity tasks creation (initial delay used to create the same task when a task is done). Event system simplified too — “ready” and “in progress” could be not enough, each intermediate computation could bring different events.

in_action.append(coro(e.value, tasks[e.value]))

The loop is infinity so a new task with the same arguments will be added when a task is done.

in_action.append(task)

Put “in progress” tasks back into the loop.

The clue difference

coro coroutine awaits from an instance of Delay awaitable object. There is no computations. Also the pipeline doesn’t communicate with outer processes. But both steps of the pipeline diligently pretend asynchrony using the main feature of coroutines — state suspending. OS thread is a shared resource for all the started tasks. So if to implement any computation (blocking operation) inside the tasks it will take exactly the same time regardless it executed in strict order or in partial order. The sum is the same regardless of how you group the numbers. Is it useless? No if intermediate results matter. No if time between obtaining final results matter.

Understanding asynchrony

In the loop we didn’t implemented true (useful) asynchrony but there is concurrency which helps us to understand how asynchrony works.

We are paying with time here. And time is common for each task in the loop. Defining delay in absolute numbers (seconds) for each task we can operate with tasks relatively to a current moment because both steps of the pipeline can suspend its’ state. Delay instance will be kept in coro until coro return. Time of ‘ready’ event will be kept by Delay awaitable object until it became actual. So each task spent exactly given time (in delay argument) waiting but the tasks spent these time periods simultaneously but not coherently.

So this is asynchrony between tasks and time. Time is common, objective and produces events for the tasks.

What is missing to implement useful asynchrony?

  • Blocking operation should be performed in different process(es). This is asynchrony between co-op task scheduler and independent computation processes (in wide meaning).
  • Events from these processes should be provided using interprocess communication tools — non-blocking sockets.
  • Also tasks could be provided by events/messages source outside of the loop. (Reactor pattern)

Conclusion

It is good to understand concurrency and asynchrony using simple practical examples. But don’t believe the hype, real asynchronous programs more complex and uses much more tools than bare event loop. Albeit understanding of co-op essence of event loop makes understanding of asynchrony a low hanging fruit.

--

--

Illia Pekelny

Self — the source of sense centralized in time and space.