Photo by Alex Jones on Unsplash

Raise event vs. event:send

Understanding event raising

Recently at Pico Labs, we ran into some big problems regarding event scheduling, so much so that we dedicated a good chunk of time towards addressing it in our weekly meeting. This goes to prove at least two things:

1. It doesn’t matter how familiar you are with KRL, there is always more to learn.
2. Raising events can cause some headache for KRL developers.

With that being said, let’s dive into the context of what was happening and what we can learn.

The Problem

The problem came in the format of a KRL ruleset that was installed on a Pico that had about three-thousand child Picos installed. There is no need to examine the entire ruleset, suffice it to say that there were many child Picos that did not have any valuable information and needed to be deleted, and some needed to be kept. Look closely at lines 37–47 and see if you can locate why this code was buggy.

Couldn’t find anything? Neither could any of us at the lab; it seems to be flawless KRL. In fact, it is flawless, if we were to be dealing with only a small handful of child Picos. To understand what goes wrong with this code requires a firm understanding of how events can be placed on a schedule or a queue, depending if you use raise event or event:send(), and how a schedule can be more fragile than the queue when dealing with many child picos.

Schedule vs. the Queue

The problem with this code lies in line 47. The syntax in and of itself is perfectly fine. But what goes on under the hood is what causes the errors. If you make the decision to use raise wrangler event, it is important to know that each of those events will be placed on a schedule. In other words, the first wrangler child_deletion event will not be raised until the other 2,999 picos have been processed in the foreach loop and possibly had their own wrangler child deletion events added to the same schedule.

A Pico spends most of its life sitting and waiting to receive an event to process. This is the queue. When that Pico receives an event, it will raise it, and now causing the Pico to create a schedule of rules to run for that event. So the important thing to know here is that a Pico has a queue, and the queue holds events; and the Pico has a schedule that holds rules to run. Whenever we use raise event, then more rules are added onto the end of the schedule, and those need to run to completion. So the problem that we were running into was that the very first child deletion that reached the top of the schedule, after many rules had been fired on the schedule for the Picos, was failing for some unknown reason. And due to a bug in the Pico Engine, the behavior of the schedule does not continue to event attempt to perform the hundreds, or even thousands, of wrangler child_deletion rules to be raised.

When dealing with a Pico Engine, often times we forget that we have a finite amount of resources to deal with. In the above case, our schedule was flooded with many rules, and since the first one failed, the rest failed too. When we send events using event:send(), events are pushed onto a queue. And even though the queue also has a finite amount of resources to allocate to events, there is a much better chance of your pico engine not crashing from it. If one child_deletion rule fails, the queue simply moves onto the next one.

Although using the raise event syntax is the most popular option for raising events in KRL, it should be understood that it can cause problems when dealing with child picos. When you use raise event, the process is as follows:

1. The Pico in which the rule was raised, adds the event to its schedule
2. Items in the Picos schedule continue to be executed, and the raised event moves up.
3. Once the event reaches the top of the schedule, it is then taken and added to the queue of the Pico where the rule is running.
4. The receiving Pico then executes the raised event when it is able to be popped from the queue.

In the code example above, the code crashed by the time it reached the first call to delete a child Pico. Now that could happen for many reasons, but the imporant thing to realize is that by using the raise event syntax in KRL, the schedule failed, and it caused the entire schedule to be thrown away as a result of it.

Using the Queue

If we wanted to look at the other way of writing the above code, we would delete line 47, and replace the noop() with the following code:

if progresss.klog("progress").get("size") == 0 then
   event:send({
      "eci" : meta:eci,
      "domain" : "wrangler",
      "type" : "child_deletion",
      "attrs" : {"id" : o.getId("id")}
   })
fired {
   ent:deletions := ent:deletions.defaultsTo([]).append(orderno)
}
.
.
.

Where the meta:eci() is the Event Channel Identifier of the current Pico, which allows it to send events to its children.

TL;DR

The most important thing to takeaway from this brief example is that there exists two ways of sending events to other picos. Raise event involves a schedule, and event:send() involves a queue. When we used raise event for the above example, we had 3000 events on the schedule of one Pico, and when we reach those events, they will be sent to the queue of each of those 3000 picos. In contrast, when we use event:send(), we have 3000 events on 3000 different Picos queues. Event:send() allowed us to send those events to each Pico directly, instead of having to wait on the parent Picos schedule to do it. It also allows us to decentralize the events so that they are not all in one place, which ends badly if there is an error.

1. The schedule is more brittle than the queue. As of now, when a Pico schedule fails, it does not fail gracefully, and all other events that have not been realized will be thrown away.
2. The main difference between raise event and event:send() is the use of a schedule and a queue respectively.
3. In our example above, when we use event:send() as opposed to raise event, we have 3,000 different Picos with their own event to fire, as opposed to having 1 schedule with 3,000 different events to send, which will only then be added to each individual Pico to fire. Event:send() removes that extra step and makes raising events safer for your Pico when it tries to communicate with other Picos.

To read more about events, click here.

For further reading on KRL, visit the Official KRL Documentation.