liveBook · Manning

Event queue

As explained in Grokking Concurrency

An event queue is an ordered collection of pending events awaiting processing by an application or event loop.

An event queue is a holding line for events that have arrived but still need attention. An event loop waits for events, retrieves them from the queue, and invokes the callback registered for each one. This separates noticing an event from handling it, allowing an application to coordinate many event types without busy-waiting. section 11.3

The basic flow is straightforward:

  • Register events and the callbacks that should handle them. section 11.3
  • Add events to the queue as they arrive. section 11.3
  • Let the event loop retrieve pending events and transfer control to the matching callback. section 11.3 section 11.5
  • Return to waiting when no queued event remains, or continue processing newly added events. section 11.5

From knock-knock events to a pizza server

A small Python event loop registers knock and who events, manually queues two knock events, and processes them one after another; a knock callback can add a who event. The same pattern scales to a server: socket-read, socket-write, and connection events enter the queue, while callbacks handle each client without requiring a separate thread. I/O multiplexing lets the operating system notify the loop when registered sockets are ready instead of forcing the application to poll constantly. section 11.3 section 11.5 section 11.4

# Chapter 11/event_loop.py
from collections import deque
from time import sleep
import typing as T

class Event:                                                       #1
    def __init__(self, name: str, action: T.Callable[..., None], 
                 next_event: T.Optional[Event] = None) -> None: 
        self.name = name 
        self._action = action
        self._next_event = next_event 

    def execute_action(self) -> None:
        self._action(self)
        if self._next_event:
            event_loop.register_event(self._next_event)

class EventLoop:
    def __init__(self) -> None:
        self._events: deque[Event] = deque()                       #2

    def register_event(self, event: Event) -> None:
        self._events.append(event)                                 #3

    def run_forever(self) -> None:
        print(f"Queue running with {len(self._events)} events")
        while True:                                                #4
            try:                                                   #4
                event = self._events.popleft()                     #4
            except IndexError:                                     #4
                continue                                           #4
            event.execute_action()                                 #4

def knock(event: Event) -> None:
    print(event.name)
    sleep(1)

def who(event: Event) -> None:
    print(event.name)
    sleep(1)

if __name__ == "__main__":
    event_loop = EventLoop()
    replying = Event("Who's there?", who)
    knocking = Event("Knock-knock", knock, replying)
    for _ in range(2):                                             #5
        event_loop.register_event(knocking)                        #5
    event_loop.run_forever()                                       #6

Shows a compact Python implementation using deque, event registration, callbacks, and events that can schedule follow-up events. section 11.3

# Chapter 11/pizza_reactor.py
class EventLoop:
    def __init__(self) -> None:
        self.writers = {}                                          #1
        self.readers = {}                                          #1

    def register_event(self, source: socket, event: Mask,
                       action: Action) -> None:
        key = source.fileno()                                      #2
        if event & select.POLLIN:                                  #3
            self.readers[key] = (source, event, action)            #3
        elif event & select.POLLOUT:                               #4
            self.writers[key] = (source, event, action)            #4

    def unregister_event(self, source: socket) -> None:            #5
        key = source.fileno() 
        if self.readers.get(key): 
            del self.readers[key]
        if self.writers.get(key): 
            del self.writers[key]

    def run_forever(self) -> None:
        while True:                                                #6
            readers, writers, _ = select.select(                   #6
                self.readers, self.writers, [])                    #6
            for reader in readers:                                 #7
                source, event, action = self.readers.pop(reader)   #7
                action(source)                                     #7

            for writer in writers:                                 #8
                source, event, action = self.writers.pop(writer)   #8
                action, msg = action                               #8
                action(source, msg)                                #8

Shows how a server event loop records readable and writable sockets and associates them with actions. section 11.5

How Grokking Concurrency covers it

How deep it goes
Implements queueing with Python deque and connects processing to socket readiness, without detailing queue policies beyond callback order.
Examples
Demonstrates knock-knock callbacks and a pizza server handling socket-read, socket-write, and connection events.
What you can do after
Build it from scratch with Python deque, callbacks, and socket readiness registration.
How it connects
Connects event loops, callbacks, I/O multiplexing, nonblocking sockets, and single-threaded concurrency.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Concurrency. The section links point to the parts of the book it draws on.