liveBook · Manning

Event-driven architecture

As explained in Distributed Machine Learning Patterns

A processing architecture in which computation is initiated in response to incoming events or requests.

Event-driven architecture initiates computation when an event arrives rather than keeping request-handling servers continuously running. In model serving, a user request is the event: the system acquires needed resources, loads a trained model from a distributed database, and produces a prediction for the request. section 4.4.2

This pattern addresses services whose traffic is sparse, irregular, or difficult to forecast. Resources come from a shared pool only when requests occur, allowing a serving system to absorb an unexpected traffic surge while avoiding resources that would otherwise sit idle during quiet periods. section 4.4.2 section 4.9

Figure 4.16 A diagram of the event-driven model serving system to predict hotel prices
04-16

The request-triggered path shows a hotel-price query activating model serving rather than relying on an always-running serving process. section 4.4.2

Event-driven serving is a practical fit when it needs to:

  • Respond with real-time, more accurate predictions immediately after users submit requests instead of returning predictions calculated on a schedule. section 4.4.3
  • Scale model-serving functions automatically as traffic grows, without developers manually allocating the serving resources. section 4.4.3
  • Deploy a trained model as a request-triggered function, with source code as the deployable input rather than a long-running service artifact. section 4.4.3

Event-driven processing is not a universal replacement for long-running serving systems. Replicated and sharded services keep servers running and suit heavy load, large in-memory data, or background processing. Regularly scheduled predictions may not need event monitoring, and applications that accept less-accurate results can recalculate predictions at a daily or weekly granularity instead. section 4.4.2 section 4.4.3

Hotel price prediction on demand

A time-and-location price query triggers the serving function. It draws resources from the shared pool, retrieves and loads the trained model, estimates the hotel price, and releases the system from maintaining capacity for periods without queries. section 4.4.2

Rate limits may still constrain an event-driven service. When four serving requests arrive but only two are permitted concurrently, the allowed requests run first and the remaining requests wait in a queue until capacity becomes available. section 4.4.2

How Distributed Machine Learning Patterns covers it

How deep it goes
Explains the request-to-prediction flow, shared resources, automatic scaling, and trade-offs against long-running serving.
Examples
Uses hotel price prediction queries, an unexpected pre-Christmas traffic spike, and a rate-limited queue of four requests.
What it assumes
Assumes familiarity with trained models and model serving while defining request-triggered processing through a concrete serving flow.
What you can do after
Evaluate/understand when request-triggered model serving can reduce idle resources and provide real-time predictions.
How it connects
Connects event-driven processing to model serving, function-as-a-service, replicated services, sharded services, rate limiting, and shared resource pools.
Cover of Distributed Machine Learning Patterns
liveBookmanning.com
Read Distributed Machine Learning Patterns on liveBook Get the book at manning.com

Also explained in

Compare how 10 books explain event-driven architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Distributed Machine Learning Patterns. The section links point to the parts of the book it draws on.