liveBook · Manning

Bentoml runnable

As explained in Machine Learning Platform Engineering

A BentoML base class for defining runnable units of model or application computation.

A BentoML Runnable packages model computation into a reusable execution unit. Its constructor initializes the model, while methods marked with @bentoml.Runnable.method become remotely invocable through a BentoML Service. section 10.4.1

import bentoml
import json
from ultralytics import YOLO

class YOLOv8Runnable(bentoml.Runnable):

   def __init__(self):
       self.model = YOLO("yolov8_custom.pt")

   @bentoml.Runnable.method(batchable=False)     #1
   def inference(self, input_img):               #2
       results = self.model(input_img)[0]        #3
       return json.loads(results[0].tojson())    #4

A YOLOv8 Runnable loads weights directly, exposes non-batchable image inference, selects the first result, and returns JSON-compatible output. section 10.4.1

Register the Runnable with bentoml.Runner, attach that Runner to a Service, and invoke its method asynchronously from an API endpoint. Direct constructor loading keeps the first service simple; model registry features support later versioning and dynamic loading. section 10.4.1

How Machine Learning Platform Engineering covers it

How deep it goes
Explains subclassing, constructor loading, decorated methods, and asynchronous invocation, but defers dynamic model management.
Examples
Uses a YOLOv8 image detector returning JSON and a separate rendering method returning an annotated image.
What it assumes
Assumes a BentoML service context, Python model code, and locally available YOLOv8 weights.
What you can do after
Build it from scratch as a BentoML Runnable, register it with a Runner, and expose inference through an asynchronous API.
How it connects
Connects Runnables to BentoML methods, Runners, Services, APIs, and model registries.
Cover of Machine Learning Platform Engineering
liveBookmanning.com
Read Machine Learning Platform Engineering on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.