Serverless computing is a cloud execution model for hosting and running models on demand. It provides flexible, modular processing without requiring a persistent application server, making it especially suitable for bursty workloads. section 9.4.2 section 9.1
Assess serverless deployment against the workload and its operational constraints:
- Choose it for highly scalable, bursty workloads where demand varies substantially. section 9.4.2
- Accept comparatively high startup latency instead of the consistently low latency available from persistent approaches such as Kubernetes. section 9.4.2
- Review pricing assumptions carefully and maintain an offramp because provider pricing changes could disrupt the system. section 9.4.2
- Include the choice in production design alongside data flows, model serving, interfaces, logging, monitoring, alerting, governance, and support documentation. section 9.1
The Bike Shop production prototype
A development environment uses a serverless processor to deploy cloud-based transformations and feature creation. The team divides related work across the user interface, model-server design, and data pipeline before reviewing the prototype and production design. section 9.10
Table 9.5 Using generic execution engines for ML models (view table figure)
| Engine |
Comments |
|---|---|
| In a database: BigQuery ML, Redshift ML, Oracle OCI, Vertica PMML |
Execution in database functions means accepting constraints on how the application runs, but this is a modular and decoupled approach. |
| Application servers: NGINX, Apache Tomcat, Flask, Appserver |
For generic low-scale and permissive latency/cost use cases. Consider because of the low barrier in terms of skills and setup requirements. |
| SPARK-like solutions: Apache SPARK, Databricks, Dataproc |
Good support for bulk and stream processing with permissive cost and latency requirements. |
| Kubernetes, Kubeflow |
Flexible and scalable support for interactive applications with permissive cost requirements. |
| Serverless: Lambda, Cloud Functions, Azure Functions, Cloud Run. |
A flexible, modular, and decoupled approach that’s cost effective and performative in some applications but can become expensive in unpredictable ways. |
Compare serverless with database execution, application servers, SPARK-like systems, and Kubernetes across flexibility, scale, cost, latency, and setup requirements. section 9.4.2
