Sidecar process

As explained in Designing Deep Learning Systems: A guide for software engineers

A container deployed alongside a primary application container to provide supporting functionality such as adaptation, proxying, or initialization.

A sidecar process runs beside a primary application container, encapsulating adaptation, proxying, and initialization requirements without changing the primary service. section 7.2.7 section 7.4.5

In model serving, the sidecar downloads model files into shared storage, registers the model, translates requests, and forwards them to a black-box inference server. section 7.2.7 section 7.4.5

A sidecar supports deployment by:

• Keeping the public prediction API and external model storage unchanged while replacing or integrating the backend. section 7.4.5
• Sharing a volume across server pods to provide a common model repository and reduce repeated downloads and network bandwidth. section 7.2.7 section 7.4.5
• Letting a load balancer distribute prediction requests across pods without exposing server-specific registration requirements to callers. section 7.2.7

Figure 7.7 Add a proxy container in the TorchServe pod to scale up TorchServe in Kubernetes.

Use the architecture diagram to trace the proxy, TorchServe container, shared model store, and load-balanced pod workflow. section 7.2.7

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Explains container responsibilities, shared storage, request flow, model registration, and scaling, but does not provide deployment configuration.

Examples

Uses TorchServe and Triton pods with proxy containers that download, register, and serve models from shared repositories.

What it assumes

Assumes Docker-based containers and a serving system with model storage, prediction requests, and a model server.

What you can do after

Operate/configure an adapter-proxy container that integrates a model server without changing public inference interfaces or external model storage.

How it connects

Connects Kubernetes pods, Docker containers, load balancing, shared volumes, model repositories, TorchServe, Triton, and KServe.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Effective Platform EngineeringA sidecar proxy is a proxy deployed alongside an application workload to handle communication without requiring application-code changes.
• 

  liveBookmanning.com
  Google Anthos in ActionA sidecar is a container deployed alongside a main container in the same Pod to provide supporting functionality such as proxying.
• 

  liveBookmanning.com
  Logs and TelemetryThe sidecar pattern deploys a supporting process alongside a primary application component so they share a lifecycle and execution context.
• 

  liveBookmanning.com
  Kubernetes in Action, Second EditionThe sidecar pattern places a complementary process in a separate container alongside a primary application process in the same pod.
• 

  liveBookmanning.com
  Authorization in ActionA sidecar is a separately deployed component or proxy that runs alongside a primary service to provide supporting functionality.
• 

  liveBookmanning.com
  Podman in Action: Secure, rootless containers for Kubernetes, microservices, and moreA sidecar container is an additional co-located container that supports or monitors a primary container or its execution environment.
• 

  liveBookmanning.com
  Acing the Certified Kubernetes Administrator Exam, Second EditionThe sidecar pattern runs a supporting process alongside a primary application process, typically within the same pod.
• 

  liveBookmanning.com
  Secure APIsA sidecar is a container-application pattern in which a secondary process runs alongside a primary application to provide supporting functionality.
• 

  liveBookmanning.com
  Kafka for ArchitectsThe sidecar pattern deploys an auxiliary component alongside a primary service component to provide supporting functionality without embedding it in the service.
• 

  liveBookmanning.com
  LLMs in ProductionA sidecar is a companion component deployed alongside a primary application to perform supporting processing or communication.

Compare how 11 books explain sidecar process →

Related concepts

• service meshused by · A service mesh is an infrastructure layer that manages communication and cross-cutting concerns such as traffic control, security, and observability between distributed services.
• poduses · A Kubernetes Pod is the smallest deployable unit in Kubernetes, grouping one or more co-located containers that share networking and other resources.
• containerkind of · A software container is an isolated execution environment that packages an application and its dependencies while sharing the host operating system kernel.
• Istiopart of · Istio is an open-source service mesh platform that provides traffic management, security, and observability for service-to-service communication.
• automatic sidecar injectionproduced by · Automatic sidecar injection is the admission-time addition of a proxy sidecar to newly created pods based on cluster configuration.
• NVIDIA Triton Inference Serviceuses · NVIDIA Triton Inference Server is open-source software for deploying and serving machine-learning models through standardized interfaces across frameworks and hardware backends.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.