concept readiness probe in category kubernetes

This is an excerpt from Manning's book Learn Kubernetes in a Month of Lunches MEAP V07. Login to get full access to this book.

If the probe response says the container is unhealthy, Kubernetes will take action - and the action it takes depends on the type of probe. Readiness probes take action at the network level, so they're for components which listen for network requests. If the Pod container is unhealthy, the Pod is taken out of the ready state, and it is removed from the list of active Pods for a Service. Figure 12.1 shows how that looks for a Deployment with multiple replicas, where one Pod is unhealthy.

Figure 12.1 The list of endpoints for a Service excludes Pods which are not ready to receive traffic

Readiness probes are a great way to manage temporary load issues. Some Pods might be overloaded, returning a 503 status code to every request. If the readiness probe checks for a 200 response and those Pods return 503 they will be removed from the Service and they'll stop receiving requests. Kubernetes keeps running the probe after it has failed, so if the overloaded Pod has a chance to recover while it’s resting, the probe will succeed again and the Pod will be enlisted back into the Service.

The random number generator we've used in this book has a couple of useful features we can use to see how this works. The API can run in a mode where it fails after a certain number of requests, and it has an HTTP endpoint which returns whether it is healthy or in a failed state. We'll start by running it without a readiness probe so we can understand the problem.

This happens because Kubernetes doesn't know one of the Pods is unhealthy. The application in the Pod container is still running and Kubernetes doesn't know there's a health endpoint it can use to see if the app really is working correctly. You can give it that information with a readiness probe in the container spec for the Pod - listing 12.1 shows an update to the API spec which includes the health check.

Listing 12.1 - api-with-readiness.yaml, a readiness probe for the API container

spec:             # this is the Pod spec in the Deployment
  containers:
    - image: kiamol/ch03-numbers-api
      readinessProbe:        # probes are set at the container level
        httpGet:
          path: /healthz     # this is an HTTP GET, using the health URL
          port: 80      
        periodSeconds: 5     # the probe fires every five seconds

Kubernetes supports different types of container probe - this uses an HTTP GET action, which is perfect for web applications and APIs. The probe tells Kubernetes to test the /healthz endpoint every five seconds; if the response has an HTTP status code between 200 and 399 then the probe succeeds; any other status code and it will fail. The random number API returns a 500 code when it's unhealthy, so we can see the readiness probe in action.

This is an excerpt from Manning's book Kubernetes in Action. Login to get full access to this book.

Kubernetes also supports readiness probes, which we’ll learn about in the next chapter. Be sure not to confuse the two. They’re used for two different things.

Listing 5.17. RC creating a pod with a readiness probe: kubia-rc-readinessprobe.yaml

apiVersion: v1
kind: ReplicationController
...
spec:
  ...
  template:
    ...
    spec:
      containers:
      - name: kubia
        image: luksa/kubia
        readinessProbe:           #1
          exec:                   #1
            command:              #1
            - ls                  #1
            - /var/ready          #1
        ...

The readiness probe will periodically perform the command ls /var/ready inside the container. The ls command returns exit code zero if the file exists, or a non-zero exit code otherwise. If the file exists, the readiness probe will succeed; otherwise, it will fail.

Understanding why readiness probes are important

Imagine that a group of pods (for example, pods running application servers) depends on a service provided by another pod (a backend database, for example). If at any point one of the frontend pods experiences connectivity problems and can’t reach the database anymore, it may be wise for its readiness probe to signal to Kubernetes that the pod isn’t ready to serve any requests at that time. If other pod instances aren’t experiencing the same type of connectivity issues, they can serve requests normally. A readiness probe makes sure clients only talk to those healthy pods and never notice there’s anything wrong with the system.