6 Troubleshooting large-scale network errors

This chapter covers

Confirming cluster functionality with Sonobuoy
Tracing a Pod’s data path
Using the arp and ip commands to inspect CNI routing
A deeper look at kube-proxy and iptables
An introduction to Layer 7 networking (the ingress resource)

In this chapter, we’ll go over a few touchpoints for troubleshooting large-scale network errors. We also introduce Sonobuoy, a Swiss Army knife for certifying, diagnosing, and testing the functionality of live Kubernetes clusters, which is a commonly used diagnosis tool for Kubernetes.

Sonobuoy compared with the Kubernetes e2e (end-to-end) tests

Sonobuoy runs the Kubernetes e2e testing suite in a container and makes it easy to retrieve, store, and archive the overall results.

For advanced Kubernetes users that commonly work on Kubernetes from the source tree, you can directly use the test/e2e/ directory (found at http://mng.bz/Dgx9) as an alternative to Sonobuoy. We recommend it as an entry point to learn more about how to run specific Kubernetes tests.

Sonobuoy is based on the Kubernetes e2e testing library. Sonobuoy is used to verify Kubernetes releases and validate whether the software correctly follows the Kubernetes API specification. After all, Kubernetes is ultimately just an API, and so the way that we define a Kubernetes cluster is as a set of nodes that can successfully pass the Kubernetes conformance test suite.

6.1 Sonobuoy: A tool for confirming your cluster is functioning

6.1.1 Tracing data paths for Pods in a real cluster

6 Troubleshooting large-scale network errors

This chapter covers

Sonobuoy compared with the Kubernetes e2e (end-to-end) tests

6.1 Sonobuoy: A tool for confirming your cluster is functioning

6.1.1 Tracing data paths for Pods in a real cluster

6.1.2 Setting up a cluster with the Antrea CNI provider

6.2 Inspecting CNI routing on different providers with the arp and ip commands

6.2.1 What is an IP tunnel and why do CNI providers use them?

6.2.2 How many packets are flowing through the network interfaces for our CNI?

6.2.3 Routes

6.2.4 CNI-specific tooling: Open vSwitch (OVS)

6.2.5 Tracing the data path of active containers with tcpdump

6.3 The kube-proxy and iptables

6.3.1 iptables-save and the diff tool

6 Troubleshooting large-scale network errors

This chapter covers

Sonobuoy compared with the Kubernetes e2e (end-to-end) tests

6.1 Sonobuoy: A tool for confirming your cluster is functioning

6.1.1 Tracing data paths for Pods in a real cluster

6.1.2 Setting up a cluster with the Antrea CNI provider

6.2 Inspecting CNI routing on different providers with the arp and ip commands

6.2.1 What is an IP tunnel and why do CNI providers use them?

6.2.2 How many packets are flowing through the network interfaces for our CNI?

6.2.3 Routes

6.2.4 CNI-specific tooling: Open vSwitch (OVS)

6.2.5 Tracing the data path of active containers with tcpdump

6.3 The kube-proxy and iptables

6.3.1 iptables-save and the diff tool

Unable to load book!