Chapter 8: Control plane services and extensions

Introduction

The introduction establishes the platform’s current control plane baseline and identifies services and extensions as the cluster-wide components still to be added.

• Control plane baseline: Describe what the Epetech platform has at this point and what its control plane still lacks.
  Why it matters: This baseline explains why the chapter turns to adding control plane components.
• Cluster-wide component domains: Explain what services and extensions collectively represent in this introduction, without assigning a definition to either category.
  Why it matters: It sets the scope of the components the chapter will address without yet explaining the distinction between the two categories.

Key terms: sandbox, production instance, Elastic Kubernetes Service (EKS), control plane, control plane services, control plane extensions, cluster-wide services

8.1 Value of services and extension domains for smaller teams

This section explains why a platform can start with one small team and how separating control plane services from extensions helps that team deliver to early customers.

• Small Team Delivery Tradeoffs: Explain why a platform serving a small developer pool may be better delivered by one team rather than several specialized teams.
  Why it matters: The number of platform teams should reflect the value they add, not just the possibility of specializing.
• Staged Platform Team Growth: Justify starting with one platform team and one or two alpha customer teams even when the platform is expected to serve hundreds or thousands of developers later.
  Why it matters: Early customers give the platform team a chance to validate both the product and its eventual ownership boundaries before investing in scale.
• Service And Extension Pipelines: Describe how a single platform team initially organizes cluster-wide control plane applications and why that arrangement helps.
  Why it matters: A simple division of orchestration work helps a small team deliver sooner without stepping on one another’s work.

Key terms: internal developer pool, platform team, team specialization, alpha customers, product management, feedback skills, domain boundaries, control plane applications, services, extensions

8.2 Value of services and extension domains for scaled platform delivery

This section distinguishes platform services from extensions and explains how that distinction guides delivery responsibilities as a platform grows.

• Services and extensions distinction: Distinguish a platform extension from a service by the developer experience it provides, rather than by the technology deployed.
  Why it matters: The distinction helps determine how a capability should be delivered and who should remain responsible for it.
• Abstracted capabilities and node pools: Explain why a capability can help an application yet remain a service when its resources are not provisioned or configured uniquely for that application.
  Why it matters: The example prevents a misleading classification based only on whether developers interact with a capability.
• Scaling delivery and responsibilities: Explain how growth changes release orchestration and why services should be considered for delegation before extensions.
  Why it matters: This sequencing aims to increase delivery velocity without sending developers back to ticket queues and manual processes.

Key terms: service, extension, Kubernetes API, developer experience, abstracted, Karpenter, node pools, release orchestration pipeline, organizational scaling, platform services

8.2.1 Which services or extensions should be a part of every platform from the start?

This section explains how to choose initial control plane services and extensions based on a platform’s needs, while identifying foundational capabilities and practices that support adoption and growth.

• Stakeholder outcome and API ownership: Explain how stakeholders can influence platform requirements without imposing solutions that create friction. Describe the two approaches the section recommends.
  Why it matters: This keeps organizational requirements compatible with a self-service platform experience.
• Realistic technology proof of concept: Justify testing a proposed service on the actual engineering platform infrastructure before committing to it.
  Why it matters: A realistic trial reveals operational difficulties that a selection checklist may miss.
• Operational observability and self-service: Describe the observability coverage and customer-facing capabilities the section calls for in an engineering platform.
  Why it matters: Teams need visibility into both their applications and the platform, without depending on operators for each observability change.
• Node autoscaling for resilience: Explain why node autoscaling belongs in the platform and when operators may need to supply it as a control plane service.
  Why it matters: The platform needs the capability even when its provider determines who manages it.
• Deployment methods and secrets support: Distinguish direct pipeline deployment using service account credentials from deployment through a service such as Argo or Flux, and explain the associated secrets-management need.
  Why it matters: The deployment choice determines both how teams reach the cluster and how application secrets must be supplied.
• Initial security service baseline: Explain why every cluster needs essential security services from the outset and how those guardrails should evolve.
  Why it matters: Early guardrails give adopters confidence without making security a lasting obstacle to delivery.

Key terms: stakeholders, outcomes, APIs, manual handoffs, software selection criteria, proof of concept, engineering platform infrastructure, operational observability, logs, metrics

8.3 Control plane services

This section proposes a simple starting point for deploying control plane services and extensions and explains why nonproduction environments should test production-like resilience.

• Separate initial release pipelines: Explain why the platform starts with basic release pipelines that deploy services and extensions separately.
  Why it matters: This explains why a deliberately simple deployment setup can still support later growth.
• Production-like nonproduction resilience: Justify testing high-availability and resilience configurations outside production, despite the cost of nonproduction environments.
  Why it matters: Production-like resilience testing makes releases more trustworthy and helps avoid costly developer disruption.

Key terms: services, extensions, release pipelines, domain boundary, alpha teams, high-availability (HA), resilience, nonproduction environments, production-like behavior

8.3.1 Services pipeline orchestration

This section outlines how to deploy pinned control-plane services to each cluster with a Kubernetes-focused pipeline, configure and scan their Helm charts, and test that the services work.

• Cluster services pipeline flow: Describe how the services pipeline deploys a pinned version of each service to a cluster with Helm and then runs integration tests. Explain that this pipeline uses the Kubernetes API rather than provisioning AWS-managed resources through the AWS API.
  Why it matters: It establishes what this pipeline releases and how it differs from the earlier infrastructure pipelines.
• Pipeline environment and stages: Explain how the pipeline obtains cluster access and how its deployment and repeat-test stages are organized. Describe the sandbox deployment on a main-branch change, the production deployment on a Git tag, and nightly reuse of the tests.
  Why it matters: Cluster credentials and reusable stages let deployments and scheduled checks run against the intended environment.
• Helm chart security and configuration: Explain why merely mirroring third-party Helm charts does not guarantee availability or meaningful security review, and describe the pipeline-based scanning and automated mirroring proposed instead. Describe how pinned versions and cluster-specific values files configure deployments.
  Why it matters: The proposed approach seeks useful security checks without turning chart adoption and upgrades into a slow ticket process.
• Service state and functional tests: Distinguish state checks from functional tests for deployed services. Give the section’s metrics-server example: use an application with a Horizontal Pod Autoscaler, generate load, and check whether its pods scale up.
  Why it matters: Testing both health and actual behavior catches services that appear to be running but are not usable.

Key terms: services pipeline, extensions pipeline, Helm, pinned version, Kubernetes API, set-environment, kubeconfig, pipeline service account, sandbox cluster, nightly job

8.4 Control plane extensions

Introduces common control plane extension categories and previews their treatment in the Epetech example.

8.4.1 Kubernetes storage classes

Explains why Epetech uses AWS-managed Kubernetes storage classes by default and when additional storage capabilities would count as control plane extensions.

• Storage class extension boundary: Explain why Epetech does not add storage classes as routine control plane extensions, and when a storage capability should be treated as one.
  Why it matters: This distinction keeps already-managed storage separate from capabilities the platform team would need to extend.

Key terms: storage classes, control plane extensions, Kubernetes API, Elastic Block Store, Elastic File System

8.4.2 Service mesh

This section explains how a service mesh adds application-independent traffic capabilities and walks through deploying and testing Istio, cert-manager, and external-dns as platform extensions.

• Service mesh proxy model: Explain how a service mesh uses managed proxies to provide communication capabilities without changing application code, and describe the tradeoff between dedicated and shared proxies.
  Why it matters: The proxy model explains how the platform can add traffic features without requiring every application team to build them.
• Mesh capability boundaries: Describe the kinds of capabilities the mesh can provide and justify the section's caution about bundling them tightly with the container network.
  Why it matters: Keeping capabilities separable helps the platform evolve without making upgrades or security decisions unnecessarily entangled.
• Initial ingress extension stack: Trace the distinct jobs of Istio, external-dns, and cert-manager in making an application reachable through a platform-managed HTTPS domain.
  Why it matters: Each extension solves a different part of the path from a domain name to an application over HTTPS.
• Extension deployment and permissions: Describe how the extensions pipeline prepares AWS permissions and deploys the initial extensions, including why its role test is limited.
  Why it matters: The extensions need working cloud permissions before they can manage DNS and certificates.
• Istio upgrade tradeoffs: Explain why the starting platform uses in-place Istio upgrades and what operational precautions or follow-up they require.
  Why it matters: A simple initial upgrade method still has consequences for application availability and sidecar versions.
• Default ingress resources and validation: Describe the default resources and tests that establish whether the newly deployed extensions can deliver HTTPS traffic to an application.
  Why it matters: An HTTPS request to a test application checks the combined result rather than merely whether extension pods are running.

Key terms: service mesh, sidecar proxy servers, control plane, Kubernetes Gateway API, CNI, domains of change, zero-trust networking, Istio ingressgateway, external-dns, Route53

8.4.3 Using operators for persistent data platform capabilities

This section explains how operators can make common persistent data resources available through the Kubernetes API, and why infrastructure starter kits do not offer the same platform-service model.

• Operator-backed persistent data resources: Explain how infrastructure operators let developers provision common persistent data resources through the Kubernetes API while platform engineers retain responsibility for their operational lifecycle.
  Why it matters: The API can simplify developers’ work without leaving them responsible for operating the underlying infrastructure.
• Operator adoption tradeoffs: Justify the section’s cautious approach to operators as an alternative to traditional infrastructure as code.
  Why it matters: The potential improvement in developer experience must be weighed against the platform team’s operational learning curve.
• Starter kits versus platform services: Distinguish an infrastructure starter kit from a platform capability delivered as a service interface.
  Why it matters: A copyable template can speed up provisioning without solving the continuing support problem that a platform service addresses.
• Starter kit value and ownership: Explain the lasting value and support cost of infrastructure starter kits, and why the authors recommend separate ownership for them.
  Why it matters: Recognizing the ongoing support work helps preserve the platform team’s capacity to build platform services.

Key terms: infrastructure operators, Kubernetes API, persistent data resources, Helm chart, operational lifecycle, operator pattern, traditional IaC, Terraform Operator, infrastructure starter kits, self-serve

8.5 Platform management APIs

The section explains how team-based authorization and lightweight management APIs can give developers self-service access to an engineering platform across its many systems.

• Team-based platform access: Explain why the platform should authorize developers through their teams and let teams manage onboarding and membership rather than requiring individual access requests for each system.
  Why it matters: Team-based access removes a recurring source of delay for developers.
• Authentication and membership authority: Distinguish authenticating a user from deciding their team-based authorization, and describe the section’s GitHub-team approach and its unresolved source-of-truth question.
  Why it matters: Signing in alone does not tell the platform which team resources a developer may use.
• Team onboarding configuration: Describe how a known list of platform customer teams could drive standard namespace creation, access bindings, gateways, and necessary membership synchronization.
  Why it matters: A customer-team list turns onboarding into repeatable platform configuration rather than manual setup.
• Management APIs and integration events: Explain why lightweight management APIs are proposed for integration gaps and how team events can support configuration maintenance, self-healing, and contributions from other technical teams.
  Why it matters: These APIs help preserve a unified access experience as tools change and more teams contribute to the platform.

Key terms: development team, authorization, team onboarding, team membership, authentication, source of truth, GitHub teams, OIDC integration, engineering platform customers, standard team namespaces

8.5.1 Managing teams and namespaces for early adopters

This section describes a temporary way to manage early-adopter teams and namespaces before the platform management API is ready.

• Interim team and namespace management: Explain how Epetech can configure a few early-adopter teams and standard namespaces before its management API is ready, and why a simple, non-self-serve approach is sufficient at the alpha stage.
  Why it matters: Early adopters can start deploying applications and giving feedback without waiting for the management API.

Key terms: early-adopter teams, standard namespaces, customer management pipeline, management API, alpha stage

8.6 Summary