liveBook · Manning

Experimentation

As explained in Platform Engineering on Kubernetes

A/B testing is a technique that compares alternative versions of an application or feature by exposing different user groups to each version and measuring their outcomes.

A/B testing runs alternative versions of a feature in parallel, gives different user groups access to each, and collects outcome data. Unlike a canary release or blue/green deployment, its main question is which option works better for users, not simply whether a new release is safe to promote. section 8.1.3

Compare registration flows

Run two registration flows for the Conference application. Route groups of users to the alternatives, then compare the data to decide which flow takes less time or causes less frustration. The decision belongs to teams evaluating the business outcome, not only to developers managing a release. section 8.1.3

Choose the routing mechanism to match the experiment:

  • Separate Kubernetes deployments and services can host the alternatives, but application-level rules are still needed to decide which users reach each one. section 8.1.3
  • Knative Serving can route to revisions through tags or HTTP headers. Percentage-based splitting can run versions side by side, but it does not control where a user's subsequent requests go. section 8.2.2
traffic:
 - percent: 100                 #1
   revisionName: frontend-00001
 - latestRevision: true         #2
   tag: version110

This traffic configuration keeps the existing frontend revision at 100% of ordinary traffic while giving the latest revision a named tag. The tag provides a route to an alternative without shifting the default traffic split. section 8.2.2

The infrastructure makes both options reachable; the experiment still needs business and application-level rules for assigning users and evaluating results. Knative Serving reduces the manual work of managing multiple Kubernetes resources, but it does not choose the winning experience. section 8.1.3 section 8.5

How Platform Engineering on Kubernetes covers it

How deep it goes
Explains alternative implementations and routing choices, including a Knative traffic configuration, but does not specify measurement or cohort-assignment procedures.
Examples
Uses page layouts, registration flows, and tagged frontend revisions as examples of alternatives and routes.
What it assumes
Assumes familiarity with Kubernetes deployments, services, and kubectl; introduces their use for A/B testing rather than teaching them from zero.
What you can do after
Apply it via Knative Serving to expose alternative revisions, while defining user-routing rules and collecting data for a business decision.
How it connects
Connects A/B testing to feature flags, canary releases, blue/green deployments, Kubernetes services, and Knative Serving.

Also explained in

Compare how 16 books explain experimentation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Platform Engineering on Kubernetes. The section links point to the parts of the book it draws on.