Treatment group

As explained in Machine Learning System Design

The group in an experiment that receives the new condition or intervention being evaluated.

A treatment group receives the new system or intervention being evaluated, while a control group continues using the existing version. Splitting entities this way isolates the change’s effect and limits exposure if the system is broken. section 12.2

Use a treatment group to:

• Vary the affected fraction while protecting most entities from an unproven change. section 12.2
• Compare key metrics with a control group and account for external factors that could otherwise create false conclusions. section 12.2
• Report group sizes, observations, revenue, confidence intervals, and recommended next steps. section 12.4.1

Supermegaretail experiment

A retail experiment reports the number of shops in the treatment and control groups, total checks, and revenue, then compares primary and auxiliary metrics using confidence intervals and a Welch’s t-test. section 12.4.1

Figure 12.6 A/B testing split stage

A split-stage diagram helps visualize how entities are assigned to control and treatment groups before measurement begins. section 12.2

How Machine Learning System Design covers it

How deep it goes

Defines assignment and comparison, then extends into metric reporting and statistical testing without detailing allocation algorithms.

Examples

Uses an A/B split diagram and a retail experiment reporting shops, checks, revenue, and group-level metrics.

What it assumes

Assumes an existing system, measurable outcomes, and a comparable control group for the new condition.

What you can do after

Evaluate an ML change by comparing treatment outcomes with a control group while monitoring exposure and external influences.

How it connects

Connects treatment groups to A/B testing, control groups, metrics, confidence intervals, and hypothesis testing.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  First-Party Data ActivationThe experimentally assigned group that receives the marketing intervention being evaluated.

Compare how 2 books explain treatment group →

Related concepts

• control groupcontrasts with · A control group is a set of experimental units assigned to a baseline or comparison condition against which an intervention is evaluated.
• experimentationused by · Experimentation is the deliberate testing of an idea, intervention, or alternative by observing outcomes to gather evidence or compare effects.
• language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.