Differential evolution

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

Differential evolution is a population-based evolutionary optimization algorithm that generates candidate solutions from scaled differences between existing solutions.

Differential evolution (DE) is a population-based evolutionary search method for optimizing vector-valued candidates. Rather than producing offspring, it keeps long-running agents and proposes replacements by adding a scaled difference between two randomly selected agents to a third. A proposal replaces its target only when its fitness is better. section 4.5.1 section 5.6.1

For each target agent, select agents a, b, and c; form a candidate component as a + F(b − c); evaluate it; and retain it if it improves fitness. A crossover rate controls which components are changed, while the scale factor F controls the difference contribution. This component-wise update makes DE useful for continuous, discontinuous, and difficult function approximation. section 4.5.1 section 5.6.1

Figure 4.11 The DE workflow for producing new agents

The workflow visualizes the three-agent differential proposal and fitness-based replacement that drive a DE generation. section 4.5.1

A practical DE loop in DEAP uses:

• A fixed-size population of equal-length numeric vectors, with three agents selected at random for every target agent. section 4.5.1 section 5.6.1
• Hyperparameters for dimensionality, crossover rate, scaling factor, population size, generations, and optional time limit. section 4.5.1
• A fitness evaluation followed by greedy replacement: keep the proposed vector only when it is fitter than its target. section 4.5.1

Function approximation and hyperparameter search

DEAP notebooks apply DE to polynomial, absolute-value, and step-function approximation, then reuse the same differential update to optimize four model hyperparameters. DE is presented as especially useful on discontinuous functions, where evolutionary strategies may be constrained by gradual global approximation; results still depend on the problem and settings. section 4.5.1 section 4.5.2 section 5.6.1

DE can also optimize neural-network weights, but it is not automatically the best choice: on the showcased circles and moons datasets, a simpler genetic algorithm generally performs best and DE is less optimal. Compare methods and tune their parameters rather than assuming a universal winner. section 6.3 section 6.3.1

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Works through the three-agent update, component crossover, fitness replacement, DEAP setup, and configurable hyperparameters without formal derivation.

Examples

Uses polynomial, absolute-value, and step functions; four-hyperparameter search; and circles and moons neural-network datasets.

What you can do after

Apply it via DEAP to evolve vector candidates for continuous or discontinuous function approximation and hyperparameter search.

How it connects

Connects differential evolution with evolutionary computation, particle swarm optimization, genetic algorithms, evolutionary strategies, backpropagation, and hyperparameter optimization.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsDifferential evolution is a population-based optimization algorithm that creates trial solutions by combining vector differences among candidate solutions.

Compare how 2 books explain differential evolution →

Related concepts

• evolution strategycontrasts with · An evolution strategy is a population-based evolutionary optimization method that searches through stochastic variation, selection, and adaptation of strategy parameters.
• hyperparameter tuninguses · Hyperparameter tuning is the process of selecting or adjusting configurable model or procedure settings and evaluating them against a chosen objective.
• particle swarm optimizationcontrasts with · Particle swarm optimization is a population-based optimization method in which candidate solutions adjust their positions using individual and collective search information.
• populationuses · A population is a defined group or complete set of people, units, observations, or candidate solutions considered together in a particular context.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.