Local response normalization (LRN) scales each neuron’s activation relative to neighboring neurons at the same spatial location across adjacent feature maps. By encouraging competition, it was intended to suppress uniformly strong activations and emphasize selective responses. section 2.5.2
AlexNet placed LRN after its first two convolutional layers. Its gains to generalization were modest, and it was later displaced by more effective methods, notably batch normalization. AlexNet’s results did not critically depend on it. section 2.5.2
LRN also reflects AlexNet’s biologically inspired appeal to lateral inhibition, a parallel that helped legitimize neural-network designs at the time. section 2.5.1
