In a soft-margin support vector machine, the penalty parameter C sets the cost of margin violations relative to the preference for a wide margin. It controls how strongly the fitted boundary responds to the training observations. section 8.3.3 section 10.2.7
The objective combines a term favoring a wider margin with a penalty on total slack. C scales that penalty. section 8.3.3
Changing C shifts the balance:
- A larger C makes violations more costly, favoring closer fit to training observations and potentially a narrower margin. section 8.3.3
- A smaller C tolerates more violations to preserve a wider margin; too small a value can lead to underfitting. section 8.3.3
C interacts with kernel parameters: for an RBF kernel, large C combined with large γ can produce a highly responsive boundary. Compare candidate settings with cross-validation rather than training accuracy alone. section 8.6.3 section 8.3.5



