Domain knowledge is specialized business or engineering knowledge about a system, its application area, and its constraints. It supplies informed hypotheses and rules before measurement begins, but it does not reliably predict whether a change will improve a business metric. Engineered systems combine software, hardware, people, external conditions, and wider social effects; their interactions can defeat otherwise plausible ideas. section 1.3.1
Use domain knowledge as a cheap decision aid: identify feasible designs, narrow parameter ranges, and prioritize ideas worth testing. Treat it as a supplement to experimentation rather than evidence of impact. Experiments remain the more accurate way to measure the business effect of a system change, while domain knowledge, offline quality, and simulation trade accuracy for lower resource cost. section 1.3 section 1.4
The main safeguard is to turn domain assumptions into explicit constraints. A constraint should rule out parameter values because they violate how the system can operate, not merely because they seem unlikely to succeed. Restricting the search space this way prevents wasted measurements and avoids fitting an optimization method to invalid behavior. section 4.1.1
Constraining a trading threshold
A proprietary trading strategy computes a signal from market data and buys when the signal exceeds a threshold or sells when it falls below the negative threshold. Domain knowledge says the threshold must be nonnegative. With a negative threshold, some signal values meet both the buy and sell conditions, so the strategy has no unambiguous action. That operational constraint narrows the range before response-surface measurements are chosen. section 4.1.1
Figure 4.4 The strategy cannot function with threshold < 0. The horizonal axis represents signal value. (a) threshold > 0. The region of signal values where a BUY (resp. SELL) order is issued is far to the right (resp. left). In between, the strategy does not issue an order. (b) threshold < 0. The region of signal values where a BUY would be issued overlaps the region where a SELL would be issued. It is unclear which order should be issued in the center (overlap) region where threshold < signal < -threshold. (Note: -threshold > 0 since threshold is negative.)
Compare valid positive thresholds with the overlapping buy and sell regions created by a negative threshold. section 4.1.1
Apply domain knowledge with disciplined limits:
- State the business or engineering assumption in a form that can constrain a parameter range or design choice. section 4.1.1
- Distinguish operational impossibility from a prediction that an idea will improve a metric. section 1.3.1 section 4.1.1
- Measure the resulting change experimentally before claiming business impact. section 1.3 section 1.4
