Validity asks whether a measure accurately captures the concept or process it is meant to represent. A metric can be collected consistently yet still miss what stakeholders need to know. Check reliability, then examine validity before using results to support decisions. section 6.3.2 section 6.4
Three checks address different questions:
- Content or face validity: Does the measure cover the relevant facets of the concept? Review its definition against existing evidence and discuss it with subject matter experts, often including stakeholders. section 6.3.2
- Construct validity: Does the measure assess the intended theoretical construct? section 6.4
- Criterion validity: Is the measure associated with other measures of similar constructs? section 6.4
Measuring job satisfaction
A job-satisfaction questionnaire should reflect more than a vague sense of whether work is going well. Relevant facets might include work tasks, environment, manager, colleagues, and compensation. Reviewing research and checking the definition with people familiar with the workplace helps assess whether the questions cover the concept. section 6.3.2
Table 7.2 Many metrics are colloquially discussed as concepts and thus require stakeholder collaboration to create specific operational definitions.
|
High-level concept
|
Specific metric
|
|---|---|
|
The engineering team is productive
|
Percentage of tasks completed on time
|
|
Customers are supported in their needs
|
Percentage of customer problems and questions resolved on the first contact (first call resolution, or FCR)
|
|
Creating an easy-to-use application
|
Median time to complete the setup flow for new users
|
Compare each broad organizational goal with its specific metric. The translation makes a concept measurable, but it also creates a validity question: does that particular metric represent what stakeholders mean by the goal? section 7.2.1
Assess validity during metric design, not only after results arrive. Agree on a clear operational definition with stakeholders, and choose checks suited to the measure: a self-report questionnaire and a behavioral measure may need different validation steps. section 7.2.1 section 6.3.3
