liveBook · Manning

Self-report measure

As explained in Effective Data Analysis

Self-report data is information that people provide about their own behaviors, experiences, attitudes, or characteristics.

A self-report measure is information a person provides about their own perceptions, attitudes, beliefs, experiences, or behavior. Surveys, interviews, questionnaires, and feedback forms are common ways to collect it; user-entered preferences and settings also count. Self-report is especially useful when the question concerns something a behavioral record cannot directly reveal, such as satisfaction or perceived support. section 6.2.1 section 6.4

Choose a measure to fit the decision. Product teams may have interaction logs, while marketing or user research teams may rely on surveys and interviews. Neither source answers every question: a survey can capture what users think, whereas a behavioral measure records an observed or logged action. Combining the two can provide a fuller picture, provided each source measures the intended concept. section 6.2.1 section 6.2.3

When constructing a questionnaire, check the choices that can change an answer:

  • Use precise wording. Replace ambiguous questions with ones that identify the experience you want to understand; avoid language that suggests a desired answer. section 6.2.2
  • Ask about one concept at a time. A question combining interface usability and customer support makes a single response hard to interpret. section 6.2.2
  • Choose the response format deliberately. Rating scales and small wording changes can affect results; a balanced scale with a moderate number of options, usually five or seven, can improve response quality. section 6.2.2 section 6.4

Recommendation intent is not a referral

The NPS question offers a quick way to ask about a person’s perceived likelihood of recommending a product. Its score does not establish that the person actually referred anyone. Responses on its 11-point scale can vary for reasons unrelated to sentiment, and changes in wording can substantially alter results. If actual referrals matter to a stakeholder’s decision, assess whether behavioral records can measure them. section 6.2.1

Table 6.2 Advantages and disadvantages of the self-reported NPS question
Advantages
Disadvantages
Data is easily stored in a spreadsheet.
Answers on the 11-point scale will differ between respondents independent of their sentiment based on immeasurable personal factors.
Assesses personal beliefs about the likelihood of recommending a product or service.
Changes to the question’s wording (e.g., company name, company type, part of a business’s offering) may drastically affect results.
Quickly asked online using a survey tool or in person as a quick survey. Often built into marketing and email campaign software.
Measures respondents’ perception of their likelihood to refer, not their actual referral behaviors.

The NPS comparison separates what a quick self-report question can assess from what it cannot establish about actual behavior. section 6.2.1

Treat responses as measurements to evaluate, not automatic facts about an underlying concept. Question wording and response format can introduce error or bias. Check reliability—whether measurement is consistent—and validity—whether it assesses the intended concept. When pairing a survey with warehouse data, also check how the behavioral data was collected and whether the sources can be connected appropriately. section 6.2.2 section 6.2.3 section 6.4

Evaluating workplace well-being

To assess a wellness program, select or develop questions about work-related stress, supervisor support, and workplace satisfaction. Explain the response scale and its possible biases, then identify behavioral measures that might relate to engagement or stress. After collecting responses, assess the questionnaires’ reliability and validity and consider how the two kinds of measures can be linked. section 6.2.4 section 6.3.3

How Effective Data Analysis covers it

How deep it goes
Defines collection routes and gives questionnaire-design guidance, then calls for reliability and validity checks without providing a full validation procedure.
Examples
Uses the NPS question and a workplace-wellness assessment involving stress, supervisor support, and satisfaction.
What it assumes
Introduces self-report and behavioral measures in plain terms while assuming an analyst can frame a measurement question.
What you can do after
Practice/perform questionnaire selection and wording, compare responses with behavioral measures, and evaluate whether measures fit the intended concept.
How it connects
Relates self-report to behavioral measures, response scales, measurement bias, reliability, and validity.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Effective Data Analysis. The section links point to the parts of the book it draws on.