Analytics telephone occurs when analytical results pass between people or teams and gradually acquire conclusions beyond the original work. Distillation can remove sample details, limitations, and precise language, turning association into an unsupported claim of causation. section 2.2
The failure often begins after technically sound analysis. Analysts must help stakeholders interpret the research design, methodology, findings, and appropriate next steps rather than allowing excitement or colloquial language to dilute the result. section 2.2 section 2.2.1
Reduce analytics telephone by making the scope of interpretation explicit:
- Document relevant methodology, comparison periods, benchmarks, sample details, and limitations. section 2.2.1
- Use precise language such as “no relationship was detected” instead of implying that an intervention caused an outcome. section 2.2 section 2.2.1
- Create stakeholder resources and addenda that preserve the intended interpretation as findings move between teams. section 2.2 section 2.2.1
Help Center search changes
A team analyzed Help Center visits and chat volume before and after search changes. Colleagues later inferred that optimizing search caused lower chat volume and recommended more work. Adding 90-day and annual comparisons, plus clearer language about the absence of a detected relationship, helped constrain that interpretation. section 2.2.1
Figure 2.3 Analytics telephone can diffuse the quality of your insights.
Use the figure to visualize how insight quality diffuses as results pass through successive summaries. section 2.2
