Chapter 8. Big data: using graphs when there’s too much data
This chapter covers
- The limits of graph visualization, both web and desktop
- Abstracting individual data endpoints
- Filtering the chart by type or other metrics
- Grouping nodes and links
In chapter 6, we talked about data volumes and explained how it’s not always realistic to draw a chart showing every data endpoint in your data set on the screen at once. This is important to keep in mind when working with large data sets, and some of the techniques we discussed in that chapter, such as allowing the user to navigate the chart, can be useful when working with thousands or even tens of thousands of nodes. One of my company’s clients is a credit card–processing company that handles 24,000 transactions every second. There’s no way to even think about trying to draw each transaction in a graph visualization. I think the big data buzzword gets thrown around needlessly sometimes, but that is truly big data. So should we even bother with graph visualization in those cases? Maybe. We’ll start small and work our way up.