Chapter 1. A new paradigm for Big Data
This chapter covers
- Typical problems encountered when scaling a traditional database
- Why NoSQL is not a panacea
- Thinking about Big Data systems from first principles
- Landscape of Big Data tools
- Introducing SuperWebAnalytics.com
In the past decade the amount of data being created has skyrocketed. More than 30,000 gigabytes of data are generated every second, and the rate of data creation is only accelerating.
The data we deal with is diverse. Users create content like blog posts, tweets, social network interactions, and photos. Servers continuously log messages about what they’re doing. Scientists create detailed measurements of the world around us. The internet, the ultimate source of data, is almost incomprehensibly large.
This astonishing growth in data has profoundly affected businesses. Traditional database systems, such as relational databases, have been pushed to the limit. In an increasing number of cases these systems are breaking under the pressures of “Big Data.” Traditional systems, and the data management techniques associated with them, have failed to scale to Big Data.