foreword
The art of translating signals from applications and system services to insights on performance and system health is a difficult task, especially when the data comes from different sources and in different formats. Although the industry is trying to evolve and create standards to solve this problem for the long term, the short-term result is that we have to deal with several protocols and data structures to enable end users to perform meaningful analysis. In parallel, data volume is a constant challenge for companies as they see year-over-year data growth. The growth in data volume directly affects user experience. The more data there is to process, the slower the analysis gets.
When I started Fluent Bit in early 2015, little did I know that this lightweight agent, created for Embedded Linux at that time, would rule the logging world in what we now call cloud-native environments. Its ability to adapt to different protocols, pluggable architecture, and continuous focus for almost 10 years on performance (low memory, low CPU, and high throughput) has positioned it as the default solution for cloud providers such as Amazon, Google, Microsoft, and Oracle.