5 Routing log events

This chapter covers

Sending log events to multiple outputs using copying
Using stdout, files, and slack plugins
Consuming log events, and the application of the copy mechanism
Routing log events using tags and labels
Handling of errors in Fluentd
Copying log events so they can be consumed by multiple outputs
Using filters to see what is happening
Using of inclusions to enable re-use of configurations, and support team collaboration
Injecting additional contextual information into log events to better understand the context of the log message
Using more core plugins null, exec, and secondary to gain more insights into what is happening

In the preceding chapters, we saw how to capture and store log events. But in all the examples routing was simply all events going to the same output. However, this can be far from ideal. As described in Chapter 1, we may want log events to go to different tools depending on the type of log event, it may be desirable to send a log event to multiple locations or none. In this chapter, we will therefore examine the different ways we can route or exclude individual events. In addition, we will look at some smaller features that can contribute to solving the challenges of routing; such as adding information into the log event to ensure the origin of the log event is not lost along the way.

5.1 Reaching multiple outputs by copying

5.1.1 Copy by reference or by value

5.1.2 Handling errors when copying

5.2 Configuration reuse and extension through inclusion

5.2.1 Place holding with null output

5.2.2 Scenario - Replace a place holder with a MongoDB output

5.3 Injecting context into log events

5.3.1 Extraction of values

5.4 Tag-based Routing

5.4.1 Using exec output plugin

5.4.2 Exercise - Adopting tag naming conventions

5.4.3 Exercise - applying dynamic tagging with extract

5.5 Tag plugins

5.6 Labels – taking tags to a new level

5.6.1 Using a stdout filter to see what is happening

5.6.2 Illustrating Label and Tag routing

5.6.3 Connecting Pipelines

5.6.4 Label Sequencing

5.6.5 Special Labels

5.6.6 Exercise - Using a common pipeline

5.7 Summary