chapter nine

9 Creating custom plugins

 

This chapter covers:

  • Developing custom plugins using Redis as an example source and target for log events.
  • Defining the benefits that having source and target plugins for Redis can bring to Fluentd,
  • Using the Fluentd provided utilities to accelerate the plugin development process,
  • Acting upon the Fluentd plugin lifecycle and the different functions that need to be implemented for each stage,
  • Creating documentation, packaging, and testing of custom plugins.

At various points in the book, we have referred to Fluentd’s support for the development of plugins beyond those from the core product. The extensibility has led to a strong ecosystem of 3rd party plugins to make it easy to capture log events from many different sources and send log events to a great many different systems and data stores. We have also talked about how custom plugins could be an option for connecting to and monitoring esoteric and legacy solutions when things cannot easily or efficiently be achieved with the existing plugins.

In this chapter, we’re going to walk through the process of creating an input and output plugin that makes use of Redis’ list capability. We will take a closer look at Redis and the rationale behind its’ use in a moment.

9.1      Plugin Source code

9.2      What is Redis and Why build a plugin with the Redis List capability?

9.2.1      Redis List over RedisTimeSeries

9.3      Illustrating our objective using Redis CLI

9.4      Preparing for Development

9.5      Plugin Frameworks

9.5.1      Creating the skeleton plugin

9.5.2      Plugin Lifecycle

9.6      Implementing the plugin core

9.6.1      How configuration attributes work

9.6.2      Starting up and shutting down

9.6.3      Getting the plugin to work with our Fluentd installation.

9.6.4      Putting Additional Configuration Validation Into Action

9.6.5      Implementing the Redis output logic

9.6.6      Putting the Testing of synchronous output into action

9.7      Implementing the Redis Input plugin

9.7.1      Testing input and output plugin execution

9.8      Extending output with buffering

9.8.1      Improving our scenario by putting maintainability into action.

9.9      Unit testing

9.10  Putting the development of unit tests into action

9.10.1   solution for Putting the development of unit tests into action

9.11  Package and deployment

9.11.1   Documentation

9.13  Summary