This chapter covers:
- Discuss how Kafka Connect can help you start a data streaming journey.
- Designing a real-world Kafka project.
- Which data format to use for our project.
- Existing issues (like data silos) that impact usage of data.
- When data transformation should take place.
In our previous chapter, we started to see how you can work with Kafka from the command line and from a Java client. Now, we are going to expand on those first concepts and look at designing various solutions with Kafka. We will discuss some questions to look at while you start to design a strategy for our example project we introduce below. As we begin to develop our solutions, keep in mind that like most projects, we might make minor changes along the way and are just looking for a place to jump in and start developing.
After reading this chapter, we will be on our way to solve some real-world use case scenarios while having produced a design to start our further exploration of Kafka in the rest of this book. Let’s start on our new journey!