Let’s walk through the code. Imports are the same. To make it a little easier, the SparkSession ’s instance is a private member, initialized in the start() method. The rest of the code is isolated in three methods:
Note that with Spark v3.0 preview 2, the start() from StreamingQuery now throws a time-out exception, which needs to be managed. The code in the repository behaves accordingly in the appropriate branches.