We use Kafka's Streaming API and Avro a lot in the backend of nerd.vision. When it comes to stream processing, testing is - without a doubt - an important factor. As I am a big fan of Spock, it was no question to check how it can help us testing our stream processing applications.

What I was looking for is a simple way to test processing a whole sequence of messages with as little test code as possible. In our use case we process so called AuditTrace messages that are produced by various services in the backend. These messages carry information about what actions (e.g. a tracepoint has fired) were executed by which component or user, just to name a few. When processing these messages our stream application (aka audit-processor) builds up state and updates data kept in Freshsales using their REST API accordingly. To avoid having to use a dedicated database for persisting state data, the audit-processor makes use of KTables which is a really elegant way to do this.

Our Spock Specification therefore uses the following properties

together with the following code

to mock the the Freshsales stuff.

The setup method for our specification then configures the Test Topology for Kafka (yes, we haven't come up with a way to mock the schema registry yet ;-) )

so that we are ready to define an actual test:

In the specification above we load our test messages from a JSON file, convert them into Avro types and then produce each message into the consumer record factory which is then used as input to the test driver. Note the offset of 60001ms we use - this is because we use the suppress method in the audit-processor to ensure we don't run into rate limiting when communicating with Freshsales.
In the assertion block we finally test that the result of all messages being processed is the expected data - in our case a FreshContact with a property 'customField' that contains the expected keys and values.

Looking at the above specification one can see that the actual test code is short, focusing on business logic and easy to use in additional tests. Mission accomplished :-)

‍