Recently I started using Kotlin and Micronaut, and I wanted to just quickly discuss some of the problems I have encountered while developing. Firstly I just want to say that Kotlin and Micronaut are both fantastic and anyone looking to start a new project should seriously put both of these at the top of the list of candidates. Before we get into some of the problems I had, lets just take a quick look at KotlinTest.

Setup & Tear down

Like most good test frameworks there are the basic setup and tear down methods available, however KotlinTest provides much more. Offering listeners for nearly every stage of the tests execution. This allows you to implement extensions to the test framework as well as ensuring the state of the test environment for each test execution.

Testing Styles

In KotlinTest there are many test styles available to you, allowing you to write the tests how you like. There is a great example of each style available on the KotlinTest github page, in the styles section.

My experiences

I have come from a world of Spock (Vulcan), where we use behaviour driven tests (given, when, then). So I naturally started with the BehaviorSpec styles of tests with KotlinTest. This seemed quite similar for the most part, but as the tests became more complex I started to experience failures that I did not understand.

Verification Failed

Verification failed: call 1 of 1: IAPIKeys(#3).authenticate(eq(token))

The first error I got was a verification failure, this one really confused me as I was clearly calling the mock correctly. I spent ages debugging it again and again every time convinced I was calling the mock correctly. I even tested by removing the mock method definition and getting a MockKException instead.

no answer found for: IAPIKeys(#1).authenticate(eq(token))

This clearly meant the mock was being called, but why was the verification failure happening? After some random trail and error I found a solution that worked, although I was not happy with it.

So what was the problem?

As I came from the Spock world where you would do the setup, calls and validation in each stage of the tests, e.g. like this:

def "test call to IAPIKey"() {
    given: "a mock IAPIKey":
        IAPIKey apikey =mock(IAPIKey)
    when: "we call the filter"
        filter.auth("token")
    then: "we expect the mock to be called"
        1 * apikey.authenticate("token")
}

So I followed this pattern in KotlinTest which resulted in the following test:

given("a path request with auth") {
   ServerRequestContext.set(HttpRequest.GET<any>("http://localhost:8888/this/v1/path"))</any>
   `when`("We call the auth provider") {
       val validateToken = authService.validateToken("token")
       then("api keys is called with the token") {
           assertNotNull(validateToken)
           val mockApi = getMock(apiKey)
            verify { mockApi.authenticate("token") }
       }
   }
}

I was happy with this test as it seemed to make sense, but would not pass due to the verification failure. After spending some time trying to resolve the problem I decided to rework the test to just see where things started to fail. That is when I found that I have to put all the code in the 'then' block or the test just wont work as expected. The result was this:

given("a path request with auth") {
   // set the request for the test
   ServerRequestContext.set(HttpRequest.GET<any>("http://localhost:8888/this/v1/path"))</any>
   `when`("We call the auth provider") {
       then("api keys is called with the token") {
             val validateToken = authService.validateToken("token")
            assertNotNull(validateToken)
           val mockApi = getMock(apiKey)
            verify { mockApi.authenticate("token") }
       }
   }
}

Only a minor change and now the test passes as expected, I am not able to find anything that specifically says this is needed. Although it is touched on in the style guide in the section about BehavorSpec it does have a comment '// test code' although this is quite subtle if in fact all the test needs to be in there.

tl; dr;

After initially using KotlinTest to test a new Micronaut project, I found that when using the BehaviorSpec tests, all the test code needs to be inside the 'then' block of the test to get the tests to be reliable.