Unit Testing in Kotlin is fun and tricky at the same time. We can benefit a lot from Kotlin’s powerful language features to write readable and concise unit tests. But in order to write idiomatic Kotlin test code in the first place, there is a certain test setup required. This post contains best practices and guidelines to write unit test code in Kotlin that is idiomatic, readable, concise and produces reasonable failure messages.

TL;DR

• Use JUnit5 and @TestInstance(Lifecycle.PER_CLASS) to avoid the need for static members, which are non-idiomatic and cumbersome in Kotlin.
• Test fixtures
  • Reuse one instance of the test class for every test methods (by using @TestInstance(Lifecycle.PER_CLASS))
  • Initialize the required objects in the constructor (init) or in a field declaration (apply() is helpful). This way, the fields can be immutable (val) and non-nullable.
  • Don’t use @BeforeAll. It forces us to use lateinit or nullable types.
• Instead of annotating every class with @TestInstance(Lifecycle.PER_CLASS) we can change the default lifecycle with the junit-platform.properties file.
• Put the names of the test methods in backticks and use @Nested inner classes to improve the readability and the structure of the test class.
• Mocks
  • open classes and methods explicitly or use interfaces to make them mockable.
  • Use Mockito-Kotlin or MockK to create mocks in a convenient and idiomatic way.
  • For a better performance, try to create mocks only once and reset them in a @BeforeEach.
• AssertJ is still the most powerful assertion library.
• Take advantage of data classes
  • Create a reference object and compare it directly with the actual object using an equality assertion.
  • Write helper methods with default arguments to easily create instances with a complex structure. Avoid using copy() for this purpose.
  • Use data classes to carry the test data (input and expected output) in a @ParameterizedTest.

Recap: What is Idiomatic Kotlin Code?

Let’s recap a few points about idiomatic Kotlin code:

• Immutability. We should use immutable references with val instead of var.
• Non-Nullability. We should favor non-nullable types (String) over nullable types (String?).
• No static access. It impedes proper object-oriented design and testability. Kotlin strongly encourages us to avoid static access by simply not providing an easy way to create static members.

But how can we transfer these best practices to our test code?

Avoid Static and Reuse the Test Class Instance

In JUnit4, a new instance of the test class is created for every test method. So the initial setup code (that is used by all test methods) must be static. Otherwise, the setup code would be re-executed again and again for each test method. In JUnit4, the solution is to make those members static. That’s ok for Java as it has a static keyword. Kotlin doesn’t have this direct mean - for good reasons because static access is an anti-pattern in general.

//JUnit4. Don't:
class MongoDAOTestJUnit4 {

    companion object {
        @JvmStatic
        private lateinit var mongo: KGenericContainer
        @JvmStatic
        private lateinit var mongoDAO: MongoDAO

        @BeforeClass
        @JvmStatic
        fun initialize() {
            mongo = KGenericContainer("mongo:3.4.3").apply {
                withExposedPorts(27017)
                start()
            }
            mongoDAO = MongoDAO(host = mongo.containerIpAddress, port = mongo.getMappedPort(27017))
        }
    }

    @Test
    fun foo() {
        // test mongoDAO
    }
}

Fortunately, JUnit5 provides the @TestInstance(Lifecycle.PER_CLASS) annotation. This way, a single instance of the test class is used for every method. Consequently, we can initialize the required objects once and assign them to normal fields of the test class. This happens only once because there is only one instance of the test class.

//Do:
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class MongoDAOTestJUnit5 {
    private val mongo = KGenericContainer("mongo:3.4.3").apply {
        withExposedPorts(27017)
        start()
    }
    private val mongoDAO = MongoDAO(host = mongo.containerIpAddress, port = mongo.getMappedPort(27017))

    @Test
    fun foo() {
        // test mongoDAO
    }
}

First, this approach is more concise. Second, it’s idiomatic Kotlin code as we are using immutable non-nullable val references and can get rid of the nasty lateinit. Please note, that Kotlin’s apply() is really handy here. It allows object initialization and configuration without a constructor. But using a constructor (init { }) is sometimes more appropriate if the initialization code is getting more complex.

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class MongoDAOTestJUnit5Constructor {
    private val mongo: KGenericContainer
    private val mongoDAO: MongoDAO

    init {
        mongo = KGenericContainer("mongo:3.4.3").apply {
            withExposedPorts(27017)
            start()
        }
        mongoDAO = MongoDAO(host = mongo.containerIpAddress, port = mongo.getMappedPort(27017))
    }
}

In fact, we don’t need JUnit5’s @BeforeAll (the equivalent of JUnit4’s @BeforeClass) in Kotlin and with @TestInstance(Lifecycle.PER_CLASS) anymore because we can utilize the means of object-oriented programming to initialize the test fixtures.

Side note: For me, the re-creation of a test class for each test method was a questionable approach anyway. It should avoid dependencies and side-effects between test methods. But it’s not a big deal to ensure independent test methods if the developer pays attention. For instance, we should not forget to reset or reinitialize fields in a @BeforeEach block and don’t (re-)assigned fields in general - which is not possible when we use val fields. ;-)

Change the Lifecycle Default for Every Test Class

Writing @TestInstance(TestInstance.Lifecycle.PER_CLASS) on every test class explicitly is cumbersome and easy to forget. Fortunately, we can set the default lifecycle for the whole project by creating the file src/test/resources/junit-platform.properties with the content:

junit.jupiter.testinstance.lifecycle.default = per_class

Use Backticks and @Nested Inner Classes

• Put the test method name in backticks. This allows spaces in the method name which highly improves the readability. This way, we don’t need an additional @DisplayName annotation.
• JUnit5’s @Nested is useful to group the tests methods. Reasonable groups can be certain types of tests (like InputIsXY, ErrorCases) or one group for each method under test (GetDesign and UpdateDesign).

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
internal class TagClientTest {
    @Test
    fun `basic tag list`() {}

    @Test
    fun `empty tag list`() {}

    @Test
    fun `empty tag translations`() {}

    @Nested
    inner class ErrorCases {
        @Test
        fun `server sends empty body`() {}

        @Test
        fun `server sends invalid json`() {}

        @Test
        fun `server sends 500`() {}

        @Test
        fun `timeout - server response takes too long`() {}

        @Test
        fun `not available at all - wrong url`() {}
    }
}

Handle Mocks

Final By Default

Classes and therefore methods are final by default in Kotlin. Unfortunately, some libraries like Mockito are relying on subclassing which fails in this cases. What are the solutions for this?

• Use interfaces
• open the class and methods explicitly for subclassing
• Enable the incubating feature of Mockito to mock final classes. For this, create a file with the name org.mockito.plugins.MockMaker in test/resources/mockito-extensions with the content mock-maker-inline.
• Use MockK instead of Mockito/Mockito-Kotlin. It supports mocking final classes by default.

Mockito-Kotlin

I recommend using Mockito-Kotlin as it provides a convenient and idiomatic API. For instance, it facilitates Kotlin’s reified types. So the type can be inferred and we don’t have to specify it explicitly.

//plain Mockito
val service = mock(TagService::class.java)
setClient(mock(Client::class.java))

//Mockito-Kotlin
val service: TagService = mock()
setClient(mock())

There are many other useful features to discover. Check out the documentation.

Update: Check out MockK as an alternative to Mockito-Kotlin. It provides mocking final classes by default.

Create Mocks Once

Recreating mocks before every test is slow and requires the usage of lateinit var.

//Don't
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class DesignControllerTest_RecreatingMocks {

    private lateinit var dao: DesignDAO
    private lateinit var mapper: DesignMapper
    private lateinit var controller: DesignController

    @BeforeEach
    fun init() {
        dao = mock()
        mapper = mock()
        controller = DesignController(dao, mapper)
    }

    // takes 1,5 s!
    @RepeatedTest(300)
    fun foo() {
        controller.doSomething()
    }
}

Instead, create the mock instance once and reset them before or after each test. It’s significantly faster (1,5 s vs 220 ms in the example) and allows using immutable fields with val.

// Do:
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class DesignControllerTest {

    private val dao: DesignDAO = mock()
    private val mapper: DesignMapper = mock()
    private val controller = DesignController(dao, mapper)

    @BeforeEach
    fun init() {
        reset(dao, mapper)
    }

    // takes 210 ms
    @RepeatedTest(300)
    fun foo() {
        controller.doSomething()
    }
}

Handle Classes with State

The presented create-once-approach for the test fixture and the classes under test only works if they don’t have any state or can be reset easily (like mocks). In other cases, re-creation before each test is inevitable.

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class DesignViewTest {

    private val dao: DesignDAO = mock()
    // the class under test has state
    private lateinit var view: DesignView

    @BeforeEach
    fun init() {
        reset(dao)
        view = DesignView(dao)
    }

    @Test
    fun changeButton() {
        assertThat(view.button.caption).isEqualTo("Hi")
        view.changeButton()
        assertThat(view.button.caption).isEqualTo("Guten Tag")
    }
}

AssertJ for Assertions

Even in times of dedicated Kotlin libraries I still stick to the powerful AssertJ for assertions. It provides a really huge amount of assertions and a nice fluent and type-safe API. There is an assertion for everything. It impresses me every day and I highly recommend it. I don’t see a serious reason to switch to a (potentially less powerful) Kotlin-native API just to safe same dots and parenthesis.

For the sake of completeness, I have to point to kotlintest. By using infix functions it offers a cleaner DSL for assertions (without the dots and parenthesis). However, it doesn’t contain as many assertions as AssertJ does and the auto-completion in my IDEA became noticeably slower. But check it out yourself. At the end, it’s a matter of taste.

Utilize Data Classes

Data Classes for Assertions

If possible don’t compare each property for your object with a dedicated assertion. This bloats your code and - even more important - leads to an unclear failure message.

//Don't
@Test
fun test() {
    val client = DesignClient()

    val actualDesign = client.requestDesign(id = 1)

    assertThat(actualDesign.id).isEqualTo(2) // ComparisonFailure
    assertThat(actualDesign.userId).isEqualTo(9)
    assertThat(actualDesign.name).isEqualTo("Cat")
    assertThat(actualDesign.dateCreated).isEqualTo(Instant.ofEpochSecond(1518278198))
}

This leads to poor failure messages:

org.junit.ComparisonFailure: expected:<[2]> but was:<[1]>
Expected :2
Actual   :1

Expected: 2. Actual: 1? What is the semantics of the numbers? Design id or User id? What is the context/the containing class? Hard to say.

Instead, create an instance of the data classes with the expected values and use it directly in a single equality assertion.

//Do
@Test
fun test() {
    val client = DesignClient()

    val actualDesign = client.requestDesign(id = 1)

    val expectedDesign = Design(id = 2, userId = 9, name = "Cat", dateCreated = Instant.ofEpochSecond(1518278198))
    assertThat(actualDesign).isEqualTo(expectedDesign)
}

data class Design(
    val id: Int,
    val userId: Int,
    val name: String,
    val dateCreated: Instant
)

This way, we get a nice and descriptive failure message:

org.junit.ComparisonFailure: expected:<Design(id=[2], userId=9, name=Cat...> but was:<Design(id=[1], userId=9, name=Cat...>
Expected :Design(id=2, userId=9, name=Cat, dateCreated=2018-02-10T15:56:38Z)
Actual   :Design(id=1, userId=9, name=Cat, dateCreated=2018-02-10T15:56:38Z)

We take advantage of Kotlin’s data classes. They implement equals() and toString() out of the box. So the equals check works and we get a really nice failure message. Moreover, by using named arguments, the code for creating the expected object becomes very readable.

We can take this approach even further and apply it to lists. Here, AssertJ’s powerful list assertions are shining.

//Do
@Test
fun test() {
    val client = DesignClient()

    val actualDesigns = client.getAllDesigns()

    assertThat(actualDesigns).containsExactly(
        Design(id = 1, userId = 9, name = "Cat", dateCreated = Instant.ofEpochSecond(1518278198)),
        Design(id = 2, userId = 4, name = "Dogggg", dateCreated = Instant.ofEpochSecond(1518279000))
    )
}

java.lang.AssertionError: 
Expecting:
  <[Design(id=1, userId=9, name=Cat, dateCreated=2018-02-10T15:56:38Z),
    Design(id=2, userId=4, name=Dog, dateCreated=2018-02-10T16:10:00Z)]>
to contain exactly (and in same order):
  <[Design(id=1, userId=9, name=Cat, dateCreated=2018-02-10T15:56:38Z),
    Design(id=2, userId=4, name=Dogggg, dateCreated=2018-02-10T16:10:00Z)]>
but some elements were not found:
  <[Design(id=2, userId=4, name=Dogggg, dateCreated=2018-02-10T16:10:00Z)]>
and others were not expected:
  <[Design(id=2, userId=4, name=Dog, dateCreated=2018-02-10T16:10:00Z)]>

How cool is that?

Use Helper Methods with Default Arguments to Ease Object Creation

In reality, data structures are complex and nested. Creating those objects again and again in the tests can be cumbersome. In those cases, it’s handy to write a utility function that simplifies the creation of the data objects. Kotlin’s default arguments are really nice here as they allow every test to set only the relevant properties and don’t have to care about the remaining ones.

fun createDesign(
    id: Int = 1,
    name: String = "Cat",
    date: Instant = Instant.ofEpochSecond(1518278198),
    tags: Map<Locale, List<Tag>> = mapOf(
        Locale.US to listOf(Tag(value = "$name in English")),
        Locale.GERMANY to listOf(Tag(value = "$name in German"))
    )
) = Design(
    id = id,
    userId = 9,
    name = name,
    fileName = name,
    dateCreated = date,
    dateModified = date,
    tags = tags
)

//usage:
val testDesign = createDesign()
val testDesign2 = createDesign(id = 1, name = "Fox")
val testDesign3 = createDesign(id = 1, name = "Fox", tags = mapOf())

• Don’t add default arguments to the data classes in the production code just to make your tests easier. If they are used only for the tests, they should be located in the test folder. So use helper methods like the one above and set the default arguments there.
• Don’t use copy() just to ease object creation. Extensive copy() usage is hard to read. Prefer the helper methods.
• The builder pattern is not required anymore.

Data Classes for Parameterized Tests

Data classes can also be used for parameterized tests. Due to the automatic toString() implementation, we get a readable test result output in IDEA and the build.

@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ParseTest {

    @ParameterizedTest
    @MethodSource("validTokenProvider")
    fun `parse valid tokens`(testData: ValidTokenTestData) {
        assertThat(parse(testData.value)).isEqualTo(testData.expectedToken)
    }

    private fun validTokenProvider() = Stream.of(
        ValidTokenTestData(value = "1511443755_2", expectedToken = Token(1511443755, "2"))
        , ValidTokenTestData(value = "1463997600_10273521", expectedToken = Token(1463997600, "10273521"))
        , ValidTokenTestData(value = "151144375_1", expectedToken = Token(151144375, "1"))
        , ValidTokenTestData(value = "151144375_id", expectedToken = Token(151144375, "id"))
        , ValidTokenTestData(value = "1511443755999_1", expectedToken = Token(1511443755999, "1"))
        , ValidTokenTestData(value = null, expectedToken = null)
    )
}

data class ValidTokenTestData(
    val value: String?,
    val expectedToken: Token?
)

Other Libraries

There are many Kotlin libraries setting out to ease testing with Kotlin. I personally don’t use them but you should definitely have a look at them. Maybe they convince you.

• KotlinTest
• Spek

Source

The source code can be found at GitHub.

Further Reading

• I highly recommend the book Kotlin in Action by Dmitry Jemerov and Svetlana Isakova