Everything that glitters is not UI testing

Test development is nowadays a regular practice to ensure the quality of the software. Not only QA engineers but also software engineers are involved in adding tests to their developments, with the aim of avoiding errors and regressive situations, as well as taking advantage of continuous integration systems for their execution.

It is well known that there are multiple kind of tests, and each development team/QA can take care of certain types of tests (it is something to determine depending on the skills within the team), but it always took my attention that the conceptual line that separates specific types of tests is very thin, and it is not always well distinguished.

What is a UI test

UI tests:

  • Only and exclusively test the UI.
  • Test how the UI reacts to different configurations of the underlying logic.
  • Test how the UI reacts to atomic user events.

In order to abstract the UI from the rest of the logic, correct architecture and design are a must. Following the SOLID principes is the basis, not only to perform a good development, but also to have a good testability.

It is common to find some confusion in this regard. Not everything the UI uses to test is a UI test. If we are testing any kind of logic not related to UI using UI events and assertions, maybe we are not doing something right. The test may not be well focused or our code may not be as testable as it should be. At this point we will have to rethink what and how we can do the tests to be the most optimal for us.

What is not a UI test

Acceptance tests (sometimes also called platform, end-to-end, or even end-tests) are also widely used to test complete scenarios, and as such, they usually use the UI as entry point. This implies that we can even use the same frameworks to develop tests in some cases, but we must not forget that this does not make them UI tests.

Let’s see an example that shows it, using one of the most typical features: the login. We have a class that collects the data from UI and the underlying logic checks and returns the result. Here we have the code (we will use the syntax of Espresso and Mockito):

fun testLoginCorrect(){

    `when`(authenticator.checkCredentials(anyString(), anyString())).thenReturn()

    //User events to fill UI components
    onView(withId(R.id.loginTextView)).perform(replaceText("username"))
    onView(withId(R.id.passwordTextView)).perform(replaceText("password"))

    //Submit the credentials to the authenticator entity
    onView(withId(R.id.buttonSubmit)).perform(click())

    //Assert the final status
    onView(withId(R.id.statusTextView)).check(matches(withText("Correct Credentials")))

}

and the code in case of acceptance test:

fun testLoginCorrect(){

    //User events to fill UI components
    onView(withId(R.id.loginTextView)).perform(replaceText("username"))
    onView(withId(R.id.passwordTextView)).perform(replaceText("password"))

    //Submit the credentials to the authenticator entity
    onView(withId(R.id.buttonSubmit)).perform(click())

    //Assert the final status
    onView(withId(R.id.statusTextView)).check(matches(withText("Correct Credentials")))

}

pretty similar, aren’t they? really, they are very different tests. In the case of the UI test we are including a fake response (thanks to Mockito), which assures that the logic that will verify the credentials in production is not involved in such test. This logic will have its own tests, typically unitary and completely separated from these tests and from any other component or layer with which it could be related. In the case of the acceptance test, the same logic that will validate the credentials in production will be validated in the test.

The difference, as told, is the scope of the test. The abstraction allows the first case to ensure that the statements focus on the UI, while in the second, a failure in the underlying logic would fail the test independently of the expected behavior of the UI. But we read the code and they are practically the same, which is why the distinction is not always clear.

Some ideas to start

To mention some useful frameworks to develop UI tests for mobile apps: KIF, Earlgrey (iOS), Espresso, Robotium (Android). If we are interested in acceptance tests for mobile, calabash is nice to mention. Do you already know any of them? Which one is your favorite?

In Solid GEAR we work in different projects developing tests of different kinds. In the case of ownCloud, my colleague David González and I developed tests to cover the needs of the Android application, both unitary and UI.

Leave a Comment

Responsable » Solidgear.
Finalidad » Gestionar los comentarios.
Legitimación » Tu consentimiento.
Destinatarios » Los datos que me facilitas estarán ubicados en los servidores SolidgearGroup dentro de la UE.
Derechos » Podrás ejercer tus derechos, entre otros, a acceder, rectificar, limitar y suprimir tus datos.

By completing the form you agree to the Privacy Policy

¿Necesitas una estimación?

Calcula ahora