Firebase Test Lab for Android – Testing Real Apps on Real Devices

Nevercode’s improved integration with Firebase Test Lab enables you to make it a breeze to run tests on real devices for your native Android app as part of your CI/CD process. This article will explain the benefits and how to set up the integration.

Benefits of using Firebase Test Lab for Android

As we’ve discussed in this blog before, continuous testing is the new frontier of CI/CD which enables teams to deliver higher quality solutions to customers at speed.

Nevercode already supports running tests on emulators and simulators which also have distinct advantages. Emulators simulate both software and hardware, you can use them for early testing during development, and you can find free and open-source solutions which means it’s cost-effective to use. Simulators are fast and easy to set up to find interesting behaviors in the app. However, simulators may give you false positives since apps run differently on hardware which can’t be tested with simulators. Emulators tend to be slow and unstable and even though they simulate hardware, using them won’t answer questions about such quality aspects as battery drain and conflicts with other apps.

Running tests on real devices and doing it in the cloud means you can exercise your app in an environment close to real life (minus weather conditions), which means getting relevant feedback on your app’s behavior. Considering the fragmentation of Android phone market, it’s no less important to make sure the device pool is sufficient and provides required test coverage of devices.

Firebase Test lab for Android enables you to run tests on real devices hosted at a Google data center. You can control the configuration of the devices and also run Espresso and UI Automator 2.0 instrumentation tests. Using Robo test helps to capture logs, annotated screenshots, and it also creates a video of those screenshots helping you to track the user operations performed on the device.

Setting up Firebase Test Lab integration in Nevercode

There are two main steps when setting up Firebase Test Lab integration: creating an account in Firebase and configuring your settings, and configuring Firebase Test Lab in Nevercode. Note that the supported test types are instrumentation and robo tests. Also, note that you would be billed to your Firebase account separately for the test runs.

First, you need to create a Firebase project in the Firebase console. You will need to create a service account and select your role as Editor. Then create a new private key with key type JSON and save the generated key to your computer. Don’t forget to enable Google Cloud Testing API and Cloud Tool Results API in the Google Developers console API Library page. Type the API names in the search box and enable the found APIs.


Now that you’ve configured your service account and APIs, it’s time to finalize configuration in Nevercode.

Navigate to the Test page in your project configuration settings.

Select Firebase as the emulation type. Test target is set by default based on the target selected for your project in Build configuration. You can define additional parameters in the Test parameters field to limit the scope of tests. Then select the emulators and/or real devices that should be used and provide the command line arguments you need. Finally, provide the cloud credentials JSON file that you created previously. Click Save and you’re done with the setup!

After running the tests, it’s time to check what happened and analyze the results. To do that, navigate to the Artefacts tab of your build and download the zip file that stores video recordings and logcat.

In addition, you can view your test results in Firebase Test Lab.

More details about setup and configuration are available in the Firebase Test Lab integration for Android guide.