Firebase Test Lab integration for Android

Nevercode integrates with Firebase Test Lab for native Android which provides a cloud-based infrastructure for testing Android apps. The key benefit is that you can efficiently test your app across various devices (see the list here) and device configurations, and you can also review relevant artefacts, such as screenshots and videos, from your Firebase console.

Two test types are supported: instrumentation and robo.

🚧

Firebase Test Lab billing

When you're using Firebase Test Lab, you will pay for it separately based on your chosen plan.

This guide covers:

🚧

Getting the setup right

Note that you must complete the Firebase Test Lab setup before proceeding with configuring the integration in Nevercode.

Setting up Firebase Test Lab

Note that you don't need to set up a local gcloud SDK to be able to use Firebase Test Lab.

Creating Firebase project

If you don't have a Firebase project yet:

  1. Go to Firebase console
  2. Click Create project. You need to have ownership or edit permissions in your project.

Creating a service account

To create a service account:

  1. Navigate to the Manage resources page on Google cloud.
  2. Select the project that your created in Firebase Test Lab.
  3. In the left menu select Service accounts and create a service account.
  4. Select the role Editor.
  5. Select Furnish a new private key and choose JSON key type.
  6. Click Create and save the generated private key to the disk.

Creating Google service account

Enabling required APIs

After logging in to Google Cloud console:

  1. Enable the Google Cloud Testing API and Cloud Tool Results API in the Google Developers Console API Library page.
  2. Type the API names into the search box at the top of the console.
  3. Click Enable API on the overview page for that API.

Configuring Firebase Test Lab in Nevercode

If you have completed all of the steps above, you can proceed with configuring the integration in Nevercode.
Also, you need to have Instrumented Unit Tests in your Android project, otherwise the Instrumentation tests option won't be available.

Navigate to the Test page in your project settings:

  1. Select Enabled in the Instrumentation tests section.
  2. Select Firebase as the emulation type.
  3. Test target is set by default based on the target selected for your project in Build configuration.
  4. Test parameters are empty by default and you can define additional parameters to limit the scope of tests.
  5. Provide the necessary command line arguments. See Gcloud Firebase test run command line arguments for the default arguments we use and how to define more.
  6. Select emulators to be used and/or real devices.
  7. Provide the gcloud credentials JSON file that you created and saved to your hard drive.
  8. Click Save to finish the setup.

Gcloud Firebase test run command line arguments

If you leave the cmd arguments text box empty, then your test run will be executed with the following default parameters:

gcloud firebase test android run --app=<path_to_apk> --device <devices_selected_in_dropdown_menu> --test=<path_to_test_apk> --timeout=30m --type=instrumentation --quiet

Some examples of cmd arguments to use:

  • --results-bucket=<bucket_name> -- provide bucket name as cmd argument to collect all test results for your project in one bucket. You need to have paid Google Cloud storage available to define the bucket name.
  • --type=robo-- will run robo test type.

Note that you cannot override --results-dir argument because we require a predictable path where test results are stored. The value of --results-dir as a cmd argument will be ignored if set to anything but default.

Read more about Firebase test run here.

Analyzing test results

After the build is finished, you can view test results both in Nevercode UI and in the Firebase console or in a Google Cloud storage bucket in your project.

  1. Open your build in Nevercode.
  2. Navigate to the Tests tab.
    You can find the test result artefacts, such as logcat and video recordings, in the build view under the Artefacts tab, they are packed into one zip archive and also available separately.

Test results available in the artefacts tab

Here's an example of test results in Firebase Test Lab:

Updated 2 years ago

Firebase Test Lab integration for Android


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.