Testing Checklist

Owned by Ben St John
Last updated: Feb 15, 2021
8 min read

This is a template that can be copied and duplicated for all following builds. Simply Plug in the information and check off the list as you go.

Build Validated for Release Candidate?

Validated by QA Lead
Released Live

Summary:

This is a checklist for the QA lead to run through the tests of the app and make sure they don’t miss any important steps

Test Started: Dec 8, 2020

Current available HAs:

PR, Guam, MN, LA

iOS -

Device used:

HA used:

Android - 

Device used:

HA used:

*There are significant differences between Android and iOS and the different versions of the GAEN api, some (non-exhaustive) are noted below:

  • iOS: Bluetooth and EN

  • Android 10 or below: Bluetooth, EN, and Location

  • Android 11: Bluetooth and EN

Verification General Test (only need one Health Authority per Operating System)

ONBOARDING / BLUETOOTH / LOCATION

Goal: a user can download the app and make it to the core features.

#

Instruction

Check (iOS)

Check (Android)

1

Install App

No Errors
No Errors

2

Proceed through onboarding wizard (5 screens)

Text displays and is readable
Text displays and is readable

3

Open "How we protect your privacy"

Text displays and is readable

4

Try to Activate Exposure Notifications

6

Enable Notifications

7

Access Home Page

8

*Enable Location (*only necessary on older Android OS's)

 

9

Verify Exposure Detection is ON

 

--

MANUALLY DETECT EXPOSURES

Goal: a user can manually prompt the app to check for exposures

#

Instruction

Check (iOS)

Check (Android)

1

Select the "Exposures" tab from the bottom menu

2

Click the "Check for exposures" button

--

EXPOSURE HISTORY

Goal: a user can access their exposure history populated with exposures

#

Instruction

Check (iOS)

Check (Android)

1

In Settings, Use the EN Debug menu to simulate 1 or more exposures

2

Navigate to Exposure History

3

Tap on an exposure

--

GAEN (Google Apple Exposure Notifications) Permissions

Goal: an onboarded user can enable required permissions as expected.

#

Instruction

Check (iOS)

Check (Android)

*

Disable Exposure Notifications (device OS), Enable

*

Disable Location (if applicable*) on device / Re-enable

 

--

VERIFICATION CODE & KEY SUBMISSION

Goal: an onboarded user can submit a verification code as expected.

#

Instruction

Check (iOS)

Check (Android)

1

Generate verification code*
On Health Authority’s “Issue Code” page, input required data for positive results and click “Create Verification Code”

2

Access App, select Report Result area

3

Enter verification code

 

 

4

"Next" > "Share" > "I understand and consent"

*

Offline: disable devices network connectivity and execute above test

*

Invalid Code: Submit a non-valid verification code

--

MANUALLY DELETE APP DATA

Goal: a user can manually delete their data stored by the app

#

Instruction

Check (iOS)

Check (Android)

1

Access the Settings page

2

Press "Delete My Data"

3

Press "Delete My Data" red banner

4

Press “Confirm” on pop-up

--

APP UPDATE

Goal: a user can update the app and verify that simulated exposure is still shown

#

Instruction

Check (iOS)

Check (Android)

1

Install build 1.0.x (older) from AppCenter

2

Navigate to debug menu and tap "simulate exposure"

3

Navigate to Exposure history screen and view exposure

4

Download Staging build 1.0.y (newer) from AppCenter

5

Return to updated app’s “Exposure History” Tab

--

GAEN EXPOSURE NOTIFICATION (END-TO-END VERIFICATION)

Goal: a user can receive an exposure notification and see it on the exposure history.

NOTE: Two (or more) phones running app w/ Exposure Notifications ON required.

#

Instruction

Device #2 (iOS)

Device #1 (Android)

1

Let phone sit for AT LEAST 1 hour (app in background)

2

Move phone to within 6 ft of another testing phone (at least 15 minutes)

 

 

3

"Report positive test result" > "Start" > enter code > share keys

 

4

Wait (up to 24 hours)

 

5

(On 2nd phone) Open Exposure History

 

 

Interface General Check

General UI/UX

 iOS

Android

Goal: a user can navigate the app with any supported text or language settings

 

 

Large Font: execute any of the above tests with device's font set to maximum*

Translation: execute any of the above tests after selecting (and toggling) a non-English language

 

GAEN Permissions*

iOS

Android

Goal: an onboarded user can enable required permissions as expected.

 

 

Disable Exposure Notifications (device OS), Enable Proximity Tracing (in app)

Disable Location (if applicable) on device / Re-enable

 

 

+3 Day Symptom Tracking Test

Example:

  • Person 1 (Who does not have symptoms at the time) is within six (6) feet of Person 2, and Person 3. Lets call this ‘Day A’

  • Three (3) days go by after the occurrence of Persons 1, 2, and 3 within six (6) feet and Person 1 then starts having symptoms. This is ‘Day B’

  • Within 14 days of ‘Day A’, Person 1 is tested and receives a positive test. This is ‘Day C

  • Person 1 reports to a tracker, when the tracker asks when the test happened, Person 1 denotes Day C, the tracker then asks when symptoms started, Person 1 denotes Day B, and the tracker then gives Person 1 their Verification Code.

  • Person 1 then inputs their Verification Code into the App, it is registered with the server and completes the transition of data
    --
    Because Day B is 3 (or more) days before Person 1’s symptoms appear, Persons 1 and 2 are NOT notified.

Steps for Testing:

Please make sure that phones have been wiped of any previous data.
- If on iOS make sure to access the App’s Settings tab, and click “Delete my data”, uninstall the app THEN make sure to have turned off Exposure Notifications in the Settings > Exposure Notifications page to purge any previous data.
-If on Android, while app is still installed, access Settings > Exposure Notifications > Exposure Checks > Delete exposure checks, THEN in the App’s Settings Tab, click “Delete my data” and uninstall the app.

  1. Access 3 phones (Android or iOS)

  2. Install the app, make sure they're on the same HA

  3. Set all phones to a date (hereto referred to as DATE A) - Make sure this is within the 14 day window, but also have the ability to give 3 days after for symptoms.
    [EX: if today is 11/18, make sure phones are set to a 'DATE A' of 11/9 (within 14 days, but room within that window to add 3 days for the first day symptoms are reported)]

  4. Allow phones to sit at least 1 hour (1.5 hours if possible)

  5. Change all phones to current date

  6. Access the HA's Test Verification Interface, login, choose correct realm

  7. Create a verification code, make sure to set:
    A) Symptom Onset to 3 days past DATE A (hereto referred to as DATE B) [Ex: if DATE A is 11/9, DATE B is 11/12]
    B) Testing Date is within 14 days of DATE A

  8. Select one of the phones to be the "Reporter", Input Verification Code generated

  9. Sequester “Reporter” phone out of Bluetooth distance, or turn the phone OFF)

  10. Allow adequate time for server to relay exposure notifications (up to 24 hours)

+3 Day Test Follow Up

+3 Day Test Follow Up

Notes: