Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

Build Validated for Release Candidate?

  • Validated by QA Lead
  • Released Live

Summary:

This is a checklist for the QA lead to run through the tests of the app and make sure they don’t miss any important steps

Test Started:

Current available HAs:

PR, Guam, MN, LA, AL

iOS - for General Tests

Device used: iPhone 7

HA used: AL

Android - for General Tests

Device used: Samsung Galaxy S7

HA used: MN

*There are significant differences between Android and iOS and the different versions of the GAEN api, some (non-exhaustive) are noted below:

  • iOS: Bluetooth and EN

  • Android 10 or below: Bluetooth, EN, and Location

  • Android 11: Bluetooth and EN

Verification General Test (only need one Health Authority per Operating System)

ONBOARDING / BLUETOOTH / LOCATION

Goal: a user can download the app and make it to the core features.

#

Instruction

Check (iOS)

Check (Android)

1

Install App

  • No Errors
  • No Errors

2

Proceed through onboarding wizard (5 screens)

  • Text displays and is readable
  • Text displays and is readable

3

Open "How we protect your privacy"

  • Text displays and is readable
  • Text displays and is readable

4

Try to Activate Exposure Notifications

  • Should prompt to open settings
    Click “No thanks” button
  • Should prompt to open settings
    Click “No thanks” button

6

Enable Notifications

  • Screen “App Setup Incomplete” is shown
  • Screen “App Setup Incomplete” is shown

7

Access Home Page

  • Exposure Detection is red - shows OFF
  • Exposure Detection is red - shows OFF

8

*Enable Location (*only necessary on older Android OS's)

  • “Location” shows ON

9

Verify Exposure Detection is ON

  • Card on Home screen shows as green and ON
  • All three fields show as green and ON

--

MANUALLY DETECT EXPOSURES

Goal: a user can manually prompt the app to check for exposures

#

Instruction

Check (iOS)

Check (Android)

1

Select the "Exposures" tab from the bottom menu

  • Button will select correct page
  • Button will select correct page

2

Click the "Check for exposures" button

  • "Success" banner shown, after "last 14 days" text is updated to reflect latest check
  • "Success" banner shown, after "last 14 days" text is updated to reflect latest check

--

EXPOSURE HISTORY

Goal: a user can access their exposure history populated with exposures

#

Instruction

Check (iOS)

Check (Android)

1

In Settings, Use the EN Debug menu to simulate 1 or more exposures

  • EN is generated & success message shows
  • EN is generated & success message shows

2

Navigate to Exposure History

  • Exposure History shows expected number of exposures
  • Exposure History shows expected number of exposures

3

Tap on an exposure

  • Relevant Next Steps info is shown
  • Relevant Next Steps info is shown

--

GAEN (Google Apple Exposure Notifications) Permissions

Goal: an onboarded user can enable required permissions as expected.

#

Instruction

Check (iOS)

Check (Android)

*

Disable Exposure Notifications (device OS), Enable

  • Exposure Notifications changes from red X to green check
  • Exposure Notifications changes from red X to green check

*

Disable Location (if applicable*) on device / Re-enable

  • Location changes from red X to green check

--

VERIFICATION CODE & KEY SUBMISSION

Goal: an onboarded user can submit a verification code as expected.

#

Instruction

Check (iOS)

Check (Android)

1

Generate verification code*
On Health Authority’s “Issue Code” page, input required data for positive results and click “Create Verification Code”

  • Code is provided from HA server
  • Code is provided from HA server

2

Access App, select Report Result area

  • Graphic / Text show
  • Graphic / Text show

3

Enter verification code

 

 

4

"Next" > "Share" > "I understand and consent"

  • No errors after selecting any of the buttons
  • No errors after selecting any of the buttons

*

Offline: disable devices network connectivity and execute above test

  • Error should appear after "Next" and prevent code submission
  • Error should appear after "Next" and prevent code submission

*

Invalid Code: Submit a non-valid verification code

  • Error should appear after "Next" and prevent code submission
  • Error should appear after "Next" and prevent code submission

*Code provided from verification server, if you do not have access and are an internal QA please contact Arthur Gibson

--

MANUALLY DELETE APP DATA

Goal: a user can manually delete their data stored by the app

#

Instruction

Check (iOS)

Check (Android)

1

Access the Settings page

  • Page shows with no issues
  • Page shows with no issues

2

Press "Delete My Data"

  • Page "Delete My Data" shows, text is shown with no issue
  • Page "Delete My Data" shows, text is shown with no issue

3

Press "Delete My Data" red banner

  • Pop-up confirmation is shown
  • Pop-up confirmation is shown

4

Press “Confirm” on pop-up

  • Intro onboarding page is shown - no previous data is shown in main pages
  • Intro onboarding page is shown - no previous data is shown in main pages

--

APP UPDATE

Goal: a user can update the app and verify that simulated exposure is still shown

#

Instruction

Check (iOS)

Check (Android)

1

Install build 1.0.x (older) from AppCenter

  • Older build installs without issue
  • Older build installs without issue

2

Navigate to debug menu and tap "simulate exposure"

  • exposure simulation occurs
  • exposure simulation occurs

3

Navigate to Exposure history screen and view exposure

  • Exposure is shown in Exposure History
  • Exposure is shown in Exposure History

4

Download Staging build 1.0.y (newer) from AppCenter

  • Newer build installs without issue
  • Newer build installs without issue

5

Return to updated app’s “Exposure History” Tab

  • Previous Exposure is still shown in Exposure History
  • Previous Exposure is still shown in Exposure History

--

GAEN EXPOSURE NOTIFICATION (END-TO-END VERIFICATION)

Goal: a user can receive an exposure notification and see it on the exposure history.

NOTE: Two (or more) phones running app w/ Exposure Notifications ON required.

For testing purposes, we will use an iOS and an Android device to run this test (provides a full end-to-end experience). The Android device will be device #1 (this is the one that a positive test case will be reported on) and the iOS device will be device #2 (the device that an exposure notification should appear on).

HA Used for End-To-End:

When generating a Verification Code: You need to make sure to use a Symptoms onset and Testing date that is 24 hours or less from the actual contact of the two devices we are testing with. Example: if it’s Dec. 11 and the phones have been in contact for the past 24 hours, I would use Dec 10 as my testing and Symptoms onset date.

Symptoms Onset Date used:

Testing Date Used:

Last Date of contact for testing Devices:

Unique identifier (NOT the short code) :

#

Instruction

Device #2 (iOS)

Device #1 (Android)

1

Let phone sit for AT LEAST 1 hour (app in background)

  • Debug menu > Show Local Diagnosis Keys has keys
  • Debug menu > Show Local Diagnosis Keys has keys

2

Move phone to within 6 ft of another testing phone (at least 15 minutes)

 

 

3

"Report positive test result" > "Start" > enter code > share keys

  • No errors

4

Wait (up to 24 hours)

  • Exposure Notification appears on second testing phone

5

(On 2nd phone) Open Exposure History

  • Possible exposure is listed, Next Steps button leads to relevant HA website (if applicable)

Interface General Check

General UI/UX

 iOS

Android

Goal: a user can navigate the app with any supported text or language settings

 

 

Large Font: execute any of the above tests with device's font set to maximum*

  • All screens are readable & navigable
  • All screens are readable & navigable

Translation: execute any of the above tests after selecting (and toggling) a non-English language

  • All screens are readable & translations make sense
  • All screens are readable & translations make sense

 

GAEN Permissions*

iOS

Android

Goal: an onboarded user can enable required permissions as expected.

 

 

Disable Exposure Notifications (device OS), Enable Proximity Tracing (in app)

  • EN Check on homescreen changes from red X to green check
  • EN Check on homescreen changes from red X to green check

Disable Location (if applicable) on device / Re-enable

  • Location changes from red X to green check

+3 Day Symptom Tracking Test

  • Test skipped for this build.

This test is not always necessary. We will work with the developers to determine which builds and which HAs would require this. Only do if the release effects exposure notifications, we will work on a better way to indicate this for future testing.

This is to verify that phones DO NOT receive an EN (Exposure Notification) while in the 14 day window of tracking IF the Exposure Reporter (person who tested positive) denoted that the onset of symptoms did not occur until 3 days after the interaction.

Example:

  • Person 1 (Who does not have symptoms at the time) is within six (6) feet of Person 2, and Person 3. Lets call this ‘Day A’

  • Three (3) days go by after the occurrence of Persons 1, 2, and 3 within six (6) feet and Person 1 then starts having symptoms. This is ‘Day B’

  • Within 14 days of ‘Day A’, Person 1 is tested and receives a positive test. This is ‘Day C

  • Person 1 reports to a tracker, when the tracker asks when the test happened, Person 1 denotes Day C, the tracker then asks when symptoms started, Person 1 denotes Day B, and the tracker then gives Person 1 their Verification Code.

  • Person 1 then inputs their Verification Code into the App, it is registered with the server and completes the transition of data
    --
    Because Day B is 3 (or more) days before Person 1’s symptoms appear, Persons 1 and 2 are NOT notified.

Steps for Testing:

Please make sure that phones have been wiped of any previous data.
- If on iOS make sure to access the App’s Settings tab, and click “Delete my data”, uninstall the app THEN make sure to have turned off Exposure Notifications in the Settings > Exposure Notifications page to purge any previous data.
-If on Android, while app is still installed, access Settings > Exposure Notifications > Exposure Checks > Delete exposure checks, THEN in the App’s Settings Tab, click “Delete my data” and uninstall the app.

  1. Access 3 phones (Android or iOS)

  2. Install the app, make sure they're on the same HA

  3. Set all phones to a date (hereto referred to as DATE A) - Make sure this is within the 14 day window, but also have the ability to give 3 days after for symptoms.
    [EX: if today is 11/18, make sure phones are set to a 'DATE A' of 11/9 (within 14 days, but room within that window to add 3 days for the first day symptoms are reported)]

  4. Allow phones to sit at least 1 hour (1.5 hours if possible)

  5. Change all phones to current date

  6. Access the HA's Test Verification Interface, login, choose correct realm

  7. Create a verification code, make sure to set:
    A) Symptom Onset to 3 days past DATE A (hereto referred to as DATE B) [Ex: if DATE A is 11/9, DATE B is 11/12]
    B) Testing Date is within 14 days of DATE A

  8. Select one of the phones to be the "Reporter", Input Verification Code generated

  9. Sequester “Reporter” phone out of Bluetooth distance, or turn the phone OFF)

  10. Allow adequate time for server to relay exposure notifications (up to 24 hours)

+3 Day Test Follow Up

  • Test Passed
  • Test Skipped
  • Test Failed

Notes:

  • No labels